There is a highly loaded site, just affably loaded.

And there is a rather weak server for it.

The task is simple:

It is necessary, like before starting the main functional code, to check how much the server and MySQL are loaded

(First of all it is better to check MySQL)

The result is allowed to be displayed as a percentage or else

Here's how to do this in PHP or PERL, but PCP is desirable?

The logic is simple, if we assume the server is 85% loaded, then we throw out the statics

The python module is installed on the server, I don’t shave it very much in it, but if there is something on the topic on it and you can run it, write

  • For starters, what OS? - In general, the task is much more difficult than it seems at first glance. And the point is not the conclusion of which utility to parse (IMHO if you take one, then in * nix vmstat is the best of them), but in understanding what the server load is . Perhaps later I will return to the discussion of this issue. - avp
  • CentosOS Occupancy is when the database crashes and the tables crash. I need to avoid it. The base is written a message.
  • @dgfhgjljhjjd, in this case, measuring the percentage of CPU and memory (etc.) will not help you. It is necessary to track the connection to the database server. I do not know, in your case, these are sockets, FIFO or shared memory segments (you need to look somewhere in the manuals). These resources should be monitored. It may be worth looking at the files in / proc for the MySQL server (read man 5 proc). In general, the tables spoil - it is strange. With normal transaction organization, this should not happen even when the power is turned off. - avp

4 answers 4

And why actually write performance tests in php? You can make a simple benchmark literally in a minute right in the terminal window by writing a simple script for bash:

while :; do ps waux | grep mysql| grep -v grep >> mysql_benchmark.log; sleep 3; done 

Here an infinite loop starts, which calls the ps command with options, grep filter the necessary process from the list and redirect the result to a file. It’s not hard to guess that ps run every 3 seconds until you press CTRL + C.

After running the above command, you give the server a load (with pens or another script). After the end of the work you stop the script.

It's time to analyze the resulting log file. The most interesting columns for us are the third (% CPU - how many percent of the processor mysql consumed), the fifth (VSZ - how much virtual memory our experimental person took) and the sixth (RSS - how many real memory pages the process ate). The description of all columns needs to be looked in man ps .

 $ cat mysql_benchmark.log | awk '{print $3, $5, $6}' 0.0 1979852 32376 0.0 1979852 32376 0.0 1979852 32376 и ещё много циферок... 

After the filter CPU became the first, VSZ - the second, and RSS - the third column. As you can see, the mysql process is asleep and eats little memory. Large VSZs can often be scored on 64-bit boxes, since each process has its own address space and is virtual.

The main thing is to look at the CPU and RSS. If the CPU is close to 100% during the test, it will not hurt to buy the processor more powerfully, if RSS mysql is large and close to the total memory on the machine, the system will soon start to push unnecessary processes into the swap, and then kill the fattest processes. At the same time, mysql is the first candidate to kill, by strange default. This is solved conceived by the purchase of memory strip.

If mysql uses too little memory and is slow, then most likely you need to tweak a number of settings in mysql.conf for the storange engines used. Here this report can be useful in this question.

It still does not prevent me from trying to measure the amount of I / O on the hard disk and the network interface, but this is not so simple, and that is another story. An indirect sign of the people: if the brakes are observed - most likely, there is an active I / O, and most likely on the hard disk. For normal operation of the entire application, it is necessary that the amount of hot data does not exceed the amount of memory allocated for mysql.

I would also add that before the test, it is desirable to let the whole system " warm up " for a few minutes - just use your application without measurements, so that everyone can read the most frequently used data from the disk into RAM.

A similar simple technique has been used more than once for simple benchmarks of various applications: mysql, apache, server-side Python-processes. It works even in Windows :) But there are used batch file and some other utilities.

  • beautiful answer, thank you - thunder
  • @dred, the answer is good, but it looks like the author has a different problem: The base is written a message - Many connections are chopped off. As a result, the tables get corrupted by this he wrote in one of the comments. - avp
  • Well, I wrote as a general methodology, as an answer to the original question. Now I’m just delving into the mysql benchmarks at work, and that’s not enough. - dred

There is mtop , you can try to parse its output, to determine the download of MySQL.

Or parsit answer ordinary top.

 [root@swift ~]$ top -b -n 1 -u mysql top - 08:57:15 up 33 days, 22:35, 2 users, load average: 0.21, 0.31, 0.20 Tasks: 110 total, 1 running, 109 sleeping, 0 stopped, 0 zombie Cpu(s): 14.4% us, 1.0% sy, 0.0% ni, 83.7% id, 0.3% wa, 0.1% hi, 0.6% si Mem: 6106276k total, 3953372k used, 2152904k free, 376748k buffers Swap: 4192956k total, 124k used, 4192832k free, 1574960k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4046 mysql 16 0 170m 42m 5092 S 0.0 0.7 1300:11 mysqld 

Well, everything in the crown and all that ..

  • @Anton Shevtsov, and how do you think this data should be interpreted to calculate the loading threshold that the author is interested in? - avp
  • empirically, how else))% CPU +% MEM + load average + spherical horse in vacuum - Anton Shevtsov
  • Here in the horse (in general, the coefficient for it (and even non-constant)) is the whole problem. - avp 1:21

CONTINUED

In general, I am doing such requests

 system('cat /proc/cpuinfo'); system('cat /proc/meminfo'); system('free'); $load = sys_getloadavg(); foreach($load as $k => $v) { echo $k.' ---> '.$v; } 

and get the result

 processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Xeon(R) CPU E5345 @ 2.33GHz stepping : 7 cpu MHz : 2327.558 cache size : 4096 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu de tsc msr pae cx8 apic sep cmov pat clflush acpi mmx fxsr sse sse2 ss ht nx constant_tsc up pni ssse3 bogomips : 4663.96 clflush size : 64 power management: MemTotal: 786628 kB MemFree: 9972 kB Buffers: 14276 kB Cached: 639356 kB SwapCached: 0 kB Active: 617804 kB Inactive: 117144 kB HighTotal: 44800 kB HighFree: 180 kB LowTotal: 741828 kB LowFree: 9792 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 24336 kB Writeback: 0 kB AnonPages: 81320 kB Mapped: 16844 kB Slab: 18412 kB SReclaimable: 12924 kB SUnreclaim: 5488 kB PageTables: 0 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 393312 kB Committed_AS: 402556 kB VmallocTotal: 114680 kB VmallocUsed: 3040 kB VmallocChunk: 111272 kB total used free shared buffers cached Mem: 786628 775976 10652 0 14292 639400 -/+ buffers/cache: 122284 664344 Swap: 0 0 0 0 ---> 9.47 1 ---> 7.96 2 ---> 5.68 

How can you take advantage of this?

  • one
    You can admire, understand what your system. But about mysql nothing can be skzat. To analyze the total system load, you have 3 real numbers, how to analyze them, you can read on the wiki: en.wikipedia.org/wiki/Load_(computing) . For example, 5.68 means that in the last 15 minutes it was loaded by 468% relative to the average load (i.e., 4.68, we waited on average for the process, we could not wait for your processor) - dred

for skli it is possible to use

 show processlist; show status; 
  • yes but how from PHP to fulfill these requests? there you need to log in and then only execute them using what I wrote above does not roll - dgfhgjljhjjd
  • mysql_query same ... if there is any problem, then the request will return an error, which you will analyze ... - thunder
  • one
    although it is better to think about updating either at least about load-balancing - thunder