There is a working site written in Python (Django), and the database is MySQL.
Since the developers didn’t work hard on writing optimized code, the site was very heavy. And with attendance of ten thousand per day, at peak times, the server does not cope with the load and throws out those who did not have time to serve.
Here is the physical server system:
1. 2 cores at 1.8GHz
2. 2GB RAM

This is what the server is now deployed to:
mysql + nginx + fastcgi

So the question is : I would like to hear the opinions of experienced, experienced, or simply knowledgeable, on which you can still deploy this site, so that was as fast as possible, resource-intensive? Or is this bundle the fastest? Maybe someone did a speed comparison.
And it is also possible someone in the know how to optimize the server so that the giant number of queries in the database does not lie down the server, can some setting?
Thank.

    6 answers 6

    Basically, the bundle is fast enough. But you, most likely, simply have not strongly optimized code and / or SQL queries, do not use caching, etc. I deployed approximately on your configuration on a bunch of Apache + mod_wsgi + Nginx (for statics), and there was enough power for such a load. Look towards the MySQL query caching settings. Configuration parameters of the type (quote):

    > query_cache_size — размер кеша запросов. > query_cache_limit — размер максимальной выборки, хранимой в кеше; > query_cache_min_res_unit — минимальный размер блока, хранимого в кеше; > query_cache_wlock_invalidate - определяет будут ли данные браться из кеша, если таблица, к которым они относятся заблокирована на чтение.
    • memcached is on the site, everything goes through it. However, I will check the configuration parameters (thanks). But you correctly assumed that there are a lot of complex SQL queries, the server actually cannot cope with it, according to monitoring the server, MySQL almost always takes more than 60% of the processor, it also eats memory, although the python process still eats more memory. - trec

    Replacing fastcgi with uwsgi can give a slight boost . This, of course, will not save from the hundreds of requests that ORM jungi creates.

      Finish on a quick site. Add caching of the content sent to user requests in the redis database. Most likely the same pages Pts. and very good lot. Perhaps this root will solve your problem.

      • Plus It is necessary to treat the cause, not the effect. New iron is good, but as a temporary measure. - ling
      • one
        All the guys advise the right approaches, but they are "long-term". One of the answers has good advice: find an optimization specialist. Rewriting and transferring systems and increasing capacity may not save. PS: Redis is not a purchase of iron, it is free. Many so save their projects, about half a year. All that slows down, stick in there: page templates, frequently used directories (not changeable, for example: for lists). - org
      • Yes, apparently need to dig here. Thank. - trec

      From MySql I recommend to refuse in favor of PostgreSql. It’s easier to get over the 100k problem and scale when needed.

      Here is the current performance ideal:
      Python - PostgreSql - Django - uwsgi - nginx

      And remember that iron is cheaper code. :-)

      • In simple queries, MySql seems to be faster PostgreSql. Judging by the fact that the DBMS is loaded, the ORM jung is used, so PostgreSql is not a silver bullet here. - rnd_d
      • There was an attempt to rewrite the database on PostgreSql and engine, but did not calculate with the scale of the project and everything remained as it is. - trec

      You do not need to bother with bundles now, but focus on optimizing the DBMS and database. Find someone who really understands this.

      • That is, I can regard this answer, that whatever, was not, a bunch, it almost will not affect the performance. - trec
      • I mean, the matter can be not only in MySQL. Perhaps, database optimization is performed too rarely, or it is not performed at all. Before you drastically change something, first try optimizing the queries, the database structure and the server. - Avalon
      • The structure is very hard, everything is connected with the models, and there is such a mess. That's why I decided to ask for advice about the bundle, I will try to make friends with the server or calm down requests. - trec

      For one of my loaded projects I use tornado + nginx the solution itself is here. Withstands more than 20K per day on the same hardware as in the described configuration,

      • Interesting. But it seems to me that all the same it is a jumble, Tornado + wsgi. But in general, you need to try. Thanks for the detailed link. - trec