Open webinar "Development of highly loaded systems for PHP"
Good evening! On the eve of the start of the course “Backend-developer on PHP” we traditionally held an open lesson. They talked about high-load systems, scaling, architecture. Considered in detail HighLoad, as well as the main approaches and tactics in the development of high-load systems.
The following occupation goals were set:
The teacher is Igor Sakhankov , the developer on Booking.com.
A couple of hundred years ago, the Swedish king Gustaf II Adolf wanted to build the world's largest ship. He invited Dutch shipbuilder Henrik Hübertsson and gave him his instructions. The master never built anything of the kind, but he had a lot of experience in building small ships. It would seem that there is nothing difficult - take a drawing of a small ship and increase it in the required number of times. No sooner said than done. And now, after a few years of work, a remarkable and grandiose structure appeared:
All anything, but during the launching of this ship did not last half an hour, immediately went to the bottom. It was the sailing warship "Vasa", and it happened in 1628.
Now imagine that you were asked to make a birdhouse or kennel for a dog. Of course, most of us will cope with this task. And if asked to make a skyscraper?
All these examples suggest that the more complex the system, the more it has various interdependencies, implementation nuances, pitfalls. And in order to successfully cope with the design and creation of complex systems, just there are architectural patterns.
In academic terms, information system architecture is a concept that defines the model, structure, functions performed and the interconnection of IP components.
However, let's go back from building ships and houses to programming. See, we always have requirements that come to us from business. And architecture helps to convert them into technical requirements. Based on the obtained technical requirements, you can easily decompose the system, write the dependencies between the modules, and proceed to direct implementation.
Thus, the architecture:
In itself, the concept of high load is very relative, and there is no fixed definition. However, it is safe to say that this moment comes when your current infrastructure ceases to cope with the load. And that means it's time to do the scaling. It happens:
Everyone knows such a thing as a queue ( FIFO ). This is a way to organize data processing, which allows you to handle conflict requirements by streamlining the process according to the principle of “first come, first go”. The queue can be used as a means of load balancing, decoupling the client and server, and much more.
Advantages:
Disadvantages:
Now let's talk about what you can do with the repository . Storage refers to any storage system, for example, relational databases. So, we have the following solutions for storage problems:
1. Replication.
One of the known problems with repositories is related to the situation when we have a lot of readings and few records. This may be a news site - everyone reads news, but only admins add new materials. So, replication is just about when our storage has rested on reading, and we have delays in the operation of the system. It is important to note here that replication solves the problem of reading, but not writing. Moreover, it somewhat impairs the ability of databases to write, because it has to distribute portions of its data to replicas, as is noticeable in the picture above. In the simplest case, when setting up replication, we have one Master - the main database to which we write, and the slaves are copies of the database (replicas from which all data reads come).
2. Partitioning . Suppose we have a large table, but we use only some data. We can break it apart. For example, we take our data and put them in different tablets depending on the month. The result is small tables instead of one large and, as a result, faster samples for a particular month. This optimizes the query execution time.
3. Vertical sharding . In the case of partitioning, it was about one server and one database. In the case of sharding, we have several machines and several bases. We take all our data, we break them into logical groups, and each logical group is stored in a separate database.
The method is easily implemented and well scales the load. But it also has disadvantages, which are described in more detail in the video.
4. Horizontal sharding . Here, every shard lies on a separate machine.
Recipe
The following theses will help you create a scalable system:
Colleagues, the retelling came out rather concise, so it's better to watch the webinar completely. Also, do not miss the Open Day of the Backend Developer for PHP course.
The following occupation goals were set:
- discuss what systems architecture is and why it is needed;
- discuss what High Load is;
- consider architectural tactics;
- discuss the course.
The teacher is Igor Sakhankov , the developer on Booking.com.
Architecture
A couple of hundred years ago, the Swedish king Gustaf II Adolf wanted to build the world's largest ship. He invited Dutch shipbuilder Henrik Hübertsson and gave him his instructions. The master never built anything of the kind, but he had a lot of experience in building small ships. It would seem that there is nothing difficult - take a drawing of a small ship and increase it in the required number of times. No sooner said than done. And now, after a few years of work, a remarkable and grandiose structure appeared:
All anything, but during the launching of this ship did not last half an hour, immediately went to the bottom. It was the sailing warship "Vasa", and it happened in 1628.
Now imagine that you were asked to make a birdhouse or kennel for a dog. Of course, most of us will cope with this task. And if asked to make a skyscraper?
All these examples suggest that the more complex the system, the more it has various interdependencies, implementation nuances, pitfalls. And in order to successfully cope with the design and creation of complex systems, just there are architectural patterns.
In academic terms, information system architecture is a concept that defines the model, structure, functions performed and the interconnection of IP components.
However, let's go back from building ships and houses to programming. See, we always have requirements that come to us from business. And architecture helps to convert them into technical requirements. Based on the obtained technical requirements, you can easily decompose the system, write the dependencies between the modules, and proceed to direct implementation.
Thus, the architecture:
- helps to ensure non-functional system requirements;
- is a means of communication between stakeholders;
- optimizes costs, reduces the cost of large systems;
- allows to reuse architectural patterns, take into account previous experience, do not repeat past mistakes
High load
In itself, the concept of high load is very relative, and there is no fixed definition. However, it is safe to say that this moment comes when your current infrastructure ceases to cope with the load. And that means it's time to do the scaling. It happens:
- vertical. The essence is simple: if the iron does not cope, we buy new, super powerful, productive and expensive;
- horizontal. We have one server that can not cope. We buy a second server with the same characteristics, then a third, and so on. At the output, we have a lot of relatively simple machines.
Architectural Tactics
Everyone knows such a thing as a queue ( FIFO ). This is a way to organize data processing, which allows you to handle conflict requirements by streamlining the process according to the principle of “first come, first go”. The queue can be used as a means of load balancing, decoupling the client and server, and much more.
Advantages:
- weak client and server connectivity;
- the status of the task is known;
- configuration on the fly;
- load balancing.
Disadvantages:
- additional entity in the system;
- more complex task processing;
- loss / creation of copies of messages.
Now let's talk about what you can do with the repository . Storage refers to any storage system, for example, relational databases. So, we have the following solutions for storage problems:
1. Replication.
One of the known problems with repositories is related to the situation when we have a lot of readings and few records. This may be a news site - everyone reads news, but only admins add new materials. So, replication is just about when our storage has rested on reading, and we have delays in the operation of the system. It is important to note here that replication solves the problem of reading, but not writing. Moreover, it somewhat impairs the ability of databases to write, because it has to distribute portions of its data to replicas, as is noticeable in the picture above. In the simplest case, when setting up replication, we have one Master - the main database to which we write, and the slaves are copies of the database (replicas from which all data reads come).
2. Partitioning . Suppose we have a large table, but we use only some data. We can break it apart. For example, we take our data and put them in different tablets depending on the month. The result is small tables instead of one large and, as a result, faster samples for a particular month. This optimizes the query execution time.
CREATE TABLE my_table (id INT, amount DECIMAL(7,2), some_date DATE) ENGINE=INNODB PARTITION BY HASH( MONTH(some_date) ) PARTITIONS 6; 3. Vertical sharding . In the case of partitioning, it was about one server and one database. In the case of sharding, we have several machines and several bases. We take all our data, we break them into logical groups, and each logical group is stored in a separate database.
$users = new PDO("pgsql:db_name=my_db;host=192.168.1.1", "user", "password"); $photos = new PDO("pgsql:db_name=my_db;host=192.168.1.2", "user", "password"); $users->prepare('SELECT * FROM users ...'); $photos->prepare('SELECT * FROM photos ...'); The method is easily implemented and well scales the load. But it also has disadvantages, which are described in more detail in the video.
4. Horizontal sharding . Here, every shard lies on a separate machine.
Recipe
The following theses will help you create a scalable system:
- simplify at the level of requirements!
- don't complicate the decision
- try to be stateless,
- cache,
- monitoring + alert,
- be competent.
Colleagues, the retelling came out rather concise, so it's better to watch the webinar completely. Also, do not miss the Open Day of the Backend Developer for PHP course.
Source: https://habr.com/ru/post/440334/