Methodically went through the links from the first page of Google on the request "tornado chat" and noticed that in all examples the connections are written to an array (list, set ... not the point): someone declares a list variable at the beginning of the file, someone then in the application class, but no one writes them to the database.
Is this normal and in production, is it still kept in a common array, and does this array go around in a loop every time looking for the right client? Or did I stumble upon very educational materials and in these applications it is worth doing something else?

  • Well, it is technically impossible to write connections to the database :) And I don’t know about normality, but depending on the task, you can also create a dictionary instead of an array, for example; I do this in my bikes and have not noticed anything bad yet - andreymal
  • @andreymal, technically in python there is a pickle serializer, and sqlalchemy has a PickleType type field, but when I tried to use them with an instance of the class in which the connection lives (well, somehow I imagine it to myself), I got Can't pickle < type 'function'>: attribute lookup builtin. function failed upd although, probably, this means “impossible” :) - fobedep
  • one
    Regardless of pickle (and of Python in general), all connections are closed when the operating system finishes the process, so even if you write to the database, for example, identifiers of sockets (it is not difficult), you still can’t be used after restarting the application, and without it you can store the connection in the database simply does not make any sense and the array functionality is quite enough (but, perhaps, there are some special types for this that I don’t know about, so I don’t write the answer) - andreymal
  • I personally thought about using the database, because I would like to be able to send messages to the client from different parts of the application. Here I thought that to choose from the database somehow ... or something more correct than importing an array from one module to another. In this case, does it make sense, or is it an extra fuss with the base, which will not improve anything? - fobedep
  • 3
    If different parts of the application are different processes, then for this it is necessary to organize interprocess communication, so that one process works with connections, and other processes send messages to this process. You can write your bike on sockets for this, you can use ready-made solutions like RabbitMQ and Redis. And the base here does not help. (Although you can send messages through a table in the database, but it is very crooked) - andreymal

1 answer 1

  1. For a couple of thousands of simultaneously connected subscribers (chat participants) pass an array list, etc. quite enough. In small projects, the attendance is more than enough.

  2. For tens of thousands of simultaneous connections, you need to complicate the algorithm, connect Redis + LUA (in radish there are tools for organizing subscriptions, you can use hashes, everything works very quickly), you can use other queue managers like RabbitMQ or Gearman. It is clear that in one chat more than a thousand participants no one will hold. Such approaches are needed more for other tasks, for example, for mass notifications of large groups of visitors about events or organization of communities.

  3. When there are a lot of visitors and at the same time you need to manage correspondence for hundreds of thousands of participants and more, both options will flood servers, no matter how lush the beds are. For the organization of communication (chat) you need to use engineered engines like jabber for years (it integrates very well with tornadoes). You can go further doing the perversion on crutches in C using a library like LevelDB (these are very fast and very serious crutches, suitable for chat of the scale of a world orgy).