I'm new to RabbitMQ. Read the article " Deploying Microservices Architecture with C #, Part 2 ".

After reading asked the following question. Are there any optimal solutions for determining the number of queues, if the number of customers is initially unknown.

Given ASP.NET MVC + RabbitMQ (Pub / Sub).

  • Option 1: one queue for all customers.
  • Option 2: one queue for each client.
  • Option 3: pool queues.

How to determine the number of queues in this pool?

    1 answer 1

    In general, the practice depends on the specific task, but the queue option for each user is definitely not relevant to the Rabbitmq theme, and if you use it, then you do something wrong, the message queue is not created for such structures. Messages in fact should always be more than queues.

    For example, we need to provide fast and efficient multi-threaded video processing uploaded by users. To do this, we create a queue with the name video_processing and publish processing tasks there.

    1. The user went to the site and uploaded the video, sent a message to the queue.
    2. Consumer saw this task and blocked it for the duration of the execution.
    3. Consumer began to process the task
    4. Consumer completed the task, wrote a report somewhere
    5. Consumer removed the task from the queue, thus saying that it completed successfully
    6. Received your converted video

    In this scenario, the consummer could crash (the server rebooted), and the rebbit would easily give the task to another concierge for execution, and therefore the performance of your queue depends only on the number of workers who perform tasks from the queue. And also you could throw this message into another queue and continue processing it using other algorithms.

    The queue server is mainly used for tasks related to

    1. Data processing
    2. Process management
    3. Integration and interoperability

    Basically, they make one queue for a specific task (for example, the queue for sending letters to users), how to determine how many queues you need, again depends on your task.

    • The video example is clear. but how to be if you need a small response time (for example, many users are harassing authentication) - Atlantis
    • 2
      @Atlantis RabbitMq is not suitable for such tasks and using the queue for such tasks is impractical. If you need to increase the response rate for authorization, you need to optimize the database, use caching, scaling, you can take the authorization service to microservice and scale only it, etc. - Firepro
    • Thanks for such comprehensive answers! Regarding the feasibility of using the "rabbit" for such a task, doubts began to creep in the middle of the article, but since I am just starting to master this tool, I thought that somewhere there was not enough knowledge (although in fact I had already learned a lot of theory). There was only one question left then: maybe there are some materials from which you can find out when it’s profitable to use queues when there isn’t, or will it only give practical experience? Thanks again in advance! - Atlantis
    • @Atlantis most of the use of queues is the integration of individual components and deferred tasks, here are described quite well typical scenarios insight-it.ru/erlang/2012/rabbitmq - Firepro