Under the hood of the chat bot: what can and how does RocketBot work
RocketBot is a programmable chat bot that integrates with VK, Telegram and Bitrix24. Today we will talk about the infrastructure and other technologies on which it is built.
/ photo Darryl Braaten CC BY-SA
When developing RocketBot, we focused on the speed of development and prototyping. For this reason, the system is written in Python, and MySQL is selected as the database.
Today, Python ranks third in the TIOBE ranking in popularity. This means that the language has a large community and an endless amount of reference books: books, websites, courses, and source codes. For example, on GitHub, you can find collections of best practices for developing in Python , and a large number of threads are open on StackOverflow. A lot of books are dedicated to this language by O'Reilly, you can highlight “ Hitchhiker 's Guide to Python ” and “ Machine Learning with Python ”.
In addition to a large number of sources, one of the main advantages of Python is a variety of libraries and functional ML-frameworks. Examples include PyTorch and SciKit-learn - these are powerful tools that simplify the processing of natural speech, dialogs and scripts.
As for the database, we chose it for reasons of “not complicating”. MySQL also remains the most popular of the powerful server databases. For this reason, a large amount of information can be found on the Internet (for example, there is a detailed manual from the developers ), and an extensive community will always prompt a solution to the problem, if any. Plus, MySQL offers extensive functionality, although it does not follow all SQL standards.
Our chat bot allows you to customize automatic answers to user questions (for example, to implement the FAQ), as well as automate routine operations (for example, polls) using scripts.
The system is able to work with two types of questions: custom and standard. In the first case, the administrator independently registers the essence of the question and the keywords to which the bot will respond. List all keywords is optional. RocketBot algorithms will appreciate the meaning of the phrase and find the answer by semantic proximity.
To assess semantic proximity, a vector representation of the query is used. It highlights the main components ( PCA ) and searches for the closest option in the database. In our case, the cosine_similarity (sciki-learn) metric best showed itself.
In the second case, the bot owner does not need to configure anything. He simply chooses a standard question from the list - then the bot itself understands what it will be asked of. Now for the selection of standard themes we use the Dialogflow service, but we will change it to our own implementation. We are not satisfied with the dependence on the Google service, which introduces a delay in data synchronization. Note that the user can change the standard template at any time by creating his own variant of phrases and assigning an appropriate action.
In both cases, the chat bot can respond either with a text message or with a script. A text message is just a standard response to a situation. As for the script, it implies a tree-like structure of the dialogue. In fact, these are full-fledged communication scenarios in which a chat bot performs the programmed actions depending on the user's responses. For example, this feature can be used to conduct surveys or collect feedback.
The bot owner fills all the blocks independently in the script editor. So far, the editor has no visualization tools (but they will appear in the near future). In our experience, the presence of a visual designer in a chat bot application does not help in creating a scheme of its work. Most of the users do not think over the algorithms in advance, so there is simply nothing to transfer to the editor.
An example of settings with exit from the script when you answer "No" to the question "Are you 18 years old?"
But with all this, we can help the client customize his scripts according to the flowchart, which he draws in a graphics editor like xmind or draw.io. We will independently carry out the verification of the algorithm and carry out the initial setup - it usually takes several hours.
In addition to answering questions and implementing scripts, a bot can work with natural language processing systems. However, while they perform a very narrow range of tasks: they search for an answer to a question in the knowledge base and determine the user's intentions (the same Dialogflow is used for this).
We have limited the range of intellectual decision tasks due to potential difficulties with unpredictable behavior. Modern AI systems are still subject to errors, and when creating business services, it is impossible for a bot to respond to 80% of the questions in one way, and to the remaining 20% - to others.
To implement the natural speech processing system, a very simple search algorithm is used so far. Over time, we want to implement a new version based on LSTM networks, using reinforced learning. Most likely, the new implementation will be based on the DeepPavlov framework already mentioned.
Another feature of our bot is the integration with CRM-systems. RocketBot is able to automatically create a lead and a task in CRM (Bitrix24) after running scripts. The bot sends everything that the user has written, plus information about him from the messenger or social network. Everything works according to standard protocols and through the main API of CRM systems in the form of HTTP requests and data in JSON.
In future releases, we plan to add the bot the ability to create car-stoppers. This will allow the business to familiarize the potential customer with the product and close the transaction directly in the chat. The system will independently add the necessary entities to CRM - leads, transactions, tasks - based on the entered answers and questions.
/ photo spencer cooper CC BY-ND
All our services operate in the IT-GRAD cloud - frontend servers, databases and application servers are located there. In general, the cloud infrastructure helps us to scale our services quickly, to cope with the increasing load and sudden jumps in traffic. For this reason, in the future we only plan to expand our presence in the cloud.
The bot itself is a set of five microservices that interact with each other via HTTP – JSON (all internal communications between services are carried out in a protected segment). Although this leads to delays in answering a question in one to two seconds, this architecture has great potential for scaling and allows us to modify individual blocks without affecting others. For example, updating the microservice search in the FAQ will not affect the microservice integration with CRM.
Additional reading - posts from the First Corporate IaaS blog:
From our Telegram channel:
/ photo Darryl Braaten CC BY-SA
Tools
When developing RocketBot, we focused on the speed of development and prototyping. For this reason, the system is written in Python, and MySQL is selected as the database.
Today, Python ranks third in the TIOBE ranking in popularity. This means that the language has a large community and an endless amount of reference books: books, websites, courses, and source codes. For example, on GitHub, you can find collections of best practices for developing in Python , and a large number of threads are open on StackOverflow. A lot of books are dedicated to this language by O'Reilly, you can highlight “ Hitchhiker 's Guide to Python ” and “ Machine Learning with Python ”.
In addition to a large number of sources, one of the main advantages of Python is a variety of libraries and functional ML-frameworks. Examples include PyTorch and SciKit-learn - these are powerful tools that simplify the processing of natural speech, dialogs and scripts.
In particular, we used PyTorch to develop learning models, since it has ample opportunities for prototyping. Additionally, we implemented scikit-learn, which performs preliminary data processing, as well as pymorth2 and gensim (for working with morphology and textualization of texts, respectively). We also experience DeepPavlov - we evaluate its capabilities.
As for the database, we chose it for reasons of “not complicating”. MySQL also remains the most popular of the powerful server databases. For this reason, a large amount of information can be found on the Internet (for example, there is a detailed manual from the developers ), and an extensive community will always prompt a solution to the problem, if any. Plus, MySQL offers extensive functionality, although it does not follow all SQL standards.
What can our chat bot
Our chat bot allows you to customize automatic answers to user questions (for example, to implement the FAQ), as well as automate routine operations (for example, polls) using scripts.
The system is able to work with two types of questions: custom and standard. In the first case, the administrator independently registers the essence of the question and the keywords to which the bot will respond. List all keywords is optional. RocketBot algorithms will appreciate the meaning of the phrase and find the answer by semantic proximity.
To assess semantic proximity, a vector representation of the query is used. It highlights the main components ( PCA ) and searches for the closest option in the database. In our case, the cosine_similarity (sciki-learn) metric best showed itself.
In the second case, the bot owner does not need to configure anything. He simply chooses a standard question from the list - then the bot itself understands what it will be asked of. Now for the selection of standard themes we use the Dialogflow service, but we will change it to our own implementation. We are not satisfied with the dependence on the Google service, which introduces a delay in data synchronization. Note that the user can change the standard template at any time by creating his own variant of phrases and assigning an appropriate action.
In both cases, the chat bot can respond either with a text message or with a script. A text message is just a standard response to a situation. As for the script, it implies a tree-like structure of the dialogue. In fact, these are full-fledged communication scenarios in which a chat bot performs the programmed actions depending on the user's responses. For example, this feature can be used to conduct surveys or collect feedback.
Three types of blocks are used to compile the script:
- Question block - asks a question to the user and writes an answer to the variable in the variable.
- Condition block - implements the script branching.
- Block of checking variables - manages the user's route according to the script.
The bot owner fills all the blocks independently in the script editor. So far, the editor has no visualization tools (but they will appear in the near future). In our experience, the presence of a visual designer in a chat bot application does not help in creating a scheme of its work. Most of the users do not think over the algorithms in advance, so there is simply nothing to transfer to the editor.
An example of settings with exit from the script when you answer "No" to the question "Are you 18 years old?"
But with all this, we can help the client customize his scripts according to the flowchart, which he draws in a graphics editor like xmind or draw.io. We will independently carry out the verification of the algorithm and carry out the initial setup - it usually takes several hours.
In addition to answering questions and implementing scripts, a bot can work with natural language processing systems. However, while they perform a very narrow range of tasks: they search for an answer to a question in the knowledge base and determine the user's intentions (the same Dialogflow is used for this).
We have limited the range of intellectual decision tasks due to potential difficulties with unpredictable behavior. Modern AI systems are still subject to errors, and when creating business services, it is impossible for a bot to respond to 80% of the questions in one way, and to the remaining 20% - to others.
To implement the natural speech processing system, a very simple search algorithm is used so far. Over time, we want to implement a new version based on LSTM networks, using reinforced learning. Most likely, the new implementation will be based on the DeepPavlov framework already mentioned.
Another feature of our bot is the integration with CRM-systems. RocketBot is able to automatically create a lead and a task in CRM (Bitrix24) after running scripts. The bot sends everything that the user has written, plus information about him from the messenger or social network. Everything works according to standard protocols and through the main API of CRM systems in the form of HTTP requests and data in JSON.
In future releases, we plan to add the bot the ability to create car-stoppers. This will allow the business to familiarize the potential customer with the product and close the transaction directly in the chat. The system will independently add the necessary entities to CRM - leads, transactions, tasks - based on the entered answers and questions.
/ photo spencer cooper CC BY-ND
The infrastructure on which the chat bot is built
All our services operate in the IT-GRAD cloud - frontend servers, databases and application servers are located there. In general, the cloud infrastructure helps us to scale our services quickly, to cope with the increasing load and sudden jumps in traffic. For this reason, in the future we only plan to expand our presence in the cloud.
The bot itself is a set of five microservices that interact with each other via HTTP – JSON (all internal communications between services are carried out in a protected segment). Although this leads to delays in answering a question in one to two seconds, this architecture has great potential for scaling and allows us to modify individual blocks without affecting others. For example, updating the microservice search in the FAQ will not affect the microservice integration with CRM.
We use the Nvidia Tesla M40 16GB to test and create machine learning models and neural networks. But even on her learning is not very fast. The data sets that we use for natural language processing tasks are tens of gigabytes. For this reason, in some cases (for example, for vectoring datasets) we used virtual machines with 64 CPUs.
Additional reading - posts from the First Corporate IaaS blog:
- Zscaler - how this tool works to ensure web security
- How to place 100% of the infrastructure in the cloud IaaS-provider and not regret it
- Distributed firewall in vCloud Director 8.20: solution features
From our Telegram channel:
Source: https://habr.com/ru/post/438536/