How Retentioneering is Implemented in the App in the Air

To keep a user in a mobile application is a whole science. Its basics in our article on VC.ru was described by the author of the Growth Hacking course : mobile application analyst Maxim Godzi, Head of Machine Learning at App in the Air. Maxim talks about the tools developed by the company using the example of analyzing and optimizing a mobile application. Such a systematic approach to product improvement, developed in App in the Air, is called Retentioneering. You can also use these tools in your product: some of them are freely available on GitHub.

App in the Air is an application with more than 3 million active users worldwide, with which you can track flights, get information about changes in departure / landing time, registration and airport characteristics.

From funnel to trajectory

All development teams build an onboarding funnel (a process aimed at user acceptance of a product). This is the first step that helps to look at the entire system from above and find the problems of the application. But in the course of product development, you will feel the limitations of this approach. With the help of a simple funnel, it is impossible to see unobvious growth points for a product. The goal of the funnel is to give a general view of the stages of users in the application, to show you the metrics of the norm. But the funnel will providently conceal deviations from the norm in the direction of obvious problems or, on the contrary, special activity of users.



We built our own funnel in App in the Air, however, due to the nature of the product, we got an hourglass. Then we decided to expand the approach, and use the rich information that the application itself gives us.

When you build a funnel, you lose onboarding paths by users. Trajectories consist of a sequence of user actions and the application itself (for example, sending a push notification).



By timestamps, you can very easily restore the user's trajectory and make it a graph for each of them. Of course, there are a lot of graphs. Therefore, you need to group similar users. For example, you can arrange all users in rows of a table and describe how often they use a certain function.



On the basis of such a table, we made a matrix and grouped users according to the frequency of use of functions, that is, the nodes in the graph. This is usually the first step towards insights: for example, at this stage you will see that some users do not use some of the functions at all. When we did frequency analysis, we began to study which nodes are the “biggest”, that is, on which pages users most often come to. Immediately stand out categories that differ fundamentally by some criterion important to you. Here, for example, two clusters of users, which we divided on the basis of the subscription decision (there were 16 clusters in total).

How to use it

By looking at your users in this way, you can see by what functions you keep them or, for example, you seek a subscription. Naturally, the matrix will show obvious things as well. For example, that those who bought a subscription, were on the subscription screen. But besides this, you can find patterns that you would never know otherwise.

So we accidentally found a group of users who add a flight, actively monitor it during the day and then disappear for a long time, until they fly away again. If we analyzed their behavior with the help of ordinary tools, we would think that the functionality of the application simply did not suit them: how else to explain that they used it for one day and never return. But with the help of graphs, we saw that they were very active, just all their activity fit into one day.

Now our main task is to push such a user to connect his carrier’s loyalty program while he is using our statistics. In this case, we import all the flights that he buys and will try to push him to a subscription as soon as he buys a new ticket. To solve this problem, we also began to cooperate with Aviasales, Connected. Travel and other applications. When their user buys a ticket, the application offers him to add a flight to the App in the Air, and we immediately see it.

Thanks to the graph, we saw that 5% of people who enter the subscription screen refuse it. We began to analyze such cases, and we saw that there is a user who comes to the first page, initiates the connection of his Google account, and immediately cancels it, goes back to the first page, and so four times. At first we thought: “With this user, something is clearly wrong.” And then they realized that, most likely, there is a bug in the application. On the funnel, this would be interpreted as follows: the user did not like the set of permissions that the application requested, and he left.

In another group, 5% of users were lost on the screen, where the application offers to select one of all calendar applications on the smartphone. Users chose different calendars again and again, and then simply exited the application. It turned out that there was a UX problem here: after the person had selected the calendar, he had to press Done in the upper right corner. Just not all users have seen it.


First screen App in the Air

On our graph, we saw that about 30% of users do not go beyond the first screen: this is due to the fact that we are quite aggressively pushing the user to a subscription. On the first screen, the application offers to register using Google or Triplt, and there is no information that you can skip the registration. Of those who leave the first screen, 16% of users click "More" and return again. We found out that they are looking for a way to internally register with the application, and we will release it in the next update. In addition, 2/3 of those who immediately leave, do not press anything at all. To find out what is happening with them, we built a heatmap - a heat map. It turned out that customers click on the list of application functions that are not active links.

Catch the moment

It is often possible to see that people tread paths along an asphalt road. Retentioneering is an attempt to find these paths and, if possible, change the roads.

Of course, it’s bad that we learn from real users, but at least we’ve started automatically tracking patterns that tell users about the problem in the application. Now the product manager receives email notifications if a large number of “loops” occur - when the user returns to the same screen again and again.

Let's look at what patterns in user trajectories are generally interesting to look for in order to analyze problems and application growth points:

• Loops and loops. The loops mentioned above are when one event repeats in the user's trajectory, for example, calendar-calendar-calendar-calendar. A loop with a large number of repetitions is an obvious pointer to a problem with the interface or insufficient marking of events. The cycle is also a closed path, but unlike a loop, it includes more than one event, for example: viewing the flight history — adding a flight — viewing the history of flights.
• Flowstoppers - when a user, due to some obstacle, cannot continue his desired movement through the application, for example, a screen with an interface that is not obvious to the client. Such events slow down and shift the trajectory of users.
• Bifurcation points are significant events, after which the trajectories of different types of clients are separated. In particular, these screens, which do not contain a direct transition or a call-to-action to the target action, effectively push some users towards it. For example, some screen that is not directly related to the purchase of content in the application, but on which customers tend to buy or not to buy content, will behave differently. Bifurcation points can be points of influence on the actions of your users with a plus sign - to influence the decision to buy or the right click, or minus - they can determine that after a few steps, the user will leave the application.
• Aborted conversion points are potential bifurcation points. They can be thought of as screens that might prompt a target action, but do not. This may also be the point in time when the user has a need, but we do not satisfy it, because we simply do not know about it. Analysis of the trajectory should allow this need to identify.
• Distraction point - screens / pop-up, which do not carry value to the user, do not affect the conversion and can at the same time "blur" the trajectory, distracting the user from the target action.
• Blind spots are hidden application points, screens and features that are very difficult for the user to reach.
• Drains - points where traffic leaks

In general, the mathematical approach has allowed us to understand that the client is using the application in a completely different way than product managers usually think, trying to plan a user’s own standard usage scenario. Sitting in the office and attending the coolest product conferences, it is still very hard to imagine all the variety of real field conditions in which the user will solve his tasks using the application.

It reminds a cool joke. The tester comes into the bar and orders: a beer mug, 2 beer mugs, 0 beer mugs, 999999999 beer mugs, a lizard in a glass, –1 beer mug, qwertyuip beer mugs. The first real customer comes into the bar and asks where the toilet is. The bar erupts in flames, everyone dies.

Grocery analysts, deeply immersed in this problem, began to introduce the concept of micro-momentum. The modern user needs an instant solution to their problem. Google started talking about it several years ago: in the company, such actions of users were called micro moments. The user is distracted, accidentally closes the application, does not understand what is required of him, comes in again after 24 hours, forgets again, and then via the link that a friend sent to him in the messenger. And all these sessions can last no more than 20 seconds.

So we began to try to customize the work of the support service so that employees could understand what the problem was in almost real time. By the time the person came to the page of contacting the support service and started writing his own question, we can determine the essence of the problem, knowing its trajectory - the last 100 events. Previously, we automated the distribution of all calls to the support service by category using ML-analysis of the text of requests to the support service. Despite the success of the categorization, when 87% of all calls are correctly distributed among one of 13 categories, it is the work with trajectories that is able to automatically find the most appropriate solution for the user's situation.

We cannot release updates quickly, but we are able to notice a problem even if the user follows the scenario we have already seen, send him a push notification.

We see that the task of optimizing the application requires a rich toolkit for the study of user trajectories. Then, knowing all the paths that users follow, you can pave the necessary paths, and with the help of customized content, push notifications and adaptive UI elements “by the handle”, lead the user to target actions that best suit his needs and bring money, data and other values ​​for your business.

What to note

• Explore the conversion of users only by the example of funnels - to lose the rich information that the application itself gives us.

• Retentioneering-analysis of user trajectories on graphs helps to see which functions you use to keep users or, for example, encourage them to subscribe.
• Retentioneering tools help automatically track real-time patterns that tell users about an in-app problem, find and close bugs where they are hard to spot.

• They help to find non-obvious patterns of user behavior.

• Retentioneering tools provide the ability to build automated ML tools for predicting key events with the user and metrics: user loss, LTV, and many other metrics that are easily identified on the graph.

We create a community around retentioneering for a free exchange of ideas. You can take the tools that we develop as a language in which analysts and products from different mobile and web applications can share insights, best techniques and methods. You can learn how to use these tools on the Growth Hacking course : Binary District mobile app analytics .