In Moscow, test unmanned tram. We talked to autopilot developers.

The other day, "Vedomosti" said that Moscow will soon begin testing the unmanned tram. Now they are testing it at the depot, but after a couple of months they plan to launch it on route 17 - so far without passengers and with a driver in the cabin.

At the next stage, the driver will also continue to monitor the tram. The system will only record the triggering, but it can affect the management of the tram only in two cases - it slows down if it sees a foreign object on the tracks and slows down if the driver accelerates too much in bad weather conditions.

Most likely, a completely unmanned tram will be only a few years. How to write "Vedomosti", by 2021-2022 years.

The development is based on the Vityaz M model, which is produced by PC Transport Systems, while Cognitive Technologies is involved in the autopilot itself. The head of its department for the development of unmanned vehicles, Yuri Minkin, told us more about the project.

Sensors

The system will use 20 cameras and 10 radars - if we talk about the configuration for a completely unmanned tram. We are now working on a solution for sensors to fully cover all 360 degrees around the tram. This is our current quantity estimate.

Most will be installed in front. There in different configurations costs up to five cameras. Three cameras behind, and the rest distributed on the car. More individual sensors control the perimeter of the door. At the door will work either cameras or other sensors - we are still working on this issue. They need to see when no one is in the doorway, and they can be closed.

Separate cameras and radars are good devices, but they do not provide complete information when working in isolation, especially in difficult conditions. In ideal conditions, there are enough cameras for everything. But unfortunately we are not living in a perfect world. Cameras can for various reasons cease to function normally.

For example, the sun shines, or very low light levels, heavy snow, heavy rain. That is, those conditions in which neither a person nor the camera can see anything. It would be strange to release a system that will not go anywhere in a thick fog. It is not needed and even dangerous, because the fog can sneak up unnoticed. A radar is not so sensitive to weather conditions, but it can not see the full picture. For example, it does not recognize a traffic light.

To solve all this, we use data fusion technology - when we simultaneously process data from cameras, and radars, and make decisions based on two different sensors. So it turns out all-weather solution, which is quite acceptable at the same time.

Accuracy of detection in difficult conditions, of course, decreases. To ensure traffic safety and to ensure an acceptable level of quality, precautions must be taken - speed reduction and so on. This reduces the braking distance, and increases the time to assess the current situation.

Radars will look forward and backward. They have a large radius of action - up to two hundred meters, maybe even a little more. The radar will be at the corners so that we can control all the blind spots. And a few more around the perimeter of the tram.

Radar is our development. Cameras we take ready. But what exactly will be put, in what quantity, and at what stages is a joint issue with the manufacturer of trams. Therefore, we will better talk about everything at the stage of the final project decision.

Soft

First, it is the simultaneous acquisition and processing of data from different sensors - cameras, radars, high-precision positioning, inertial sensors. The main thing is that the data is received synchronously, so that the system understands that all information belongs to the same time period.

We also receive information from onboard tram systems. For example, the position of controls, the speed of rotation of the engine, the state of the doors and the various tram units

There is still high-precision cartography - information that we have collected in advance, and it is constantly updated. With it, we can always restore the location from information from cameras and radars, even if the GPS signal is lost. We know where all the objects of interest on the route, stops, traffic lights and so on.

For example, in order not to spend computer resources on detecting traffic signals along the entire route, we, knowing where they are, turn on the corresponding component if necessary. This allows you to optimize the computational load.

The next level of software is processing. We restore the road scene around us, arrange the objects, analyze the road scene and make decisions on how to influence the controls.

If we are talking about a driver warning system, then we evaluate whether it is time for the system to intervene, or not for it. If about autopilot, he constantly evaluates the road scene.

Machine vision and learning of neural networks

With the help of machine vision, we detect a variety of objects - pedestrians, cars, traffic lights, the position of arrows and so on. All the many objects that are monitored by the driver are also recognized by the system.

We perform recognition on the basis of neural networks - this is the most proven approach. But this task requires a relatively powerful computer so that it gives an acceptable quality at an acceptable speed. In the car there is an opportunity to place such equipment, connect it to the power supply. At the same time, the cost of equipment is acceptable for this type of transport.

We have been collecting and collecting data on existing trams for a long time. The project lasts more than six months, just now it was announced. We collect at different times of day, in different lighting conditions. We are constantly expanding datasets - this is the most valuable thing. Algorithms are developed and improved, and datasets can be used for testing and training for many years. This is the basis of any machine learning.

For some of the detectors, for example, car and pedestrian recognition, we use data that we collected before the tram project.

Security, backup system, attacks

Due to the high manufacturability of trams, we can completely electronically control the tram using an appropriate interface unit. We can affect all controls and get information about the current state of all tram systems.

There is a backup system. All nodes are duplicated with a margin. Our system always sends a signal to the autonomous unit that it works. As soon as this signal disappears, the unit simply stops the machine.

All information is contained locally. We do not control the tram outside. First, otherwise it would not be safe. Secondly, the existing communication channels do not provide sufficient guarantees. We all understand that if suddenly in the process of movement something happens to the communication channel, then the situation will be unpredictable. Therefore, everything is processed exclusively on board.

This system has no input outside. It is completely closed to attack. Only if you attack the tram itself, open the panel, connect to the wires - but this is already a fantastic story. Hacking via the Internet and control the tram is impossible. The entire system is closed and completely isolated.

In the case of attacks with the help of image-tricks the radar will help us. For example, the vision will be deceived - they will see a non-existent car and brake. But on the radar, we will see that there is nothing ahead. Yes, it will become clear - here something is not right, the tram will slow down or give a signal.

But then again, we collect all the methods that allow us to deceive our eyesight, and work out methods for how to get around them, to make sure that the vision system does not react to such pictures. Cheats are specific to each implementation, they are not universal. For example, on Tesla there is a certain system, and it becomes clear to someone how to deceive it. And most likely, what deceives Tesla will not deceive us.

You can fight this by constant monitoring. The bad guys come up with new ways to steal, the good guys come up with how to protect themselves from it.

Differences tram from the usual unmanned vehicles

On the one hand, responsibility is increasing, because it is a passenger transport. You should always ensure smooth running. You understand that there are dozens of people on board for whom you are responsible.

On the other hand, the tram goes on rails, it almost always has an advantage on traffic rules. It is not necessary to solve the steering problem, its path is always well known, all key points of interest, all traffic lights, stops. This greatly simplifies the task.

In addition, the tram is large, fewer problems with the placement of equipment, with its powering. In a car, generator power is not always enough to accommodate such equipment that allows you to drive autonomously. And in the tram with no problem.

The tram we worked with is very modern. Everything is electronically controlled, and there are already a lot of built-in security systems. For example, he cannot move at all until the door is locked. If the door bumps into something when it closes, they will open themselves, never squeeze anyone. Therefore, it turned out very successful base machine, on which we have already put their systems.

That is, from the point of view of implementation, it is simpler, but more responsible.

But a lot of things have to be coordinated. First, we agree with the manufacturer of the tram, then go to the "Mosgortrans", and this slightly complicates the course of testing. If we can drive a car to the landfill, then testing can be done either in a small depot - but there are not many trains there - or we can specifically organize testing in urban areas and ensure safety. For example, ride at night.

Autopilot and schedule

This will work just as it does now. Any tram has a timetable to follow. If there is a car on the tram tracks, the schedule will move. It is already being prepared taking into account that there may be hindrances on the way of movement. There is a very large experience in the operation of trams, it has long been known how much on average a tram can deviate from its ideal schedule. These deviations we lay in the route.

Naturally, something more serious could happen. In the future, a system will be provided for when we know about the movement of other trams, and we will move with this account.

Yes, the driver can wait an extra second for a person who runs to him, waves his hands and is late, and the car does not. Theoretically, this of course can be laid, but in practice this will not happen. If we wait a minute, then in a minute someone else will come running. This is a machine, and it works according to clear rules. Thanks to these fixed rules, it is safer.

What do you need to get the project out of pilot stage?

It is necessary to test everything - even that which is already ready. In a number of conditions, everything works well. But we understand that life is rich in events, so you have to try further, imitate different scenarios in the conditions of a city, in order to see whether the system will work or not.

In our plans to launch several trams that will ride, collect data and watch how the system works. That is, the system will not affect the controls, but will simply unsubscribe its operation. And we will monitor and compare the reaction of the system with the reaction of the driver. Based on this, we will analyze what is right, what is wrong. We will have all the information from the sensors, and we will see where what is going wrong.

Naturally, you need to refine the algorithms. Vision is already close enough to industrial standards, to what can already be allowed to work. It is necessary to work out scenarios and analyze the nuances. For example, analyze how pedestrians move, so that the system does not work falsely, but at the same time, it brakes when it is really necessary.

These are all setup, debug nuances that take a lot of time.