Python + Raspberry Pi + Pixhawk and quadrocopter. Or how not to do robots
Hi, Habr!
My name is Alexey, for 7 years I am the leading developer of smart TV solutions in a large custom-made company from Izhevsk. Every year we hold a competition of Christmas decorations, and every time we do not decorate, and we saw all sorts of technological things. This time, crossed the drone and Smart TV-application. And what came out of it - read below.
The idea was quite realizable. They wanted to make a quadrocopter in the form of Santa Claus sleigh, who would deliver gifts for the staff to the office and to the music. At the same time, he had to orient himself in space using the analysis of ArUco tags, interacting with television applications (“blowing out” smoke from the pipes with running screws, running out of animals for the quadcopter meeting / escorting).
And for all three months. Of course, we did not have time.
Although at various times up to seven people worked on the project, the result was far from ideal. In general, we have learned only to launch a copter and have written applications for televisions. Set up a quadcopter interaction with TVs. But first things first.
I don’t remember in whose sick head (definitely not in mine, but this isn’t exactly) the idea to launch a quadcopter bomber in the office was born. The task is new and difficult, but we haven’t found anything beyond our strength. Yes, we rummaged Internets in order to find a library to manage the controller of the kopter. Yes, it was necessary to transfer the image from the webcam to the stream client and analyze it on the server. Yes, it was necessary to make a bomb bay. Yes, it is necessary in general to write a shell that would collect all this in itself. So what? We have already done all this (except for controlling the controller). Therefore, the project gave a green light.
Each helicopter evolutionary mistake buzzing over your head, in principle, consists of one set of mechanisms and schemes. It:
The last two items are optional. You can see the scheme of the rotor winged device here:
This good is put on the body. However, the body decided to make their own. Because:
The most important part of any helicopter larvae. It has accelerometers on three axes, software firmware, inputs for installing peripheral equipment and connecting engines, an output for controlling. Without it, no drone will rise into the air. The thing is so important that it is installed on a special shock-absorbing platform to reduce the effects of vibration and shock loads. Something like this:
We have chosen relatively major controller. “Relatively,” because then I took a closer look at the monsters that are proposed for serious uncles ... Nevertheless, the controller has a GPS, a compass, an autopilot, all sorts of other good buns, supports up to 8 engines.
Strictly speaking, such controllers are designed not only for vertically flying brethren. They can be installed on any moving platforms, ranging from helicopters, airplanes, to cars and boats. A bunch of use options. But we are somewhat distracted.
There are a lot of these friends. Round, square, curves, slanting, big, small, expensive and cheap. The main difference: the maximum power consumption, current strength and number of revolutions per second. In a perfect view, the sleigh should have been able to lift a can of beer (0.5 kg). Having estimated the total weight of the structure, we calculated the approximate required lifting force and took high-quality powerful engines. As it turned out later, they never launched at full capacity. But more is better than less ...
The only source of energy through which a potential kamikaze will stay in the air. In the course of operation, several curious moments were found out.
First, the buzzing bit of technology eats energy like free energy. The can is landed at the moment, but it takes a long time to charge.
Secondly, he not only eats, but also knows how to leave a charge "in reserve". The idea is good, because when the threshold voltage is reached, the machine interrupts the flight and makes a soft landing, but this is expressed in the fact that the bank “lands” even faster.
Small chip, without which the engines simply will not work. Designed to distribute voltage from banks to the engine. Very important thing, usually comes bundled with the motor to be bought.
"
In essence, the peripheral equipment, but without it, manage the stubby branch of the development of flying objects will not work. In addition, telemetry is used not only to control, but also to transmit video from the camera, so there are usually two antennas. By the way, telemetry is carried out by applying and transmitting signals to the channel list. They can also be controlled programmatically, but this is absolutely not recommended, because in this way we actually abandon the embedded autopilot and write our own. On the knee. And substances and sticks. On the last evening. How we love.
The main and almost the only way to position a flying nerve destroyer in space. Typically, the GPS module also contains a compass. So that nothing influences it in flight, the module is placed on a special boom, so that nothing influences it at all. We used it only for the sake of a compass, because it doesn’t catch GPS indoors. Supports other positioning systems.
Like any self-respecting GPS, can make mistakes. The error varies from a few meters to several continents. But in general, the necessary device on the street for the implementation of missions at a distance.
That for which everything is done. Thunderstorm English airports can not just chat under a camera. The controller can stabilize, remotely control it and provide very detailed settings and tools for its management. But we all did not use it. Next time.
All sapiens engineering handicrafts using controllers are essentially controlled by one program: Mission Planer. It looks like this:
Read more about it here .
It presents a wide range of features and settings. There is a flight plan planning, geo-positioning, manual control, various settings and equipment calibration.
Yes, the program is necessary and useful. Basically. But not for our task, because the air divider should be managed programmatically, and we have ruined a lot of time to deal with this planner.
Manual control itself can be carried out than your heart desires. There are settings for the remote, for the joystick. You can bring control to the mouse and keyboard. The main requirement is to calibrate the control panel. Well, manage manually. Since we are lazy developers, we did not want to manage manually. I wanted the rotor winged side result of the thinking activity to fly.
On the Internet, only this library has been found, which is designed to programmatically control the similarity of aircraft. And it was written (drum roll) on python. In the general case for versions 2.x, but also on Python 3.5, it worked quite stably. The library has rich functionality and relatively good documentation. But before you start writing your mega-managing drone code, you first need to ... Yes, you need to connect to the controller.
The Raspberry Pi microcomputer was used as a control software component, on which a web server was deployed on the aiohttp framework.
The scheme and instructions for connecting the "raspberry" and the controller can be found here .
Schematically it looks like this:
The connection method is as follows. The controller and raspberry are connected by wires according to the specified scheme. Thus, the radio signal is sent and received by the controller. On raspberries, you need to start a proxy server that will cling to the controller and transfer data from it. Connection can be made through telemetry, or via USB. They use different COM ports. The figure shows a method through telemetry.
For example, to connect from python to the controller, you must specify the address and port: connectionString = '127.0.0.1:14540 ′
But it turned out that this address and port must be specified in the launch line of the proxy server. Right here:
Yes, it is obvious, but it is never obvious. And we spent a lot of time to figure it out.
However, after starting the proxy with the correct string and after a successful connection to the drone from the python, the miracle of the hostile technology did not want to run. That is, we get the data from the copter, we see them, but the future heavenly threat of mankind refuses to execute commands. At the last moment it became clear that the data to connect to the drone should be significantly more. Conventionally, he should start sending data to the connected telemetry, otherwise, he simply did not connect it and was satisfied.
Yes, the documentation for it was much less, but the stability of the connection would definitely be higher. Because it would not have to use raspberry pins.
And then suddenly it turned out that:
Let me remind you that this design, created under the influence of helicopters from the Avatar film, was launched in the office, where there are a lot of office equipment, lamps, ventilation and decor hang from the ceiling. Well, there are also employees. In general, it quickly became clear that a serious indoor flight would be fraught with 160 severed heads.
If a flying tank is 50 by 60 cm in size, its presence in the air in a confined space will inevitably lead either to damage or to those near it. An ideal test area would be a room the size of a gym.
A separate portal to hell was the task of computer vision. The idea is to read ArUco tags, each of which has a recorded position in the room plan, and relative to which the driver determines its position in space, as well as its position relative to the current point of the route.
Example tags:
For implementation, the openCV library was used . It is used to recognize in general everything that is, in particular: the faces of people, objects, license plates, and of our tags. Installing the library on the raspberry operating system - Rasbian - became a nightmare, with which four people fought in turn. Nevertheless, we successfully solved it, and now the machine has “learned” to recognize tags in a video stream from a webcam. But again, not enough time to configure everything. For example, from two or more tags the library returned the recognized numbers of all tags, and the data is only one by one. Why is that? The mystery is great ...
So that the mutant of the Icarus wings would not stick into the EARLY obstacles appearing, they wanted to use distance sensors. They were supposed to be on all sides of the car and had to signal the approach of an obstacle, so that the driver should stop, take a “step to the side” and continue on the route.
We did not manage to do this functionality at all.
We talked a lot about the routes of the copter. And what kind of animals are these? I will say right away that this functionality was implemented very first and in full, but ... it was not useful.
Movement routes are a connected graph with vertices, each of which has its three-dimensional coordinate in the space of the room. Accordingly, the kopter should follow from the starting point to the finish, in the latter to perform the flight task. It follows by moving from the current vertex to the next. Since for each vertex we know its coordinates, and at the copter we know its location (compass) and position (marks), then it is a purely technical matter to calculate where to go. The task of finding the path between the starting point and the finish is solved by a recursive function. Routes, edges and vertices are stored in a database deployed on raspberries.
Copter: Raspberry Pi , Rasbian , OpenCV , Python 3.5 , aiohttp , DroneKit , RPi.GPIO , SQLite .
Server with dynamics for TV: node.js , Express , socket.io .
TV applications: JavaScript ES6 , webpack 2 , Canvas .
The repository with the code of the copter is here . For stars - the benefits of karma.
A few words to clarify the terms (please consider any assessment as subjective).
In the original formulation, the task is realizable, although it requires significantly more time and effort. Due to the fact that the autopilot and the control methods of the copter are a little less than fully tied to GPS, they must be abandoned aside direct commands to the control channels. This entails the need to write your autopilot, that the problem is solved. And then - test, test and test.
Personally, I was interested. And I still will make an automated gift bomber, one way or another. And I’m leaving everyone interested in
Good luck everyone!
My name is Alexey, for 7 years I am the leading developer of smart TV solutions in a large custom-made company from Izhevsk. Every year we hold a competition of Christmas decorations, and every time we do not decorate, and we saw all sorts of technological things. This time, crossed the drone and Smart TV-application. And what came out of it - read below.
The idea was quite realizable. They wanted to make a quadrocopter in the form of Santa Claus sleigh, who would deliver gifts for the staff to the office and to the music. At the same time, he had to orient himself in space using the analysis of ArUco tags, interacting with television applications (“blowing out” smoke from the pipes with running screws, running out of animals for the quadcopter meeting / escorting).
And for all three months. Of course, we did not have time.
Although at various times up to seven people worked on the project, the result was far from ideal. In general, we have learned only to launch a copter and have written applications for televisions. Set up a quadcopter interaction with TVs. But first things first.
I don’t remember in whose sick head (definitely not in mine, but this isn’t exactly) the idea to launch a quadcopter bomber in the office was born. The task is new and difficult, but we haven’t found anything beyond our strength. Yes, we rummaged Internets in order to find a library to manage the controller of the kopter. Yes, it was necessary to transfer the image from the webcam to the stream client and analyze it on the server. Yes, it was necessary to make a bomb bay. Yes, it is necessary in general to write a shell that would collect all this in itself. So what? We have already done all this (except for controlling the controller). Therefore, the project gave a green light.
Device
Each helicopter evolutionary mistake buzzing over your head, in principle, consists of one set of mechanisms and schemes. It:
- controller,
- motors with screws
- battery,
- speed control (ESC),
- telemetry antenna,
- GPS module and compass,
- camera.
The last two items are optional. You can see the scheme of the rotor winged device here:
This good is put on the body. However, the body decided to make their own. Because:
- yourself with a mustache,
- they needed the hull in the form of a sleigh,
- it was indecently expensive.
Controller
The most important part of any helicopter larvae. It has accelerometers on three axes, software firmware, inputs for installing peripheral equipment and connecting engines, an output for controlling. Without it, no drone will rise into the air. The thing is so important that it is installed on a special shock-absorbing platform to reduce the effects of vibration and shock loads. Something like this:
We have chosen relatively major controller. “Relatively,” because then I took a closer look at the monsters that are proposed for serious uncles ... Nevertheless, the controller has a GPS, a compass, an autopilot, all sorts of other good buns, supports up to 8 engines.
Strictly speaking, such controllers are designed not only for vertically flying brethren. They can be installed on any moving platforms, ranging from helicopters, airplanes, to cars and boats. A bunch of use options. But we are somewhat distracted.
Engines
There are a lot of these friends. Round, square, curves, slanting, big, small, expensive and cheap. The main difference: the maximum power consumption, current strength and number of revolutions per second. In a perfect view, the sleigh should have been able to lift a can of beer (0.5 kg). Having estimated the total weight of the structure, we calculated the approximate required lifting force and took high-quality powerful engines. As it turned out later, they never launched at full capacity. But more is better than less ...
Battery
The only source of energy through which a potential kamikaze will stay in the air. In the course of operation, several curious moments were found out.
First, the buzzing bit of technology eats energy like free energy. The can is landed at the moment, but it takes a long time to charge.
Secondly, he not only eats, but also knows how to leave a charge "in reserve". The idea is good, because when the threshold voltage is reached, the machine interrupts the flight and makes a soft landing, but this is expressed in the fact that the bank “lands” even faster.
Speed control
Small chip, without which the engines simply will not work. Designed to distribute voltage from banks to the engine. Very important thing, usually comes bundled with the motor to be bought.
Telemetry antenna
"In essence, the peripheral equipment, but without it, manage the stubby branch of the development of flying objects will not work. In addition, telemetry is used not only to control, but also to transmit video from the camera, so there are usually two antennas. By the way, telemetry is carried out by applying and transmitting signals to the channel list. They can also be controlled programmatically, but this is absolutely not recommended, because in this way we actually abandon the embedded autopilot and write our own. On the knee. And substances and sticks. On the last evening. How we love.
GPS module
The main and almost the only way to position a flying nerve destroyer in space. Typically, the GPS module also contains a compass. So that nothing influences it in flight, the module is placed on a special boom, so that nothing influences it at all. We used it only for the sake of a compass, because it doesn’t catch GPS indoors. Supports other positioning systems.
Like any self-respecting GPS, can make mistakes. The error varies from a few meters to several continents. But in general, the necessary device on the street for the implementation of missions at a distance.
Camera
That for which everything is done. Thunderstorm English airports can not just chat under a camera. The controller can stabilize, remotely control it and provide very detailed settings and tools for its management. But we all did not use it. Next time.
Control
All sapiens engineering handicrafts using controllers are essentially controlled by one program: Mission Planer. It looks like this:
Read more about it here .
It presents a wide range of features and settings. There is a flight plan planning, geo-positioning, manual control, various settings and equipment calibration.
Rule number 1: use only what you need
Yes, the program is necessary and useful. Basically. But not for our task, because the air divider should be managed programmatically, and we have ruined a lot of time to deal with this planner.
Manual control itself can be carried out than your heart desires. There are settings for the remote, for the joystick. You can bring control to the mouse and keyboard. The main requirement is to calibrate the control panel. Well, manage manually. Since we are lazy developers, we did not want to manage manually. I wanted the rotor winged side result of the thinking activity to fly.
Program control
On the Internet, only this library has been found, which is designed to programmatically control the similarity of aircraft. And it was written (drum roll) on python. In the general case for versions 2.x, but also on Python 3.5, it worked quite stably. The library has rich functionality and relatively good documentation. But before you start writing your mega-managing drone code, you first need to ... Yes, you need to connect to the controller.
The Raspberry Pi microcomputer was used as a control software component, on which a web server was deployed on the aiohttp framework.
The scheme and instructions for connecting the "raspberry" and the controller can be found here .
Schematically it looks like this:
The connection method is as follows. The controller and raspberry are connected by wires according to the specified scheme. Thus, the radio signal is sent and received by the controller. On raspberries, you need to start a proxy server that will cling to the controller and transfer data from it. Connection can be made through telemetry, or via USB. They use different COM ports. The figure shows a method through telemetry.
Rule number 2: incomplete documentation. Often do not write the most obvious things.
For example, to connect from python to the controller, you must specify the address and port: connectionString = '127.0.0.1:14540 ′
But it turned out that this address and port must be specified in the launch line of the proxy server. Right here:
mavproxy.py —master=/dev/ttyAMA0 —baudrate 921600 —aircraft MyCopter Yes, it is obvious, but it is never obvious. And we spent a lot of time to figure it out.
However, after starting the proxy with the correct string and after a successful connection to the drone from the python, the miracle of the hostile technology did not want to run. That is, we get the data from the copter, we see them, but the future heavenly threat of mankind refuses to execute commands. At the last moment it became clear that the data to connect to the drone should be significantly more. Conventionally, he should start sending data to the connected telemetry, otherwise, he simply did not connect it and was satisfied.
Hence the rule number 3: use USB
Yes, the documentation for it was much less, but the stability of the connection would definitely be higher. Because it would not have to use raspberry pins.
Test flight
And then suddenly it turned out that:
- kopter flies not exactly
- his telemetry errors ± meter and more,
- For a flight, tasks of the type are adequate: climb 10-20 meters, fly 100 meters to the north, etc.
Let me remind you that this design, created under the influence of helicopters from the Avatar film, was launched in the office, where there are a lot of office equipment, lamps, ventilation and decor hang from the ceiling. Well, there are also employees. In general, it quickly became clear that a serious indoor flight would be fraught with 160 severed heads.
Rule number 4: use normal test conditions
If a flying tank is 50 by 60 cm in size, its presence in the air in a confined space will inevitably lead either to damage or to those near it. An ideal test area would be a room the size of a gym.
Computer vision
A separate portal to hell was the task of computer vision. The idea is to read ArUco tags, each of which has a recorded position in the room plan, and relative to which the driver determines its position in space, as well as its position relative to the current point of the route.
Example tags:
For implementation, the openCV library was used . It is used to recognize in general everything that is, in particular: the faces of people, objects, license plates, and of our tags. Installing the library on the raspberry operating system - Rasbian - became a nightmare, with which four people fought in turn. Nevertheless, we successfully solved it, and now the machine has “learned” to recognize tags in a video stream from a webcam. But again, not enough time to configure everything. For example, from two or more tags the library returned the recognized numbers of all tags, and the data is only one by one. Why is that? The mystery is great ...
Rule important: allocate enough time to fine-tune the decision
Other senses
So that the mutant of the Icarus wings would not stick into the EARLY obstacles appearing, they wanted to use distance sensors. They were supposed to be on all sides of the car and had to signal the approach of an obstacle, so that the driver should stop, take a “step to the side” and continue on the route.
We did not manage to do this functionality at all.
Routes
We talked a lot about the routes of the copter. And what kind of animals are these? I will say right away that this functionality was implemented very first and in full, but ... it was not useful.
Movement routes are a connected graph with vertices, each of which has its three-dimensional coordinate in the space of the room. Accordingly, the kopter should follow from the starting point to the finish, in the latter to perform the flight task. It follows by moving from the current vertex to the next. Since for each vertex we know its coordinates, and at the copter we know its location (compass) and position (marks), then it is a purely technical matter to calculate where to go. The task of finding the path between the starting point and the finish is solved by a recursive function. Routes, edges and vertices are stored in a database deployed on raspberries.
The last rule: do only what you need to implement
Technology stack
Copter: Raspberry Pi , Rasbian , OpenCV , Python 3.5 , aiohttp , DroneKit , RPi.GPIO , SQLite .
Server with dynamics for TV: node.js , Express , socket.io .
TV applications: JavaScript ES6 , webpack 2 , Canvas .
The repository with the code of the copter is here . For stars - the benefits of karma.
A few words to clarify the terms (please consider any assessment as subjective).
- Raspberry Pi. A full-fledged microcomputer with available pins and outputs for a special camera and touch screen. It is located on the control code, web server, openCV library. In fact, the copter is a flying server.
- Raspbian Linux operating system for "raspberry". There are several of them and, in general, you can put any, but it is better to put specially designed, otherwise there may be problems with the interaction with computer hardware, with the same pin, for example. There is even an adapted version of the tenth Windows.
- OpenCV. A library for recognizing elements in a video stream. Also able to stream video stream, which is used in the project to issue images to the client. (yes, the copter has a web client)
- Python 3.5. Python. 3.5. Not 2. *.
- Aiohttp. Asynchronous web server framework written in python. For some reason, he liked more than Django . But then the choice.
- DroneKit. Library to connect to the controller kopter from the program. Very rich functionality, multiplied by the relatively good documentation. Provided there are no analogues - a brilliant tool.
- RPi.GPIO. Python library for interacting with raspberry pins. Allows you to enable and disable pins, listen to their values. Cannot transfer values other than 0 or 1-tsy. Or I have not found.
- SQLite. DBMS + DB + firewood, combined into one file. Ideal for educational and non-educational projects, if you do not need to deploy something heavily loaded. Convenient to make changes.
- Node.js. Server-side javascript
- Express A web server framework written in node.js. Very comfortable and minimalist.
- Socket.io. Server and client library for webSocket protocol implementation. The main plus is the stability of work and the extensive functionality provided.
findings
In the original formulation, the task is realizable, although it requires significantly more time and effort. Due to the fact that the autopilot and the control methods of the copter are a little less than fully tied to GPS, they must be abandoned aside direct commands to the control channels. This entails the need to write your autopilot, that the problem is solved. And then - test, test and test.
Personally, I was interested. And I still will make an automated gift bomber, one way or another. And I’m leaving everyone interested in
The only rule is: set high and complex goals and go to them!
Good luck everyone!
Source: https://habr.com/ru/post/438496/