It is no secret that in the modern high-tech world, we are required to constantly confirm our identity and access rights - with passwords, contactless tags, fingerprints, and so on. You can do this with the help of a visual fingerprint - the user's face. Just look at our gadget, as he understands that he is an authorized user, and removes the screen lock.

We decided to implement a similar scheme with the help of available materials and open source solutions. As a physical object, which will have the appropriate "selectivity for discoveries," a safe was chosen.

All the details under the cut.

Housing

Figure 1. Chassis Assembly Process

The body is made of scraps of laminated MDF panels 40 x 40 cm, connected by furniture screws. Subsequently, all the joints were thickly smeared with acrylic putty on wood, Not really thoroughly sanded, after which the whole body was coated several times with acrylic white paint in a thick layer. We paint, wait until the current layer dries, sand, cover with another layer. After three layers, I got a uniform white matte coating, on which I walked with a thin layer of white glossy enamel.

Figure 2. Final body during electrical installation

From the bottom of the case, fasten the printed legs - the first parts drawn in the 3D editor.

Figure 3. Legs. PLA plastic

Electrician

The lock for the safe was made of a hybrid of the drive of the central lock and the fold tongue of the interior door - from the last one we throw out the swivel sleeve and make a hook under the traction.
With the help of printed parts, everything is assembled into a single structure and is mounted on the door. The main plate itself is quite thin (3mm) and easily bends, but everything changes after installation in place. The final design turned out strong enough for further experiments.

Figure 4. Lock Assembly Mechanism

Of course, if we are talking about a complete access control system, then you can buy an all-metal ready-made electric lock by installing it in a normal metal safe. But no.

In addition to the lock, we still need a limit switch in order to monitor the state of the door, a three-position switch to control the system, and a two-color LED to display the current program status.

Figure 5. Schematic diagram of body kit

We do not forget that the voltage on the pins is 1.8V (with an expansion board already 3.3V), so the LEDs are connected through a transistor stage - they will not light up from 1.8V.

Electronics

The main components of the system — a camera and a single-board computer — were made for each other (in our case, this is not a speech turn, but a known fact):

Figure 6. Intel Joule 570x Development Kit - single board microcomputer with 4-core 64-bit Intel Atom onboard, 4GB of memory and USB 3.0 connector

Figure 7. Intel RealSense Camera SR300 - 3D near-range camera as an organ of vision. Effective recognition distance depth of 0.2 - 1.2 m.

Literally a couple of days after the start of the project, Intel Joule managed to acquire EOL (End Of Life) status, but this did not stop us.

As a computing module, you can take almost any modern single-board with a USB3.0 processor based on Intel. For example Up Squared Grove IoT Developement Kit or MinnowBoard Turbot * . Intel NUC is also suitable.

Assembly

Install the parts of the lock on the box so that the lock closes normally. In order for the door to bounce when triggered (as in postamats), we print and install a spring pusher. His task is to pull the door off the latch by several millimeters after the tongue is pulled in, so that after removing the tension the box does not close again.

Figure 8. Latch for tongue and spring door pusher

Front mount the camera. We put a LED in the cover for the camera, to a point above i - for certain reasons, he managed to metamorphize into one green LED. Fix it with a pair of screws.

Figure 9. Installed Camera

For Intel Joule, a case is needed, so a case of the required dimensions with windows for connectors is drawn in the 3D editor.

Figure 10. Housing drawing for Intel Joule

For 40-pin GPIO-connectors, no windows are provided, since it was decided to place the entire electrician inside the case. To do this, draw a thick cover, with grooves for the relay and 4 contacts, as well as windows for switches. Do not forget that the processor on the board is not the weakest, so we make several ventilation holes in the cover.

Figure 11. Cover drawing

There are two shoals. Firstly, the development of the body kit did not take into account the connector to connect the LED, which led to the fact that:

1. The case does not provide a hole for the cable / connector and had to burn the hole with a soldering iron;
2. The assembled body has a long tail with LEDs on the end.

Secondly, the body must somehow be connected and nothing better than to do it with the help of pieces of wire as a result was not invented ¯_ () _ / ¯. About mounting generally keep quiet. You can not see, but somewhere there, under the thermo-sauce, 3 transistors and five heels of rescuers are hidden.

Figure 12. Assembled electronics unit

The inside of the box - four terminals - two for power and two for the motor, the limit switch for the door, back and forth to control the program, and the relay supplying voltage to the lock. During printing, the angle near the cavity for the relay was cut off from the table and the relay decided not to fit in it. I had to glue the switch on thermosoopl. As well as single-board.

Wi-Fi antennas of the board are glued with green tape. In general, they are on self-bonding, but we have already glued them so many times during experiments, that they have lost the property of sticking to the surface.

Figure 13. Assembled closed electronics unit

We place the guts inside the case, close it and proceed to the development of the program part.

Lyrical digression on 3D printing.

The details of the castle are my favorite and hated PLA. The material is very good for printing high-precision models, but it is completely impossible to process. And this infection floats when heated, which is a very bad property for structural elements.

All other parts of HIPS - IMHO, it is less naughty than ABS, and when printed, it gives almost no smell. The only caveat - until he raised the temperature of the table to 95 degrees, the detail on the table warped. According to the instructions, the temperature of the table should be about 100 degrees, but who reads it so far so good?

Software part

System preparation

First of all, you need to update the BIOS on the board to install the image of the operating system we need. We used this and this instructions from the official documentation on the Intel website.

Next, by connecting a monitor and keyboard to the board, we install the special Ubuntu * Desktop 16.04 LTS distribution kit for Intel Joule in the usual graphic or text mode. If Intel NUC or another computer is used as a computer, the stock image of the system will also go.

After installing the OS using standard tools, we connect to the Internet via the built-in Wi-Fi. Since there is only one USB connector on the board and we need it to connect the camera, we connect to the board via ssh. Now we can free the port from the keyboard and connect Intel RealSense.

Installing packages and dependencies.

First, install Python 3.5.4, in which our project is written. Download and build an OpenCV Python module from the source code, since the unofficial OpenCV module for Python on Intel Joule does not work correctly.

Install or update additional dependencies:

apt install -y --fix-missing build-essential cmake gfortran git wget curl graphicsmagick libgraphicsmagick1-dev libatlas-dev libavcodec-dev libavformat-dev libboost-all-dev libgtk2.0-dev libjpeg-dev liblapack-dev libswscale-dev pkg-config python3-dev python3-numpy software-properties-common zip git clone https://github.com/davisking/dlib.git /root/dlib; cd /root/dlib; mkdir build; cd build; cmake .. -DUSE_AVX_INSTRUCTIONS=1; cmake --build . ; cd .. ; python3 setup.py install --yes USE_AVX_INSTRUCTIONS --no DLIB_USE_CUDA pip3 install face_recognition 

We put libmraa - this library is useful to us for communication with the box. Detailed assembly instructions are located here .

Or you can install prebuilt PPA binaries:

 sudo add-apt-repository ppa:mraa/mraa sudo apt-get update sudo apt-get install libmraa1 libmraa-dev libmraa-java python-mraa python3-mraa node-mraa mraa-tools 

After installing all the dependencies, we can verify that everything is working correctly. To do this, raise the VNC - vnc4server is our everything, launch the sample for working with the camera and make sure that the camera is working.

The final test point is the correct import of the face_recognition module. In fact, this is our ResNet neural network, sharpened to identify people in the photo. So it looks like in the author's demo:

 import face_recognition 

As everything will be ready, we unload the source code of our application from the repository and get down to work.

For convenience, we added all the libraries as submodules to our project. To download them, run git submodule update --init --recursive from the project directory.

Job

 ./launcher.py 

When you first turn on the database is not persons, therefore the box will be closed. Open it manually (we have a built-in backdoor for this - a small hole is clearly opposite the tongue) and we carry out training.

To do this, we stand in front of the box - on the front panel the LED lights up, indicating that a certain person is in the field of view of the camera.

Click the switch in the direction of remembering the person and the corresponding entry appears in the console.

Now, if you close the box and stand in front of him - the system recognizes his face and happily opens the door.

Access grants will appear in the console.

If a stranger gets up - the system will not be able to identify him, as a result of which the door will remain closed. Another thing is that the system can be taken out of moderation - the percentage of false positives in our case was quite high and if you walk for a long time with a bunch of different objects in your hands - it will open sesame.

This can be solved by using the depth channel - in the current version it is not used, since it was never possible to get it to work stably, and it was too late to send a compatibility issue with Intel Joule. Author's face_recognition measured accuracy of 99.38% on the benchmark Labeled Faces in the Wild Home

Conclusion

Of course, this Proof-Of-Concept can be significantly improved, and its functionality has been expanded many times over. For example, you can add logging of successful and unsuccessful access, as well as convenient management via a web interface.

The basic principle of the design of this device was: "I blinded him from what it was." We just wanted to show that it is possible to make something interesting out of the materials at hand.

Thanks to Dmitry ( dmitryvodop ) and Sergey for working on the program part of the project.

All models in the * .stl format and the source code of the program is laid out on GitHub under the MIT license.