Broken for several years, Robonaut finally returns home

The mysterious problem with iron did not allow the Robonavt to work on the ISS since 2015, so they decided to return it to Earth for repair

In February 2011, NASA launched the Robonavt-2 project to the International Space Station. This was a great achievement for the NASA robotics team from the Lyndon Johnson Space Center in Houston. There were other robots in space, but the Robonaut became the first humanoid robot to go beyond Earth. Onboard the ISS, the robot had to end up doing work on a par with astronauts and perform some of the most boring and repetitive tasks, which took considerable time that people on board could spend on science and discoveries.



For a while, everything went well. The robot was unpacked, freed from the protective skin, and installed in the Destiny laboratory module . It was first turned on in August 2011, and by 2012 it was already practicing work with switches and cleaning the handrails, being controlled remotely from the Earth. About once a month, the astronauts set up a Robonavt, and he was engaged in research tasks for several hours in a row, working to transform from a pilot project into a useful assistant in the care of a human spacecraft. Robonavt even got himself a twitter: "Look, I'm in space!"


But in 2014, NASA decided to carry out a complex and risky update. The robonavt was a torso with a pair of arms and a head, and now NASA wanted to add a pair of legs to it. The idea was to add more mobility inside the station. Elongated, winding legs would work almost like a second pair of hands and allow the robot to move around the station, holding onto the handrails - this would seriously increase its capabilities.

But something went wrong when upgrading the equipment, and repeated attempts to fix the stubborn problem were unsuccessful. The last few years Robonavt practically did not work, and reports available to the public from the ISS show that the last time the robot was involved in full-fledged research work in December 2013 . This week, NASA announced that it is returning Robonauta back to earth to repair.


Robonaut behind his panel with tasks in January 2013

The decision to add mobility to Robonavtu was part of NASA's long-term plan. At first, the agency wanted to divide the work of Robonavt into three phases . The first phase is stationary operations, and it was successfully completed by the end of 2013, when Robonaut used only one torso. Phase 2 was called “internal mobility”. For her, Robonavt needed to organize a method of movement inside the ISS, for which the legs were required. These legs should also have become the basis of the third phase of “external mobility”, which would include work in a vacuum outside the station, although such a scenario would require an even more serious upgrade of the robot equipment.

“The space station program was interested in expanding our capabilities,” Julia Badger , project manager for NASA's Robonaut, told us. “If we could start moving around, we could do logistics, clean the filters, do repairs. Movement was the next level of opportunity we could develop and test. ”


Robonav's winding legs

But it was more difficult for Robonavt to add new legs than to simply send them to the ISS and screw it to its torso. Installing the legs would require a serious upgrade of a significant part of Robonavt’s key equipment, including the installation of new computers and the wiring of new connections for the feet interface from the robot’s main signal handler, not to mention the complicated mechanical assembly process. Complicating matters is that Robonavt was originally intended for research on Earth. “The robonavt was not designed at all so that it could be served by astronauts,” said Badzher. “He was supposed to be a laboratory robot, and we just jumped at the opportunity to send him into space.”

Even for NASA, such an update of the legs was a difficult task. When asked whether the team was confident that all this would work, Badger said: “I know for sure that we were not sure. The entire ISS project is about trying something new, so we learned a lot from the development of these procedures and after the astronauts went through this complicated surgery. ” She also added that the team members “did an excellent job,” and that the problem with the robot is not related to the actions of the astronauts, but follows solely from the complexity of the equipment.

NASA expected that it would take about 20 hours for the astronauts to upgrade their legs, for which the Robonaut development team took 14 hours on Earth. As a result, it took 40 hours. The work was started on July 16, 2014 and completed on August 28.


Astronaut Steve Swanson with Robonavt in 2014, after the installation of legs

And almost immediately the team of the Robonavta from the Space Center realized that something had gone wrong. On August 29, the robot was partially powered up, but they did not receive any telemetry data on the ground. Shortly after that, the bad contact of the wire was corrected, and the robot went through intermediate checks, but during the next task on December 17 (the first time the robot turned on the power to the motor), the legs did not work.

From January to August 2015, astronauts and crews on the ground encountered unpredictable device behavior, including sensor failure, communication failure, and constant processor hangs. By September, continued fault-taking indicated that the problem was with the power supply of the Robonaut processors. “We could experience a loss of power to the processors during the operation of the device, and the situation deteriorated steadily over time,” Badger told us. - The power cycle allowed the power to return, but not for long. The problem was that due to power outages, sometimes we could turn it back on, and sometimes it was simply cut off, so we could not trust the data we received - all this was very confusing. ”

Daily reports from the ISS allow us to look out of the corner of my eye at how much the astronauts and the Robonaut team on earth have been tormented in order to understand what is wrong with the robot. By 2016, it seemed that Robonaut was disassembled and analyzed simply in pieces.

Astronaut Tim Kopra set up a camera that should record video of the work of Robonaut. He used an oscilloscope, a multimeter and a probe to measure current to help the team on the ground look for the source of the problems, disconnecting the controller card and powering the robot without this card. The data collected today will be analyzed on the ground to determine the causes of power failure.
Excerpt from daily reports from ISS, 03/23/2016

Two controllers cards that the Space Center team suspected of failures were even lowered back to Earth to check. But after checking in the laboratory, it turned out that the maps were in order, and the search for ways to fix it continued - the team checked and rechecked everything, including even the cables outside the robot:

The team conducted a troubleshooting in the case of Robonaut. Previous searches talked about the possibility of having two cards that were returned to the ground and checked. Then they were expelled, and during today's work, the team received a good response from Robonaut during the first boot. Indicator lights corresponded to normal operation. When narrowing the search to determine the causes of power failure, various power cycles were tested. As a result, the team rehabilitated several cables. Cards were not removed from the CPC1 enclosure, and all cables were connected. The teams plan to analyze the results in order to develop a plan for further action.
Excerpt from daily reports from the ISS, 01.02.2017

It may seem surprising that their robot experts at NASA could not identify and fix the problem, because the equipment for space is very carefully designed, created and tested before going into space. But the Robonaut on the ISS is slightly different from the other five Robonauts created in the Space Center. The Cosmic Robonaut represents the R2-B model, and the terrestrial - R2-C, a later version of it, “a significant improvement in the scheme,” as Badger says. In the end, as she said, these differences led the engineers to the correct answer, years after the appearance of the first problem.


Astronauts Pegya Whitson, Shane Kimbrough and Thomas Paske are looking for problems with Robonaut 01.02.2017

“It took us a lot of time to figure everything out, but in the end it turned out that the robot on the ISS did not have enough grounding for the computer case,” she said. - The current found workarounds, which led to a gradual degradation of the robot. Since the robot in orbit was different from those that remained on the ground, this error with the absence of a ground loop was not easy to find. ”

This means that the electric current passed through the body of Robonavt in the wrong way. It is possible that some of its circuits and processors did not receive the necessary power at all, while others received it too much. This deficiency slowly killed Robonavt.


"We are looking for problems with Robonavt together with the ground crew," wrote Tim Kopra on Twitter in March 2016. "He has not yet fully recovered."

Last August, astronauts tried to install a ground jumper, but this did not solve the problems. According to Badger, the place where they wanted to install it was covered with a special sealant that protects sensitive components against shorting with foreign objects that can fly freely on the ISS. The astronauts tried to pick it off and “it was very difficult,” said Badzher. “There were doubts about the effectiveness of this compound. In addition, we were almost certain that the lack of the required wire resulted in other damage as well. ”

It is these, other damage, and cause the most concern. Even if the original grounding problem was resolved, and the Robonavt team is not sure about this, the Robonaut almost certainly suffers because of the significant degradation caused by the current that runs uncontrollably through the hull. It was possible to try to do something else, but when NASA was asked if they needed to use the free space in the cargo hold to return Robonavt, when the nearest mission to restore the reserve of resources at the station would be returned to the ground, the team responded affirmatively. Depending on the degree of damage, the team will either fix Robonavt and send it back to the station, or change it to a new R2-C model.

Badzher could not name the dates in which the Robonaut would again be on the ISS - as long as there are no guaranteed missions and they will have to find free space, which can be difficult, since the Robonaut is relatively large in volume. This means that they may not even be able to attach their legs to it when they send it back, although Badger promises that in this case "they will be much, much easier to assemble."


“Vande Hay and I packed Robonavt for returning to Earth for repair,” wrote Joseph Aqaba on Twitter on February 14 and attached a photo of February 9. "We will wait for his return."

On February 9, Robonauta was packed and prepared for return to earth. The team in Houston is preparing to set up the work of their robot, and Badger reminds us of the importance of remembering the meaning of the mission of Robonaut.

“The whole essence of the ISS is the ability to check something new. “I think that Robonaut gave us a lot of knowledge about what requirements we will have for humanoid robots in the future,” she said. “We are taking him home, repairing, and, in the near future, we hope that he will fly up again and fulfill his initial goals - the promotion of new technologies.”