Switch to Boost-1.65.1 and bugs that surfaced
Last year (almost a year has already passed), we still switched to the new version of Boost-1.65.1, and under the hood you will find the three boost bugs that we encountered. It is also important to mention that before that, we used boost -1.62.1 in the software, since some bugs appeared in boost earlier than version 1.65.1.
Our project has a special integration team whose main task is to migrate all software to a new version of libraries, Visual Studio, new versions of low-level components (basic ones that most other components depend on), etc. Also, the integration team is responsible for eliminating all the problems that arise, naturally with the assistance of the component maintainers, if necessary. So, the bugs that I especially remember.
This bug surfaced quickly enough. Tests began to fall with “Access violation” when searching for the full path to the specified file name. The function called the call boost :: filesystem :: exist, and the program crashed. Running a few more tests, several more similar cases were noticed, while in all cases the call to boost :: filesystem :: exist was made for global variables. Apparently, something has changed in the lifetime of the boost-variables. Ticket for the detected bug is very easy google bug bug in boost :: filesystem :: exist
It turned out that this bug was in boost, starting with version 1.64. In fact, the problem was in the make_permissions call (used in filesystem :: exist). In 1.64, the implementation of make_permissions was changed, and now used global variables, which means that when an attempt is made to call filesystem :: exist when initializing a global variable or object, the global variables used in make_permissions may not yet be initialized. Therefore, an attempt to access an uncreated variable throws an exception.
In tests using boost :: python, a strange thing was discovered. When executing a trivial eval () call for a literal (for example, "40 + 2") all the rules. And if variables are defined and then used in expressions, we get the message that undefined variables are used in calculations (ERROR: [name] not defined). To solve this problem, I have spent more time. I could not find the problem ticket in the boost tracker, so I had to ask for help from the command of this component. Information about the bug was promptly found on github .
It so happened that the global and local objects were not used in the implementation of eval. Having wished Good luck in finding fixes without recompiling the source code, the team did not bow :)
This is my least favorite part. boost :: python :: numeric has been removed and now boost :: python :: numpy appeared as an alternative. But the code that used numeric had to be pretty much redone, since it’s not just a matter of renaming namespaces, but also of object implementation.
In addition, there was misinformation in the boost header that misled me.
According to the source comment, the import_array () call is already being done in numpy :: initialize ():
But in fact, as it turned out, import_array () is required.
In addition, there were problems with testing the changes, since pieces of code with numpy (previously with boost :: python :: numeric) were not covered at all by tests, and the code itself was also used in another component. Therefore, problems were detected only when testing the corresponding component. The integration team is not required to write tests for the components, and this situation was the omission of the command itself. Wow, and I heard a lot from them that broke their code. But after the team grumbled, they finally covered their code with tests. However, the offense remained (during the next migration, the team did not want to give access to its component to my colleague, mentioning that last time we broke the code for them. Sasha, weeds! But after three days of negotiations, they surrendered).
After this work, I can note the advantages for myself, since I had not used boost very often before (mostly std), so many new things can be emphasized from migration. It's funny, but it’s a fact, for some reason after this you become a “boost expert” for many colleagues, and, accept, you will be asked questions about it for some time.
By the way, in recent years, many companies have begun to actively get rid of boost and, if possible, replace std with a library or something else if there are no possibilities in the standard library. And we also did not stand aside. The process started, but not completed, still a lot of work.
Our project has a special integration team whose main task is to migrate all software to a new version of libraries, Visual Studio, new versions of low-level components (basic ones that most other components depend on), etc. Also, the integration team is responsible for eliminating all the problems that arise, naturally with the assistance of the component maintainers, if necessary. So, the bugs that I especially remember.
Bug in boost :: filesystem
This bug surfaced quickly enough. Tests began to fall with “Access violation” when searching for the full path to the specified file name. The function called the call boost :: filesystem :: exist, and the program crashed. Running a few more tests, several more similar cases were noticed, while in all cases the call to boost :: filesystem :: exist was made for global variables. Apparently, something has changed in the lifetime of the boost-variables. Ticket for the detected bug is very easy google bug bug in boost :: filesystem :: exist
It turned out that this bug was in boost, starting with version 1.64. In fact, the problem was in the make_permissions call (used in filesystem :: exist). In 1.64, the implementation of make_permissions was changed, and now used global variables, which means that when an attempt is made to call filesystem :: exist when initializing a global variable or object, the global variables used in make_permissions may not yet be initialized. Therefore, an attempt to access an uncreated variable throws an exception.
Workaround
For tests where global variables were used only once, they were transferred to the corresponding tests and became local variables. Do not even ask why this was not done before, I am not the maintainer of this code.
In other cases, singletons were used.
In other cases, singletons were used.
Bug in boost :: python
In tests using boost :: python, a strange thing was discovered. When executing a trivial eval () call for a literal (for example, "40 + 2") all the rules. And if variables are defined and then used in expressions, we get the message that undefined variables are used in calculations (ERROR: [name] not defined). To solve this problem, I have spent more time. I could not find the problem ticket in the boost tracker, so I had to ask for help from the command of this component. Information about the bug was promptly found on github .
It so happened that the global and local objects were not used in the implementation of eval. Having wished Good luck in finding fixes without recompiling the source code, the team did not bow :)
Workaround
But then I remembered the release notes for boost-1.65.1 and there was definitely something for boost :: python.
Hurray, there is a way! The bug was allowed when adding a new implementation of eval with support for char const * argument, which is now called in the old implementation of eval with a string argument (Particularly attentive could notice the call of this function in the code by github-ovsky link). And the new feature, as expected, is working.
Hurray, there is a way! The bug was allowed when adding a new implementation of eval with support for char const * argument, which is now called in the old implementation of eval with a string argument (Particularly attentive could notice the call of this function in the code by github-ovsky link). And the new feature, as expected, is working.
boost :: numpy
This is my least favorite part. boost :: python :: numeric has been removed and now boost :: python :: numpy appeared as an alternative. But the code that used numeric had to be pretty much redone, since it’s not just a matter of renaming namespaces, but also of object implementation.
In addition, there was misinformation in the boost header that misled me.
According to the source comment, the import_array () call is already being done in numpy :: initialize ():
namespace boost { namespace python { namespace numpy { /** * @brief Initialize the Numpy C-API * * This must be called before using anything in boost.numpy; * It should probably be the first line inside BOOST_PYTHON_MODULE. * * @internal This just calls the Numpy C-API functions "import_array()" * and "import_ufunc()", and then calls * dtype::register_scalar_converters(). */ BOOST_NUMPY_DECL void initialize(bool register_scalar_converters=true); }}} // namespace boost::python::numpy But in fact, as it turned out, import_array () is required.
In addition, there were problems with testing the changes, since pieces of code with numpy (previously with boost :: python :: numeric) were not covered at all by tests, and the code itself was also used in another component. Therefore, problems were detected only when testing the corresponding component. The integration team is not required to write tests for the components, and this situation was the omission of the command itself. Wow, and I heard a lot from them that broke their code. But after the team grumbled, they finally covered their code with tests. However, the offense remained (during the next migration, the team did not want to give access to its component to my colleague, mentioning that last time we broke the code for them. Sasha, weeds! But after three days of negotiations, they surrendered).
Conclusion
After this work, I can note the advantages for myself, since I had not used boost very often before (mostly std), so many new things can be emphasized from migration. It's funny, but it’s a fact, for some reason after this you become a “boost expert” for many colleagues, and, accept, you will be asked questions about it for some time.
By the way, in recent years, many companies have begun to actively get rid of boost and, if possible, replace std with a library or something else if there are no possibilities in the standard library. And we also did not stand aside. The process started, but not completed, still a lot of work.
Source: https://habr.com/ru/post/436828/