Signal Interception in Python

Question

There are two programs that communicate with each other via named pipes. One in C ++, the second in Python. And the first one starts the second (in the standard way, via fork + exec).

Communication section of the parent program (C ++):

// ... int pipeDescr; std::string outputPipeName{"inputPipe"}; std::string inputPipeName{"outputPipe"}; char *message = new char[BUFSIZE]; // ... while (true) { int bytesNumber = 0; // ... if ((pipeDescr = open(outputPipeName.c_str(), O_WRONLY)) <= 0) break; bytesNumber = write(pipeDescr, message, strlen(message)); if (bytesNumber <= 0) break; close(pipeDescr); message[0] = '\0'; if ((pipeDescr = open(inputPipeName.c_str(), O_RDONLY)) <= 0) break; bytesNumber = read(pipeDescr, message, BUFSIZE); if (bytesNumber <= 0) break; close(pipeDescr); // ... }

In a separate thread, the watchDog function works, which is responsible for the operation of the child application in Python.

 void watchDog(int clientSocket, pid_t pid, bool &stopWatchDog) { while (true) { // Дочерняя программа завершилась с ошибкой if (waitpid(pid, NULL, WNOHANG) != 0) { // ... } // Родительская программа закрывается mutexClosing.lock(); if (closing) { mutexClosing.unlock(); if (waitpid(pid, NULL, WNOHANG) == 0) { kill(pid, SIGTERM); waitpid(pid, NULL, 0); } break; } mutexClosing.unlock(); // Программное отключение WatchDog-а mutexWatchDog.lock(); if (stopWatchDog) { if (waitpid(pid, NULL, WNOHANG) == 0) { kill(pid, SIGTERM); waitpid(pid, NULL, 0); } mutexWatchDog.unlock(); break; } mutexWatchDog.unlock(); } }

There are three outcomes:

child program crash
completion of the parent program
completing a child program without terminating the parent

In the last two options, you need to smoothly terminate the child program, so I send her a SIGTERM signal and wait for completion.

Communication section of the child program (Python):

 def sigterm_handler(signal, frame): print('\nGot sigterm!\n') sys.exit(0) def main(): input_pipe_name = "inputPipe" output_pipe_name = "outputPipe" # ... signal.signal(signal.SIGTERM, sigterm_handler) # ... while True: pipe_descr = os.open(input_pipe_name, os.O_RDONLY) request = os.read(pipe_descr, 10000) os.close(pipe_descr) reply = work_func(request) pipe_descr = os.open(output_pipe_name, os.O_WRONLY) os.write(pipe_descr, bytes(reply, 'UTF-8')) os.close(pipe_descr)

In the code I added signal processing SIGTERM. However, when this signal is transmitted, the sigterm_handler function is not called.

But! If you write something like this:

 def main(): signal.signal(signal.SIGTERM, sigterm_handler) while True: print('waiting...') time.sleep(2)

That function will be called.

Tell me how to solve this problem!

after while True: add time.sleep(1) , and check, please.
But in this case, if the program gets on waiting for data when reading from a pipe, the signal will not work?
but you can try inserting a wait in C ++ and see the result: if it works, it is already there to process for an exception.

Accepted Answer · 2016-10-03T17:20:02

When receiving a signal, the handler in C sets the flag and immediately ends (hereinafter I describe the Python implementation). The handler written in Python is executed only when the control returns to the main interpreter thread, which happens later (for example, on the next bytecode ) or never. Quote from official documentation :

A python signal handler doesn’t get a low level (C) signal handler. Signal of the corresponding signal on the signal line.

When control returns, it depends on where the execution takes place at the moment (different behavior is possible for different functions on different versions of Python on different platforms). For example, os.open() on a POSIX system in Python 3.5, comes down to :

 do { Py_BEGIN_ALLOW_THREADS fd = open(path->narrow, flags, mode); Py_END_ALLOW_THREADS } while (fd < 0 && errno == EINTR && !(async_err = PyErr_CheckSignals()));

Py_BEGIN_ALLOW_THREADS macro releases the GIL, which allows other threads to execute Python code while the current thread is blocked on the open(2) system call .

When setting its signal.signal() handler, the call resets the SA_RESTART flag so system calls such as open(2) are interrupted by a signal and return an EINTR , which in this case calls the PyErr_CheckSignals() function , which does nothing if the call is not from the main thread the program. In the main thread, PyErr_CheckSignals() checks whether the signal was (according to the flag set by the C handler) and calls the handler written in Python. If the handler PyErr_CheckSignals() exception, PyErr_CheckSignals() returns non-zero and the loop is interrupted, which leads to an exception at the os.open call os.open in Python code when os.open was called from the main thread (otherwise PyErr_CheckSignals() returns 0 ).

In other cases, there are many possible options: GIL released / not released (in C code), whether the blocking call itself is automatically interrupted / restarted (from the platform, C library, Python version may depend), whether PyErr_CheckSignals() is called in the main thread, is not reset whether there is somewhere a flag that the signal occurred before the call to the handler ( signal(SIGINT, custom_handler) does not work on Windows on Python 2.7 ).

If you cannot change your blocking C code so that it PyErr_CheckSignals() , as os.open does when it os.open a signal, to work around it: call the blocking code in the background thread, and in the main thread, sleep a little will not save if your C for CPython extension does not release GIL, as for example the re module can do):

 import threading background_thread = threading.Thread(target=fifo_loop) background_thread.daemon = True background_thread.start() while background_thread.is_alive(): background_thread.join(1) # здесь никакого другого кода, это весь цикл

Note that this is different from the suggestion to add time.sleep(1) to your loop, which works with FIFO (7) . If the signal happens outside the time.sleep(1) call in your fifo loop, the problem will remain.

The workaround works because the fifo loop is executed in the background thread, and the main thread only sleeps intermittently. In Python 3, you can use background_thread.join() without a timeout ( in Python 2, this .join() not interrupted by signals ).

In general, it works, but there are times when the script "hangs" on sys.exit (0) ... So far, I have not understood why this is happening.
@AccumPlus sys.exit () works as a raise SystemExit (stdout is displayed until the end, atexit registered handlers are executed) - it works only from the main thread (not daemon threads continue to live).
Such things are better in the form of a separate Stack Overflow question to ask with minimal code example (no need to drag all C ++, Python code, if the problem can be demonstrated with lower code)

Senior pomidor senior pomidor 7.177 2 14 35 · Answer 2 · 2016-10-03T13:47:14

after while True: you need to add time.sleep(1) , since you need to wait a bit to give a signal to the process.

ps taken from the comments

time.sleep(1) at best make the problem less likely, but it does not eliminate it.

Signal Interception in Python

2 answers 2

More articles: