I QVariant on a program with data transmission over a network; objects of type QVariant or pairs of QString, QVariant objects QString, QVariant . Prototypes work fine (there is a transmission and reception code for clarity - the QTcpSocket data is beating ). Reception is stable, but the reception code is based on the continuity of the correct data flow. If there are problems in the network, for example, packet drops, the reception fails, because the expected order of transmission is disturbed, and it is impossible to reconnect without reconnecting. The question is what mechanisms to use for ensuring fault tolerance, how to find a new beginning of a block of useful data in case of loss of communication and are there any best practices in this area, as applied to QTcpSocket ?

I emulate network problems with Climsy

Prototype of server program: https://github.com/BeardedBeaver/ServerPrototype

Client program prototype: https://github.com/BeardedBeaver/ClientPrototype

  • one
    TCP protocol is so arranged that the order of bytes is guaranteed. If any package is dropped, it will be re-sent. In modern systems, there is practically no such thing that the sequence of packets is broken in tcp (I have not seen this). In times of more ancient times, when the current was possible (as well as on lower levels), “self-synchronizing sequences” are used. The classic example is utf-8. - KoVadim
  • In theory, @KoVadim is yes, but in practice this does not always happen, and in the presence of network interference, the transmission is broken. In a stable local network, everything happens correctly. So the question is open. On account of self-synchronizing sequences, thanks, read in more detail, looks like something that looks like a solution - Bearded Beaver
  • one
    If bytes deteriorate and forwarding does not help, TCP will break the connection. So the question is closed :). - KoVadim
  • one
    Executable files are unnecessary. They after all hide errors. It's better to show the code that actually reads / writes from / to the socket. And even better - a minimal example that reproduces the situation. - KoVadim
  • one
    in the read code from the socket there is a code while (true) { } is five! (and the second repository is completely empty). - KoVadim

1 answer 1

Qt version 5.7 has such an interesting toolkit as transactional reading. In addition to the claimed integrity check of a series of data sent through a succession of statements to the QDataStream stream, this mechanism also performs a check for consistency of the type and size of each of the received positions.

Thus, data integrity (data series) can be monitored in a simple way:

 void SocketManager::socketSendMessage(const QVariant &var) { QByteArray data; QDataStream stream(&data, QIODevice::WriteOnly); stream.setVersion(QDataStream::Qt_DefaultCompiledVersion); stream << var; _socket->write(data); } void SocketManager::onSocketReceiveMessage() { QDataStream stream(_socket); stream.setVersion(QDataStream::Qt_DefaultCompiledVersion); while(_socket->bytesAvailable()) { stream.startTransaction(); QVariant var; stream >> var; if(stream.commitTransaction() == false) return; // Работаем с данными... } } 

If the data was not completely received or does not meet the expectation (in the example it is QVariant type), then the transaction will not be completed. In this case, the reaction may be different and depends on the imagination of the author of the code. You can wait for some time until the approaching lagged piece. You can increment the counter of unsuccessful attempts, and relying on its value to take certain steps.

If you are not sure that this is enough, you can use an additional mechanism that, although not intended to verify the correctness of the data, can become one. For example, this may be the usual decompression from the archive:

 QByteArray SocketManager::compressData(const QVariant &var) { QByteArray data; QDataStream stream(&data, QIODevice::WriteOnly); stream.setVersion(QDataStream::Qt_DefaultCompiledVersion); stream << var; return qCompress(data, 9); } QVariant SocketManager::uncompressData(const QByteArray &data) { QByteArray raw_data = qUncompress(data); if(raw_data.isEmpty()) return QVariant(); QDataStream stream(raw_data); stream.setVersion(QDataStream::Qt_DefaultCompiledVersion); QVariant var; stream >> var; return var; } 

And of course, it remains to tweak the sending and receiving code a little:

 void SocketManager::socketSendMessage(const QVariant &var) { QByteArray data; QDataStream stream(&data, QIODevice::WriteOnly); stream.setVersion(QDataStream::Qt_DefaultCompiledVersion); stream << compressData(var); _socket->write(data); } void SocketManager::onSocketReceiveMessage() { QDataStream stream(_socket); stream.setVersion(QDataStream::Qt_DefaultCompiledVersion); while(_socket->bytesAvailable()) { stream.startTransaction(); QByteArray data; stream >> data; // Если не докачалось, то ждём новую порцию данных. if(data.isEmpty() == true) { // QByteArray подвержен сериализации также, // как и другие контейнеры со вставкой размера // перед началом данных, поэтому если по факту // данных будет меньше, нежели чем было записано, // то буфер окажется пустой. stream.rollbackTransaction(); return; } // Целостность (соответствие размеру) проверили, // теперь пытаемся проверить корректность. QVariant var = uncompressData(data); // Порядок. if(var.isValid() == true) { // Поскольку "commit" в случае неуспеха // сделает "rollback" и возвратит позицию // чтения на начало транзакции, то в ситуации // с необходимостью отбрасывания повреждённых // данных расположение "commit" до проверки их // корректности не имеет смысла. if(stream.commitTransaction() == false) { // Этот исход вряд ли возможен // после всех проверок. } } // Возникли проблемы с корректностью данных. else { // "abort" вызовет "commit" со сдвигом // позиции на место, находящееся после // повреждённых данных. stream.abortTransaction(); } // Работаем с данными... } } 

In addition to the fact that the size of the information sent can be reduced several times, in the event of data corruption, the likelihood that unpacking will occur successfully will be reduced.

Addition

If we consider transactions with transactions without taking into account nesting (this is also supported), then each of them comes down to the following:

  • startTransaction() - remembers the current read position in the buffer;
  • commitTransaction() - deletes data in the buffer while zeroing the read position;
  • rollbackTransaction() - returns the read position without affecting the data;
  • abortTransaction() is actually the same as commitTransaction() , but will set the status to QDataStream::ReadCorruptData .
  • Thank you, it looks interesting. In my (combat) code, qCompress is already used to reduce the amount of data being transferred, so tracking down incorrect transmission is quite simple. I will study the transaction mechanism. I may have not quite correctly formulated the question - I’m wondering how to be in the case when I recorded an incorrect transfer, that is, something was beating along the way and I need to skip the broken piece, find a new data start and continue reading from the stream in normal mode. - Bearded Beaver
  • @BeardedBeaver, these transactions that I described in the response should help you. QDataStream also has an abortTransaction() method, which can be used if data acquisition failed and you do not need to wait for the failed data to come from the same block, and chop off the appeared "tails" (including clearing the socket buffer). - alexis031182
  • Yeah ... but what if at the same time the beginning of a new data block also lies in the socket? Hack and clean until you get lucky and a new piece of data will not start from the beginning of the block? Or I do not understand the principle of the socket? - Bearded Beaver
  • With abortTransaction() you can continue to read the stream (buffer) further, as if it were a new block. It is necessary to slightly change the answer for this option. - alexis031182
  • > that is, something was beating along the way and I need to skip the broken piece, find a new beginning of the data and continue reading from the stream in the normal mode. In this case, you won the jackpot. > Or do I not understand the principle of the socket? Yes, you seem to not understand him very much. - KoVadim