Is it advisable to use a convolutional neural network for pattern recognition tasks?

According to the following English-language article and the corresponding presentation , the use of an ensemble of convolutional neural networks leads to a significant reduction in errors. Shown on the example of pattern recognition numbers from the MNIST database .

It is shown that with an increase in the number of models in the ensemble, the recognition accuracy increases: enter image description here An ensemble algorithm can lead not only to an improvement in the recognition accuracy, but also to a decrease in the time spent on training.

I could not find the year of publication of this article, as well as the degree of its author. It is possible that the use of the ensemble of convolutional neural networks is no longer relevant today or has shown its inconsistency in pattern recognition problems.

I also drew attention to the following Russian-language article dating from 2012:

Current issues of using convolutional neural networks and their committees in recognizing digit patterns (Kuzmitsky NN, 2012, pdf)

It also recognizes digit patterns from various databases. The article and the work clearly demonstrated the effectiveness of the use of CNN committees trained on bases with different styles of style.

Here is one of the tables comparing the accuracy of recognition by the committees of neural networks and KADMOS systems:

enter image description here

But I myself came across opposing opinions of experts who argued that the use of convolutional neural networks ensembles is not relevant today. According to their opinion, today the solution of pattern recognition problems is more concentrated on optimizing one model of a neural network than on using their ensembles.

And nevertheless how are things going in this question? At the moment, are there any significant advantages of using an ensemble of convolutional neural networks in front of the variant with one neural network (without an ensemble) in pattern recognition problems or not?

  • one
    Too scientific question, I'm afraid there are more engineers. If you look from an engineering point of view - for example, there is a task to recognize something, the algorithms are taken: segmentation, correlation, neural networks, work with graphs (graph theory), post-processing of images. How these algorithms communicate with each other depends on a specific task: depending on what to recognize. And of course, often for specific tasks, there are already ready-made solutions, or close to ready-made ones (can be used as part of your algorithm). - Goncharov Alexander

1 answer 1

This answer is a translation of the response from Franck Dernoncourt in the next question in English. language.

At the moment, are there any significant advantages of using an ensemble of convolutional neural networks before the version with one neural network (without an ensemble) in pattern recognition problems?

Empirically, convolutional neural network ensembles (CNNs) often provide some advantage over a single convolutional neural network.

An example of image classification is given in the article by Krizhevsky Alex, Ilya Sutskever and Geoffrey E. Hinton:

" ImageNet classification with deep convolutional neural networks / ImageNet image classification using convolutional neural networks (ENG, 2012, PDF) "

It contains the following table:

enter image description here Translation: Comparison of the error rate on ILSVRC-2012 on the test / test (validation) and test (test) sets. The best results achieved by other algorithms are in italics . Models with an asterisk * were pre-trained to classify the entire ImageNet 2011 image database (autumn release) . See Section 6 for more details.

On the example of machine translation (using LSTM networks ) there is an article by the authors: Sutskever Ilya, Oriol Vinyals, and Quoc V. Le:

Sequence to sequence learning with neural networks / sequence-to-sequence training with neural networks (ENG, 2014, PDF)

The article is a table:

enter image description here Translation: The effectiveness of LSTM on WMT 2014 on the test set " From English to French (ntst14) ". Please note that an ensemble of 5 LSTM networks with a beam size (beam size 2 ) is cheaper than one LSTM network with a beam size of 12 .