The neural network works for a very long time as the network size increases

Made a simple neural network in C #. Everything works fine with a small example, and if I create 10 input neurons and even hidden layers 2. Then I have 700 eras before the perfect answer. And each time a different meaning.

I tried to change the hyperparameters, but without success.

Please look at my code and tell me what I did wrong.

And if it's not difficult can anyone explain where I have an error in finding a neural network error. I kind of did everything on the article, but it does not work.

Ps in general, the idea was that 9 values ​​are fed to the input (this is 0 or 1 depending on whether the cell is painted or not). And if the field is 3 by 3 stick vertical, then the neural network says that the stick is vertical well or horizontal.

Here is the link to the article: https://m.habr.com/ru/en/post/312450/

But the code itself:

static Random r; static void Main(string[] args) { //Входные данные double[,] inputData = new double[,] { {1,0,0,1,0,0,1,0,0}, {1,1,1,0,0,0,0,0,0}, {0,1,0,0,1,0,0,1,0}, {0,0,0,1,1,1,0,0,0}, {0,0,1,0,0,1,0,0,1}, {0,0,0,0,0,0,1,1,1} }; /*inputData = new double[,] { {0,1,1}, {0,0,1},{1,0,1},{0,0,0},{1,0,0},{0,1,0},{1,1,0},{1,1,1} };*/ //Идеальные значения double[] outputData = new double[] { 1, 0, 1, 0, 1, 0 }; //outputData = new double[] { 1, 1, 0, 0, 0, 0, 0, 1 }; //Значение момента double a = 0.8; //Скорость обучения double E = 0.8; //Все нейроны double[][,] allNeurons = new double[][,] { new double[10,1], new double[6,2], new double[4,2], new double[1,2] //new double[4, 1], new double[1, 2] }; //Все веса double[][,,] allWeights = new double[][,,] { new double[10,5,2], new double[6,3,2], new double[4,1,2] //new double[4,1,2] }; r = new Random(); //Инициализация весов в промежутке от -0.5 до 0.5 случайным образом for (int i = 0; i < allWeights.GetLength(0); i++) for (int j = 0; j < allWeights[i].GetLength(0); j++) for (int k = 0; k < allWeights[i].GetLength(1); k++) allWeights[i][j, k, 0] = r.NextDouble() - 0.5; double error = 0; for (int epoch = 0; epoch < 1000; epoch++) { Console.ForegroundColor = ConsoleColor.Red; Console.WriteLine(epoch); Console.ResetColor(); for (int stepTrain = 0; stepTrain < inputData.GetLength(0); stepTrain++) { //Инициализация входных нейронов for (int i = 0; i < allNeurons[0].GetLength(0) - 1; i++) allNeurons[0][i, 0] = inputData[stepTrain, i]; //Инициализация нейронов смещения for (int i = 0; i < allNeurons.GetLength(0) - 1; i++) allNeurons[i][allNeurons[i].GetLength(0) - 1, 0] = 1; //Прямое прохождение по всем слоям for (int i = 1; i < allNeurons.GetLength(0); i++) { int lastLayer = 1; if (i == allNeurons.GetLength(0) - 1) lastLayer = 0; for (int k = 0; k < allNeurons[i].GetLength(0) - lastLayer; k++) { for (int j = 0; j < allNeurons[i - 1].GetLength(0); j++) allNeurons[i][k, 0] += allWeights[i - 1][j, k, 0] * allNeurons[i - 1][j, 0]; allNeurons[i][k, 0] = Sigmoid(allNeurons[i][k, 0]); } } //Нахождение ошибки у выходных нейронов int index = allNeurons.GetLength(0) - 1; for (int i = 0; i < allNeurons[index].GetLength(0); i++) allNeurons[index][i, 1] = FixOutError(outputData, allNeurons[index][i, 0], stepTrain); //Нахождение ошибок для скрытых нейронов for (int i = allNeurons.GetLength(0) - 2; i >= 1; i--) { int lastLayer = 1; if (i + 1 == allNeurons.GetLength(0) - 1) lastLayer = 0; for (int j = 0; j < allNeurons[i].GetLength(0); j++) { for (int k = 0; k < allNeurons[i + 1].GetLength(0) - lastLayer; k++) allNeurons[i][j, 1] += allWeights[i][j, k, 0] * allNeurons[i + 1][k, 1]; allNeurons[i][j, 1] = allNeurons[i][j, 1] * (1 - allNeurons[i][j, 0]) * allNeurons[i][j, 0]; } } //Корректировка весов for (int i = 1; i < allNeurons.GetLength(0); i++) { int lastLayer = 1; if (i == allNeurons.GetLength(0) - 1) lastLayer = 0; for (int k = 0; k < allNeurons[i - 1].GetLength(0); k++) for (int j = 0; j < allNeurons[i].GetLength(0) - lastLayer; j++) { allWeights[i - 1][k, j, 1] = E * allNeurons[i - 1][k, 0] * allNeurons[i][j, 1] + a * allWeights[i - 1][k, j, 1]; allWeights[i - 1][k, j, 0] += allWeights[i - 1][k, j, 1]; } } error += Math.Pow(Math.Atan(outputData[stepTrain] - allNeurons[3][0, 1]), 2); Console.WriteLine(allNeurons[3][0, 0]); } Console.ForegroundColor = ConsoleColor.Green; Console.WriteLine(error / outputData.Length); error = 0; Console.ResetColor(); //Thread.Sleep(300); } Console.ReadKey(); } static double FixOutError(double[] outputData, double outputValue, int index) => (outputData[index] - outputValue) * (1 - outputValue) * outputValue; static double Sigmoid(double outputData) => 1 / (1 + Math.Exp(-(outputData))); 

Addition:

I changed the search error to the string

 error += Math.Pow(Math.Atan(outputData[stepTrain] - allNeurons[3][0, 1]), 2); 

on line

 error += Math.Abs(outputData[stepTrain] - allNeurons[3][0, 0]); 

And at the end of an era, I simply divide error by the number of elements in an array of ideal values.

And I noticed that the first 200 eras, the error value is about 0.6 and almost does not change. After 200 epochs, the error begins to fall to 0.04 and then slowly decreases.

How to fix this error when 200 epoch error almost does not change.

Addition:

I changed the random generation of scales from -0.5 to 0.5 on s -1 to 0 and the neural network began to study for 100 epochs and not for 700 or 1400 as it was.

But I do not know if I did it right