There are 2 dictionaries: Train and Test, which have the same keys.

Train dictionary:

Train: = {Key1: [A, B, C], Key2: [D, E, F], Key3: [G, H, I]}

Test dictionary:

 Test = {Key1: [J, K], Key2: [L, M], Key3: [O, P]}

I need to compare each element in the Test dictionary with all the elements of all the keys of the Train dictionary.
In other words, it is necessary to find the distance by the algorithm between the matrices, which are the elements of dictionaries.

I.e:
J is compared with A, we get a number, then J with B, J with C, and then we need to go to the next Train dictionary key and compare J with D and so on. At the end J is compared with I and after that we compare the element K of the Test dictionary with all elements of the Train dictionary.

I tried the following code:

 for class in Train: dist[class] = [] for Test1 in Test[class]: this_row = [] for Train1 in Train[class]: ("some func" dist = dist(Train1, Test)) this_row.append(dist) Ddis[class].append(this_row)

At the same time, I compare the elements only inside 1 key, which does not solve my problem. Has anyone come across this?

  • Do your real data use strings or numbers? And which metric is used to measure distance? Specify what you want to get at the exit? - MaxU
  • In real data, each element in the dictionaries is a matrix. The metric by which the distance is measured is the DTW algorithm (dynamic transformation of the time scale), 2 matrices are taken at its input (2 vectors in the one-dimensional version), the output is the distance between 2 matrices. As for my question - on exit I want to get a distance matrix, where the rows are the elements of the Test dictionary, and the columns are the elements of the Train dictionary - R0stislav
  • It would be better if you give an example of data closer to your real data (of course in a smaller version). If I understand you correctly, your data can be presented as a 3D matrix - did I understand you correctly? - MaxU
  • In short - the task of word recognition using the DTW method. The spoken word is divided into frames (M frames of 20 ms), and when processing each frame, I get the MFCC vector (N MFCC coefficients). Thus, each word is represented as a matrix of size (N x M), and N is always constant, and M is changing (the length of the words is different). Therefore, DTW, that he calculates the difference between different lengths of pronunciation. So I get a training and test sample of 3 classes (keys) with 3 elements (pronunciations of this word). It is necessary to correlate the test word with the training sample - to recognize the word. - R0stislav
  • As a rule, the number of frames varies from 20 to 45. The number of MFCC coefficients (a special measure of sound perception, translated into a set of values) is 13 elements. In general, ideally, it would be to obtain a matrix of distances between the test word and the trainer, and the minimum value in the line would mean the class to which the test word belongs. - R0stislav

2 answers 2

 # -*- coding: utf-8 -*- """ Created on Tue Apr 24 22:16:43 2018 @author: Rostislav """ import numpy as np # ============================================================================= # Имитируем обучающие и тестовые выборки # 5 раз произнесли слово "Вставить", обработали и получили # на выходе 5 фреймов по 4 уникальных коэффициента в каждом фрейме # причем, 3 произношения будут относиться к обучающей выборке, а 2 к тестовой # Все то же самое делаем со словами "Копировать" и "Открыть" # ============================================================================= VstTrain1 = np.random.randint(-100, 100, (4,5)) VstTrain2 = np.random.randint(-100, 100, (4,5)) VstTrain3 = np.random.randint(-100, 100, (4,5)) KopTrain1 = np.random.randint(-100, 100, (4,5)) KopTrain2 = np.random.randint(-100, 100, (4,5)) KopTrain3 = np.random.randint(-100, 100, (4,5)) OtkTrain1 = np.random.randint(-100, 100, (4,5)) OtkTrain2 = np.random.randint(-100, 100, (4,5)) OtkTrain3 = np.random.randint(-100, 100, (4,5)) VstTest1 = np.random.randint(-100, 100, (4,5)) VstTest2 = np.random.randint(-100, 100, (4,5)) KopTest1 = np.random.randint(-100, 100, (4,5)) KopTest2 = np.random.randint(-100, 100, (4,5)) OtkTest1 = np.random.randint(-100, 100, (4,5)) OtkTest2 = np.random.randint(-100, 100, (4,5)) # ============================================================================= # Составим 2 словаря с одинаковыми ключами, где элементы словаря представлены # матрицами, имитирующими обучающую и тестовую выборки # ============================================================================= Train = {'Вставить':[VstTrain1, VstTrain2, VstTrain3], 'Копировать':[KopTrain1, KopTrain2, KopTrain3], 'Открыть':[OtkTrain1, OtkTrain2, OtkTrain3]} Test = {'Вставить':[VstTest1, VstTest2], 'Копировать':[KopTest1, KopTest2], 'Открыть':[OtkTest1, OtkTest2]} Ddist = dict() for class_name in Train: Ddist[class_name] = [] for Test1 in Test[class_name]: this_row = [] for Train1 in Train[class_name]: dist = np.linalg.norm(Train1 - Test1) #Вместо DTW будем вычислять норму матриц this_row.append(dist) Ddist[class_name].append(this_row)

As a result, I find the distance (norm) between the test matrix and all the matrices in the training set only inside the key, and it is necessary that the test matrix first finds the norm with all the elements of the training set inside the first key, then finds the norm with all the elements of the second and third training set key. Then the next element of the test sample (VstTest2 matrix) again searches for the norms for all elements of all keys. And so with the entire test sample.
As a result, you need to get either a 6x9 matrix (6 test words (matrices) are compared with 9 words (matrices) from the training sample), where the matrix elements are the distance between the test word i and the training word j, or in cycles to keep the key at the minimum rate and correlate test word (matrix) with this key.

    Thinking today about the task came to the following decision:

     Ddist = dict() for class_nameTest in Test: Ddist[class_nameTest] = [] for Test1 in Test[class_nameTest]: this_row = [] for class_nameTrain in Train: for Train1 in Train[class_nameTrain]: np.random.seed(777) # Для одинаковых результатов у пользователей dist = np.linalg.norm(Train1 - Test1) #Вместо DTW будем вычислять норму матриц this_row.append(dist) Ddist[class_nameTest].append(this_row)

    The result is a dictionary, where the keys are classes, and the elements in the keys are vectors from numbers that reflect the distance between the matrices (or in my case, the measure of proximity between the test word and all words from the training set). Next was the task of finding the minimum value of the vector in each element of each key. At the same time, I found the index of the minimum element and it is equal to the ordinal number of the element in the training set, but without reference to the key.

     MinDist = dict() MinIndex = dict() for class_name in Ddist: this_rowMin = [] this_rowIndex = [] MinDist[class_name] = [] MinIndex[class_name] = [] for test_class in Ddist[class_name]: Min = min(test_class) this_rowMin.append(Min) Index = test_class.index(min(test_class)) this_rowIndex.append(Index) MinDist[class_name].append(this_rowMin) MinIndex[class_name].append(this_rowIndex)

    As a result, I know that the test sample element has the smallest distance with some element from the training sample, I know the sequence number of this element of the training sample, but at the same time I do not understand how you can match the key (ie class). For example, take a word from the test sample "Copy2", it has a minimum distance with 3 elements (index = 2), and 3 elements is "Paste3".
    How to get the dictionary key by the minimum distance index number, if this number is not tied to the dictionary?