There is a game. The character of the game says a lot of different replicas (~ 400 pieces). They are displayed as text in the upper left corner of the screen. Examples:

alt textalt text

The task in the game to catch 250 different replicas. My task is to keep track of what is already caught, the number of unique replicas caught. While playing, I take screenshots. I often make duplicates, because I don’t remember whether I saw this phrase or not.

Those. At the entrance there is a folder with screenshots. All screenshots are of the same type. The text is always in the same place (you can select it as a rectangle). Lines can be one or two. The font, color, etc. are always the same. Text only in English. Font unknown. The background is not quite black, it is slightly transparent (something appears from the game through it).

I see two approaches:

1. Option times
Remove duplicates. Run through all the screenshots, select a text area, compare in pairs. Something like this.

2. Option two
Recognize all these texts, save line by line in the textbook. What to do with the texts is clear.

Question
I am not a programmer, I last coded at the university, now I am creating HTML. Nevertheless, the task does not seem difficult to me, plus simplifications / omissions are acceptable, as long as it works. Familiar with C / C ++ from university courses. "Hello, World!" wrote what to do next? :)

Answer
I understand that I need a library for working with images. Probably I need to advise something appropriate for the task. So that I could begin to google.
In general, to describe the algorithm, such as "first learn to open files", "then learn to select the desired area", "after you learn to sprinkle a picture", "as a result with the help of technology X learn to display text". And, of course, what to read?

Thank!

  • 2
    Is the font and its color always the same? - avp
  • I will study. Everything has its time :) File format JPG. Here is the original screenshot: dl.dropboxusercontent.com/u/22619948/2015-01-17-044602.jpg Windows 7 is on your computer. Font size and color are always the same. The family itself is unknown. - Cypher
  • 2
    It all depends on your desire. I probably did not immediately rush with pluses, considering that they were coded for a long time ago and you know badly. I would take a python - it's right there. To it any built-in image processing library, for example, PIL (built-in) and write yourself a simple recognition (you can manage it in a day). And you can search for libraries for OCR. In general, I first recommended reading the articles on how to recognize captcha. This task is very close to her. After receiving the text, you can apply fuzzy text comparison algorithms and find duplicates. - KoVadim
  • Pro captcha is a good idea, thank you. Python? Python I have never seen live :) But once I’m going to go read about the python. - Cypher

2 answers 2

Use the OpenCV library, the easiest way to work with images, in my opinion ... Due to the documentation, examples, and support for several languages.

First of all we cut out a rectangle with the text.

Secondly, we leave only those pixels that are white (as far as I understand, the background changes, but the text is always white).

Third, reduce the size ... I would suggest that every second row and column from the image be thrown out a couple of times so that the text remains readable.

And then we find some hash function in the image. And then it will be necessary to compare with a set of numbers. I can write a little more detail, if at some point is not clear.

UPD Updates

1) OpenCV - once installed and then there will be no problems.

2) “Delete” the background is necessary so that all the resulting pictures would be strictly the same. This will ensure that you do not confuse the two pictures, even if you compare pixel by pixel.

3) The priority is to reduce the size in order to reduce collisions when creating a hash function. But yes, it should greatly improve performance. (If you throw out every second line twice and throw out every second column 2 times, then this data size will decrease by about 16 times)

4) After processing, we have black and white rectangles of the same size. You can make one big line of 0 and 1. Of course, you can store such lines and compare them, but this is also not very efficient.

The most efficient hash function for strings is a polynomial hash. Here is a good article with source code.

  • Thank. OpenCV tried today. Half a day ran an example from the installer package. Almost launched :) With a rectangle, everything is clear. Why separate pixels from the background? The background there is something like black with transparency at 10%. Why reduce the size? To work faster with a large screenshots pack? The last stage, about hash functions, I will probably think separately when I get there. - Cypher
  • However, it is better not to forget about collisions during hashing (identical hash functions for different data). It may be worthwhile to look for image-specific hashing, rather than character strings. - avp
  • @avp, even if you think about reliability, then if you calculate a hash modulo an order of 2 ^ 64 (but of course simple), then the probability of stumbling upon a collision is small. You can still reduce this probability by counting at the same time two hashes for various reasons. Well, or option, if the hashes match, then already check the images themselves. - BogolyubskiyAlexey
  • Tex, OpenCV installed, reinstalled the studio for 2013. I launched a test example, learning to open, edit and resave images. While moving in this direction. - Cypher
  • one
    @BogolyubskiyAlexey, of course, that if the hashes match, then it is necessary to compare byte-by-byte. As for specific hashes, I am sure that they are. For example, Hash Functions for Duplicate Image Retrieval . Or perhaps trivial MD5 or SHA1 will give a good result. - avp

There is a program sharpened to create bots. Cybor There is a text recognition feature. You can recognize the text directly from the screen in games. The generated script is generated. You can even create such a script without programming. For this there is a visual code editor.

Customize text recognition

Script creation

Insert script for execution

Script execution. Text recognition in the specified area

As I said, you can create a program that will recognize the text in the specified area and do what you need with it. Write to a file, display a message if the required text is found, etc. Such a script can be created without even knowing programming. In the visual code editor. These videos show the operation and customization of text recognition.

work and customize text recognition

This video shows the process of creating a ready-made bot for the game, where you need to recognize the text. A bot is created without programming

Optical text recognition in Cybor Boat for Music Wars

Cybor itself Cybor