Pay attention to the system of automated testing applications. A good article about them. There are a great many of them, there are expensive ones, there are open sources, for different platforms, etc. The most famous
- MS Coded UI - for desktop applications
- Selenium (for web applications)
- Appium (for mobile applications)
All of them are based on some macro scripting languages, and one of the main tasks of these macro commands is to get into the controls with the help of a mouse, enter text, select options from lists, and so on, that is, emulate the work of real users.
The controls identify these systems in different ways:
If the program under study already exists and does not change during operation - the script needs to be written 1 time, it will expand the program to full screen and get perfectly well even without knowing the names of the controls (by coordinates).
If the program changes rapidly (especially in the part of UI) - QA automation scripts lose their value dramatically, the question of their relevance in general before UI stabilization is correct.