Have you ever programmed like this?
It’s a project named Sikuli from MIT. From its official website:
Sikuli automates anything you see on the screen. It uses image recognition to identify and control GUI components. It is useful when there is no easy access to a GUI’s internal or source code.
I find it’s a log of fun and a good tool for GUI debugging.