SikuliX

SikuliX automates anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. It uses image recognition powered by OpenCV to identify GUI components and can act on them with mouse and keyboard actions.

This particular tool has a visual approach to GUI automation using screenshots. It allows you to take a screenshot of GUI elements—toolbar, icon, or dialog box—and ask the system/machine for help to automate the handling of elements and their actions using screenshots instead of element names. SikuliX also has an integrated visual scripting API that helps automate GUI iterations using image models for mouse and keyboard events/actions. On top of that, SikuliX has optical character recognition (OCR) features that can be used to search for text in images.