Side Project: ScreenWatcher – Alerts for Anything

Bill Gates once said:

“I choose a lazy person to do a hard job. Because a lazy person will find an easy way to do it.”

Ok, now that I’ve fulfilled the vaguely related quote requirement for a blog post we can get down to business.

We’ve all been there, you start a report running, a file downloading or a program installing and you know it’s going to take a while to finish. Some systems have the functionality to send you an email once finished but the vast majority of them don’t plus it’s quite limited.

So what you end up doing is keep looking/walking back over to the screen to check how far it’s got and getting disappointed how far it’s not moved, like the proverbial watched kettle.

To alleviate this first world problem, I’ve built a quick widget to watch the screen for me and to alert me if it changes.

It works like this (Google’s built in countdown timer used as an example of a progress bar):

• Open the widget – a screenshot is taken (multiple screens work too)

• The widget shows you the screenshot in a scrolling window – select the area of interest by dragging a rectangle around it, this is the area that doesn’t change until the action you want to monitor is done.

• After clicking OK the widget disappears but it’s actually still running and constantly monitoring the area of interest comparing the colour of the pixels to the original version.
• Once they change the widget announces the event to the very cool automation system IFTTT (If This Then That) along with the optional details entered earlier. From here you can do almost anything, have your phone get a notification, send an SMS, turn on/off a light/plug, play a song, etc.

Here it is in action:

 

Challenges (this bits a bit technical if you want to skip it)

• Handling multi monitors is a real pain. The Java library used to capture screenshots doesn’t easily deal with above and below layouts very well. I think I’ve solved this by storing the lowest possible X and Y value for the top left corner of any monitor (it goes negative if it’s above and/or left of the primary display) and then requesting a screenshot of a giant rectangle from that top corner to the total of all the widths and heights combined.
• Checking the RGB value of a pixel on the screen is computationally expensive. Each check takes around 20ms it seems. So if you have a 20×20 grid of pixels to check that’s 20x20x20=8000ms; a whole 8 seconds! I’d actually had similar problems before back at uni doing computer vision work. What I’ve done is give the system a max number of pixels to search (50) and have it skip over pixels in the grid in a predictable way so that only that number is ever checked. Luckily in all the tests I’ve done at least one of these pixels in the search area has changed allowing for the system to pick it up.

Now this didn’t take very long to code up and it’s mainly based on bolted together StackOverflow answers but it does the job very well indeed and acts as a nice proof of concept for other ideas and tools.