Eyes down. Bingo!

A newly developed utility uses machine learning to help ascertain whether or not target websites have a security vulnerability, using nothing more than screenshots.

The tool, dubbed ‘Eyeballer’, is designed to help pen testers and bug bounty hunters quickly identify which websites are “interesting” (and which ones aren’t) when looking at a large-scale external perimeter.

Eyeballer, which was designed by security researcher Dan Petro and Gavin Stroy of Bishop Fox, doesn’t actually “hack into” anything.

“Its whole job is to look at screenshots of websites and identify the ones that are most likely to contain actionable leads for the human hacker,” Petro told The Daily Swig.

But how accurate are the results from this tool in practice?

The researchers report that their latest models are hitting a benchmark of around 92% overall accuracy on an evaluation dataset.

Petro explained what this meant in practice: “If we look at 100 images, Eyeballer will label 92 correctly. When we measure accuracy, we use a set of evaluation data exclusively created for this purpose. [One in five] 20% of our screenshots are held back and not used in training.

“That way when we want to evaluate the effectiveness of a model, we can test it against this set of data that we know the answers to, but the program hasn't ever seen before,” he added.

Eyeballer, which is more suited towards looking for low-hanging fruit rather than esoteric vulnerabilities, does more than simply state that a particular site is likely vulnerable by presenting a run-down of possible problems.

More specifically, Eyeballer tags images with one or more labels that are of specific value to pen testers: things that human beings typically are looking for during large scale external engagements.

It takes into account several parameters, including ‘Is the site old-looking?’, ‘Does it have login functionality?’, ‘Is this the homepage of the app?’, and ‘Is this a custom 404 page?’

In particular, finding websites that “look old” can be extremely valuable when trying to break in, the researchers said.

“Old websites have a distinct look and feel that is hard to pinpoint an exact definition for, and impossible to make a traditional signature on,” Stroy explained, “yet they’re extremely valuable targets for pen testers.”

“Having AI that can identify ‘old-looking’ websites has proven to be very useful,” they concluded.

Petro and Stroy unveiled the tool during an Arsenal session of the Black Hat conference in Las Vegas earlier today (August 8).

“Eyeballer is designed to be a practical pen testing tool that we as security professionals would actually use in the real world, as opposed to a cool tech demo or curiosity of purely academic interest,” Stroy told The Daily Swig.

“Naturally, we shot for a very approachable problem as opposed to trying to make an ‘all-in-one’ hacking Cylon.”

“One of the main features of Eyeballer is that it actually works and isn't ‘just’ a sign of things to come,” he added.