Searchable database designed to make cloud security faster and simpler

UPDATED Hundreds of thousands of potentially sensitive files are publically available through open Amazon buckets, a new online tool can reveal.

The free tool, created by software engineer GrayhatWarfare, is a searchable database where a current list of 48,623 open S3 buckets can be found.

Amazon’s S3 cloud storage, or Simple Storage Service, is used by the private and public sector alike as a popular way to cache content. Files are allocated buckets, which are secured and private by default, but can easily be set for public access.

While it is perfectly acceptable to set S3 buckets as available for all to read, numerous data breaches have been the result of an administrator’s misconfiguration.

In March of this year, for example, an unsecured bucket at a US-based jewelry company resulted in the exposure of the personal details of over 1.3 million people, including addresses, emails, and IP identifiers.

Bob Diachenko of Kromtech Security was the first to report the incident, and has helped create a tool aimed at detecting bucket permissions, similar to the one created by GrayhatWarefare.

“On the one hand, it [GrayhatWarfare’s tool] follows the same path as Shodan does,” Diachenko told The Daily Swig.

“It gives researchers and the general audience a possibility to check if their infrastructure is safe. At the same time, it opens doors for ‘passwords-seekers’ and people with malicious intents to leverage upon data found in this ‘Semsem’ cave.”

Plugging the leak

Projects detecting open S3 buckets have rightly surfaced as the incidents related to poor security practice have continued to persevere.

As GrayhatWarefare points out in a blog post, these tools have been slow, too broad, and often provide information that ends up being useless to pen testers and those searching for vulnerabilities.

“Other projects were great, but what I didn't like was that they index only 1,000 results, while there are buckets with millions of results,“ he told The Daily Swig.

“Also they list mostly pictures files, which might be interesting some times, but most of the times are article images, social media images, etc, which on their own do not provide any good info. Also including them in the search engine, enlarges the database a lot, which introduces many performance and management problems.“

This recent tool, presently running on GrayhatWarfare servers, is meant to make the job of securing cloud infrastructure faster and simpler with an automated list of buckets inclusive of the most “interesting” files – some that have been made public on purpose, and others that should most certainly revert to private access.

This inevitably brings up ethics surrounding the creation of such tools, under the notion that they could be used by malicious actors looking for an easy score.

“I personally don’t like the idea of having all publicly accessible buckets listed in one place,” said Diachenko.

“Just imagine having a Troy Hunt’s Have I Been Pwned project with a list of account-passwords combinations – that’s the opposite for a good intention. Implementing a simple string search would be good or ethical enough.”

Further updates to the tool, GrayhatWarfare said, will include subdomains pointing to expired buckets, lists of exposed version control and cracked passwords, and vulnerable cameras and IoT devices.

“The ultimate goal (or mission) of this [GrayhatWarfare] project seems to be public awareness on security issues of S3 buckets naming and indexing,” said Diachenko.

“Amazon have been addressing these issues in the past, but with a little success. So let’s see if that approach works.”

The database will be continuously updated with new open buckets every two weeks – GrayhatWarfare notes that an open bucket doesn’t necessarily mean that the files are accessible, and search limitations related to Tor and VPNs have now been lifted.

GrayhatWarfare said: “This particular problem [open buckets] is very hard to fight with responsible disclosure. You can never now for sure who the owner of the bucket is. I believe that showing how easy the exploitation is, will drive more admins to be careful with their data. Similar to how Firesheep forced many websites to adapt HTTPS.“


This article has been updated to include comments from GrayhatWarfare.