Research highlights need for sysadmins to ‘lock down any features that end customers can access’
A newly patched bug in Open Distro, a software package that includes Elasticsearch and Kibana, enabled adversaries to gain unauthorized access to server and network resources.
The bug, discovered and reported by security researcher Rotem Bar, allowed privileged users to enumerate listening services or interact with configured resources via HTTP requests in the network of the Open Distro server, also known as server-side request forgery (SSRF).
Webhooks and SSRF
Open Distro adds many new features to Elasticsearch and makes it easy to interact with the underlying API. While investigating the software, Bar found a web module that allowed users to create an “open distro module” and define a customizable webhook to any resource in the network.
Bar was able to use this module to create a webhook that ran SSRF requests and scanned the network, accessed the metadata API, the Kubernetes API, and other local resources.
An attacker could take the scheme further to identify other vulnerabilities in the services running in the local servers and use them to stage more complicated attacks, the researcher said.
Bar was able to narrow down the vulnerability to a line of code in Open Distro that creates HTTP requests based upon user input and sends them without doing any validation.
The severity of the bug depends on the environment of the installation. For example, if only known administrators can access the Elasticsearch instance and the service is isolated from other network resources, then the risk is low.
However, if the Elasticsearch instance is accessible to all users within and outside the company, then the severity increases. And if there are no measures to restrict access between servers, it can lead to critical incidents.
“When installing Open Distro, it includes lots of services out-of-the-box, and one of them is the alerting module which enables the creation of webhook monitors,” Bar said.
In a blog post that details the vulnerability, Bar explains how web admins can check to see if their Open Distro installation includes the module in question and how they can disable it.
“There is a way to remove these plugins and configure them, but they are not intuitive and mostly configured from the file configuration and not from the UI,” Bar said. “Some integrators I've encountered did not know of these options. Even now after this fix, which adds the ability to add a blocklist to deny resources from internal resources, this is not enabled by default.”
The latest version of Open Distro has resolved the issue. In addition to installing updates, Bar recommended general protective measures such as enforcing isolation within networks, reducing permissions, monitoring for abusive usage, and considering the deployment of a web application firewall (WAF).
“I can say from seeing many security mistakes that you should always assume the worst for any external system that you are not familiar with,” he said.
A patchwork of solutions
Bar discovered the bug while performing penetration testing on the servers of a client that had combined different solutions into one big tech stack. This is a common practice among many organizations and enterprises that don’t have the in-house software talent and hire system integrators to patch together a working solution.
“Large organizations that usually do not have the development capabilities in-house create a public tender and request for services, to answer a specific need of theirs,” Bar told The Daily Swig.
“In order to win these tenders, there are companies that instead of developing a product, which can cost a lot, prefer to create a pre-packaged solution based on a number of products they glue together and provide this to the customer.”
The problem with this approach is that the finished solution often has more complexity and features than the client needs, which opens unnoticed attack vectors for crafty hackers.
This patchwork of solutions also requires configuration, maintenance, and updating procedures that are often beyond the skills of the client.
“I think it is great to use open source systems and can cut lots of costs, but we should make sure to lock down any features that end customers can access,” Bar said.
“The bigger and more complex the system is, the more likely it is missing or contains an incorrect configuration and through these, an attacker can compromise the entire system.”