‘Minutes matter, and being able to get that additional feed can give infosec teams the edge’
When a business goes offline, its customers are inconvenienced and its operators lose money. The outage might also be evidence of a cyber-attack.
Monitoring web services and identifying what has failed and why is far from straightforward. Organizations may think that all is well according to their own logs, while their customers see an entirely different picture.
Crowdsourcing outage data is one way to find out what is really happening – and it was that thinking that led to the creation of Downdetector, the outage monitoring service.
Get down
Downdetector was founded just short of 10 years ago in the Netherlands by Tom Sanders, a former journalist, and Sander van der Graaf. The original idea was to gather information for journalists covering e-commerce and internet services.
“It started by a couple of people working together in a press room. They kept seeing examples of outages or problems with internet hosted companies coming up,” Luke Deryckx, CTO at Downdetector parent company Ookla, told The Daily Swig.
“The reporters they were working with didn’t really have much in the way of information or data to go on about what was wrong.
“Their idea was to create a tool, a website or an app to help the community or the userbase track outages or problems with internet hosted services.”
Luke Deryckx (left) and Brennen Smith
By crowdsourcing data from web services’ own users, Downdetector was able to build up a more accurate, and timely, picture of outages.
The value of this to consumers is clear: losing access to the internet or an online service is frustrating, and not all organizations are transparent about outages.
By looking at Downdetector or other crowdsourced data, users can at least start to determine if the problem is their local connection, at the service provider, or somewhere in between.
Over the years, though, Downdetector has also been adopted by online businesses themselves, feeding the data into their network operations centres.
Read more of the latest network security news
This is especially useful in a world where businesses rely on complex networks of content distribution networks, mirrors, peering, and ISPs to deliver their services to users.
In one example, a fault in a bank’s peering arrangements with a telco meant that the telco’s customers were unable to access services despite the bank’s own logs claiming all was well.
“The users were able to report on Downdetector that, hey, there’s clearly an issue,” says Downdetector VP of technology, Brennen Smith.
“It does help provide that early warning signal that effectively will help drive these investigations and help companies to figure out that they need to do a root cause analysis that something is going wrong.”
This is increasingly of interest to the cybersecurity industry, too. Sometimes the cause of an outage is clear cut – more often it is not.
Early warning signal
Using crowdsourced data can help narrow down whether a problem is due to criminal hackers, technology failure, or some other cause. Aggregating this data across firms and across industries can help security operations centres and emergency response teams to focus their response.
“Ultimately, I’ll say that this product was not directly targeted at security,” says Smith.
“However, there are many cases where it could be applicable. Ultimately, it is a form of threat intelligence. It is a form of getting that early warning signal.”
He does not see Downdetector as replacing conventional threat intelligence feeds, but rather working alongside other data sources to flag incidents quickly.
“Minutes matter, and being able to get that additional feed from users saying, ‘Hey, I’m seeing this weird certificate error or hey, I’m seeing this odd issue’ and it’s localized in a particular geo or country, that can give infosec teams the edge that they need to catch something before it becomes a widespread event.”
Even data from members of the public with little technical knowledge can provide valuable red flags to security teams, Deryckx argues. He believes that Downdetector provides contextual information that is not otherwise easy to uncover.
“One of the things the teams in the NOC [network operations center] are always looking for is any sort of signal around security incidents or security problems. Again, it’s not that Downdetector is positioned as a security-focused tool entirely.
“But we do know that there are many, many content providers out there who are watching the Downdetector dashboards in real time, 24 hours a day.
“If there were to be a security related incident that is highlighted as a problem on Downdetector, it would absolutely be something that that team would see and would be able to investigate earlier than they would with without that context,” he explains.
Navigating hurdles
Downdetector has the potential to fill a gap between security emails aimed at consumers, which tend to be overwhelmed by spam, and “hyper-technical” vulnerability alerts from security researchers or bug bounty programmes.
One hurdle could be opposition from companies that the service monitors. In the early years of Speedtest, owned by Downdetector parent company Ookla, some ISPs were critical of a service that exposed their performance.
“We got a lot of push back from the industry about being measured by a third party,” Deryckx admits. “Not everybody was entirely comfortable with us being that objective, third-party measurement tool that empowered consumers.”
Similar reservations applied to Downdetector – at least in its early days.
Deryckx explained: “Some organizations feel as though they have the full picture of the performance and availability of their services. And they want to own the consumer interface and the messaging end to end.
“Our point of view is that that does a disservice to the consumer, when the content provider owns that message and may or may not be updating the status page or even maintaining a status page, for example.
“It doesn’t always answer the questions that the consumer has.” Mature organizations, on the other hand, appreciate the value of an “unfiltered” consumer viewpoint.
READ Unpatched plugins threaten millions of WordPress websites
The company is working to make its service “bi-directional”, allowing organizations to upload messages about outages or other problems on the Downdetector site, rather than forcing users to trawl through status pages.
Another area that Downdetector wants to develop is outage correlation. Again, although it is not specific to security, pooling information across service providers should give infosec teams more accurate information about whether it is their service, a third party, or even a wider entity such as national infrastructure, that is being attacked.
“So many services are hosted centrally on a few cloud platforms or using a few CNDs,” says Deryckx. “And so, it’s much more common now that when there’s an incident, it affects a lot more services than it would have in the past.
“Downdetector is really the only voice out there that is in real time [that is] able to communicate that with customers and frankly the engineers who are responsible for the products impacted so we all know what's happening.”
YOU MAY ALSO LIKE Lagging behind? New study highlights weaknesses in open source patch process
