Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More
The AI boom has set off an explosion of data. AI models need massive datasets to train on, and the workloads they power — whether internal tools or customer-facing apps — are generating a flood of telemetry data: logs, metrics, traces and more.
Even with observability tools that have been around for some time, organizations are often struggling to keep up, making it harder to detect and respond to incidents in time. That’s where a new player, Observo AI, comes in.
The California-based startup, which has just been backed by Felicis and Lightspeed Venture Partners, has developed a platform that creates AI-native data pipelines to automatically manage surging telemetry flows. This ultimately helps companies like Informatica and Bill.com cut incident response times by over 40% and slash observability costs by more than half.
The problem: rule-based telemetry control
Modern enterprise systems generate petabyte-scale operational data on an ongoing basis.
While this noisy, unstructured information has some value, not every data point is a critical signal for identifying incidents. This leaves teams dealing with a lot of data to filter through for their response systems. If they feed everything into the system, the costs and false positives increase. On the other hand, if they pick and choose, scalability and accuracy get hit — again leading to missed threat detection and response.
In a recent survey by KPMG, nearly 50% of enterprises said they suffered from security breaches, with poor data quality and false alerts being major contributors. It’s true that some security information and event management (SIEM) systems and observability tools have rule-based filters to cut down the noise, but that rigid approach doesn’t evolve in response to surging data volumes.
To address this gap, Gurjeet Arora, who previously led engineering at Rubrik, developed Observo, a platform that optimizes these operational data pipelines with the help of AI.
The offering sits between telemetry sources and destinations and uses ML models to analyze the stream of data coming in. It understands this information and then cuts down the noise to decide where it should go — to a high-value incident alert and response system or a more affordable data lake covering different data categories. In essence, it finds the high-importance signals on its own and routes them to the right place.
“Observo AI…dynamically learns, adapts and automates decisions across complex data pipelines,” Arora told VentureBeat. “By leveraging ML and LLMs, it filters through noisy, unstructured telemetry data, extracting only the most critical signals for incident detection and response. Plus, Observo’s Orion data engineer automates a variety of data pipeline functions including the ability to derive insights using a natural language query capability.”
What’s even more interesting here is that the platform continues to evolve its understanding on an ongoing basis, proactively adjusting its filtering rules and optimizing the pipeline between sources and destinations in real time. This ensures that it keeps up even as new threats and anomalies emerge, and does not require new rules to be set up.
The value to enterprises
Observo AI has been around for nine months and has already roped in over a dozen enterprise customers, including Informatica, Bill.com, Alteryx, Rubrik, Humber River Health and Harbor Freight. Arora noted that they have seen 600% revenue growth quarter-over-quarter and have already drawn some of their competitors’ customers.
“Our biggest competitor today is another start-up called Cribl. We have clear product and value differentiation against Cribl, and have also displaced them at a few enterprises. At the highest level, our use of AI is the key differentiating factor, which leads to higher data optimizations and enrichment, leading to better ROI and analytics, leading to faster incident resolution,” he added, noting that the company typically optimizes data pipelines to the extent of reducing “noise” by 60-70%, as compared to competitors’ 20-30%.
The CEO did not share how the above-mentioned customers derived benefits from Observo, although he did point out what the platform has been able to do for companies operating in highly regulated industries (without sharing names).
In one case, a large North American hospital was struggling with the growing volume of security telemetry from different sources, leading to thousands of insignificant alerts and massive expenses for Azure Sentinel SIEM, data retention and compute. The organization’s security operations analysts tried creating makeshift pipelines to manually sample and reduce the amount of data ingested, but they feared they could be missing some signals that could have a big impact.
With Observo’s data-source-specific algorithms, the organization was initially able to reduce more than 78% of the total log volume ingested into Sentinel while fully onboarding all the data that mattered. As the tool continues to improve, the company expect to achieve more than 85% reductions within the first three months. On the cost front, it reduced the total cost of Sentinel, including storage and compute, by over 50%.
This allowed their team to prioritize the most important alerts, leading to a 35% reduction in mean time to resolve critical incidents.
Similarly, in another case, a global data and AI company was able to reduce its log volumes by more than 70% and reduce its total Elasticsearch Observability and SIEM costs by more than 40%.
Plan ahead
As the next step in this work, the company plans to accelerate its go-to-market efforts and take on other players in the category — Cribl, Splunk, DataDog, etc.
It also plans to enhance the product with more AI capabilities, anomaly detection, data policy engine, analytics, and source and destination connectors.
According to insights from MarketsAndMarkets, the market size for global observability tools and platforms is expected to grow nearly 12% from $2.4 billion in 2023 to $4.1 billion by 2028.
READ SOURCE
