The AI infrastructure landscape at the end of 2024 looks like the early cloud era circa 2008: a large number of point solutions, considerable overlap in functionality, customers building bespoke integrations, and no obvious winner in most categories. 2025 will be the year consolidation begins in earnest. Here is our view of how the stack shakes out.
First, the caveat: we are writing predictions in early January, which means we are mostly right about January and progressively less right about the rest of the year. But having an explicit view forces clarity of thinking, and being wrong publicly is how we learn.
What Consolidates
Observability and tracing: this is the most mature category in the agentic stack. The patterns are well understood — trace spans, token counts, latency distributions, error rates. The tooling is getting good. We expect two or three platforms to become the default choices for teams building LLM applications, with the same inevitability that Datadog became the default for conventional infrastructure. The window for new entrants in pure observability is closing.
Prompt management: less mature than observability but converging. Teams that have been managing prompts as string literals in code are moving toward dedicated systems. The key differentiator will be tight integration with evaluation — prompt management without eval is just version control. The winners will be the platforms that close this loop.
What Stays Fragmented
Agent orchestration remains in flux. ReAct, plan-and-execute, multi-agent debate, tree-of-thought — the right pattern depends heavily on the specific task, and no single abstraction has proven universally superior. We expect the major frameworks to continue competing without a dominant winner through most of 2025. This is the category most likely to be disrupted by an unexpected architectural insight.
Memory and retrieval: still early. Vector databases have been the default, but hybrid approaches combining semantic search with structured metadata filtering are consistently outperforming pure vector retrieval in production. The right abstraction for agent memory across the episodic/semantic/procedural spectrum is still being discovered.
The Surprise
Our prediction for the biggest surprise of 2025: the emergence of agent-native security and permission infrastructure as a serious category. As agents get access to more external systems and start taking consequential actions, the "run it and see what happens" approach stops working. The companies building proper identity, authorization, and audit infrastructure for agents will matter more than people currently expect.