The first wave of LLM applications, which crested in mid-2023, followed a consistent pattern: identify a task that humans currently do using text, replace the human with an LLM call, wrap it in a minimal UI, and ship. Copywriting tools, summarization tools, Q&A tools, email assistants. These products had high initial conversion rates and poor retention. The value was real but thin. The first time a user got a good email draft from an AI, it was remarkable. The fifteenth time it was routine. The underlying product was a convenience, not a workflow transformation.
The second wave, which is only beginning to emerge, looks different. It is characterized by products that have learned enough about a specific workflow to be genuinely better than a generic LLM at that workflow — not because they have a better base model, but because they have accumulated domain-specific data, developed specialized retrieval infrastructure, and built evaluation systems that allow them to improve continuously.
The Moat Question
The question that separated strong first-wave companies from weak ones was whether they were building a moat or just riding the wave. The moat question for LLM applications is fundamentally about data and workflow integration. A company that has six months of feedback signal on what good legal brief drafting looks like — thousands of examples of what a lawyer accepted, revised, or rejected — is in a categorically different position than a company that started with the same base model last week.
This proprietary feedback loop is the real asset, not the model or the prompt. The second wave companies understand this. They are designing their products from the ground up to accumulate feedback signal, not just to produce outputs.
Where Infrastructure Fits
The second wave puts more pressure on the infrastructure layer, not less. Continuous evaluation, feedback integration, data pipeline management, and prompt versioning are all requirements for the kind of compounding improvement that creates application-layer moats. The infrastructure companies that build these capabilities well will be critical enablers of the most durable LLM application businesses. The two layers are not in competition; they are mutually reinforcing.