For this episode of The New Stack Makers, we speak with Matthew O’Riordan, CEO and co-founder of Ably. Ably’s platform handles trillions of transactions a month for customers like HubSpot and Intercom. Originally built for human collaboration, its technology now also nicely enables long-running AI agents. In this episode, we discuss why HTTP starts to break for long-running agents, how a “durable session” layer is emerging in AI infrastructure, and why Ably wants developers to stop thinking about pub/sub.
Ably, a company renowned for its real-time data delivery platform powering millions of concurrent connections for leading organizations such as HubSpot and Intercom, is increasingly finding its robust infrastructure to be a critical enabler for the burgeoning field of artificial intelligence, particularly for long-running AI agents. Matthew O’Riordan, CEO and co-founder of Ably, recently shared insights on The New Stack Makers podcast, detailing how the company’s foundational technologies, initially designed for seamless human collaboration, are proving exceptionally well-suited to address the unique communication challenges posed by advanced AI systems.
The AI bandwagon Ably tried not to jump on
O’Riordan recounted how, approximately 18 months ago, both external investors and internal teams at Ably began urging the company to articulate an "AI story." However, O’Riordan initially resisted, emphasizing that Ably was fundamentally an infrastructure provider, not an AI company in the traditional sense. Its core competency lay in building a reliable communication layer focused on critical aspects like presence, message ordering, state synchronization, and seamless reconnections—features essential for human-to-human interaction platforms.
This reluctance stemmed from a desire to stay true to Ably’s core mission. The company had spent years perfecting a system capable of handling the complexities of real-time, multi-user applications where dropped connections, intermittent network availability, and the need for persistent, synchronized states were paramount. Examples of Ably’s success in this domain include powering real-time chat functionalities for Intercom, enabling collaborative document editing features for various platforms, and ensuring live data updates for financial trading applications. These use cases, while demanding, were primarily focused on human users interacting with each other or with applications in real-time.
The paradigm shift occurred with the advent and rapid evolution of AI agents. These are not merely simple chatbots executing single, short-lived queries. Instead, modern AI agents are increasingly designed as long-running processes that engage in complex reasoning, interact with external tools and APIs, and maintain context over extended periods, sometimes hours. This extended operational lifespan and the intricate interaction patterns introduced new demands on communication infrastructure that, surprisingly, mirrored the challenges Ably had already solved for human collaboration.
"This is what we’ve been building for humans to communicate. And agents are becoming more human-like," O’Riordan explained. He candidly admitted that while the current applicability to AI wasn’t the initial grand vision, it represented a natural and powerful extension of Ably’s capabilities. "I’d love to say we had the vision, and that’s why we built it all, but it’s not. We’re just very well suited to solving this communication layer between agents and humans." This statement highlights a common theme in technology evolution: foundational innovations often find new life and purpose in unforeseen, emergent applications.
Where HTTP starts to break for agents
The default communication protocol for many modern web applications, including those utilizing AI frameworks like LangGraph, is HTTP. For simple, transactional exchanges—such as a user asking a chatbot a quick question and receiving a concise answer—HTTP performs adequately. Its request-response model is straightforward and well-understood by developers. However, as O’Riordan pointed out, this model begins to falter when dealing with the complexities of long-running AI agents.
Consider an agent tasked with a multi-step process: it might need to access a database, call an external API to fetch data, process that data through a series of reasoning steps, and then perhaps interact with another tool, all while the human user is expecting a continuous and responsive experience. In such scenarios, the limitations of standard HTTP become apparent.
A long-running agent can face numerous disruptions. A user might switch browser tabs, lose their internet connection temporarily, or even attempt to interrupt the agent’s process mid-execution. In a traditional HTTP request-response cycle, a dropped connection or a prolonged processing time can lead to an incomplete request, a timeout, or a corrupted state. The user experience suffers as they are forced to restart the entire process or deal with an inconsistent application state. This is particularly problematic when users have become accustomed to the seamless continuity offered by consumer-facing AI services like ChatGPT and Claude.
"What most people are doing is they click between tabs and they just expect the experience," O’Riordan observed. "You ask it something, it takes a while, you go away, you come back five minutes later to a different tab, and you expect everything to just be in sync." This expectation of persistent, always-on connectivity is a direct consequence of modern user interface design and the increasing sophistication of web applications, and it’s a standard that AI agents must now meet.
The emergence of "durable sessions"
To address these shortcomings, a new architectural concept is gaining traction in the AI infrastructure space: the "durable session." O’Riordan credits EMQX, a prominent MQTT broker, with originating the term, and ElectricSQL, a real-time relational database, with popularizing it within the AI context. O’Riordan favors "durable session" over alternatives like "durable streams" because it encompasses a broader set of requirements. A durable session, in his view, must not only handle the continuous flow of data (streams) but also manage presence information (who is connected and active), shared state synchronization, the ability to rehydrate application state after unexpected crashes, and the delivery of push notifications to users who may be offline.
Ably has adapted its platform to meet these new demands by extending its core primitives. For instance, in scenarios involving token streaming—where an agent might be sending a continuous stream of information—Ably has moved beyond purely immutable messages. They now support mutable messages, allowing a client that reconnects mid-stream to request the current state of the stream rather than having to replay every single message from the beginning. This significantly reduces latency and improves the user experience.
Furthermore, Ably has introduced "live objects." This feature provides a mechanism for collaborative storage, ensuring that an AI agent and a human user remain synchronized on shared data. This is crucial when an agent might take a considerable amount of time to reason through a complex task, during which the human user might have navigated away or become engaged in other activities. Live objects ensure that when the user returns or the agent completes its task, both parties are working with the most up-to-date and consistent state.
The drop-in replacement strategy
A critical requirement from developers adopting AI agents is the need for new infrastructure components to be easily integrated without requiring a complete overhaul of their existing systems. The goal is a "drop-in replacement"—a solution that can be seamlessly integrated into existing stacks and design patterns. This philosophy is central to Ably’s approach with its AI Transport solution.
Ably’s AI Transport is designed to be highly interoperable. It maintains the client-to-agent communication path over HTTP, acknowledging that developers are already familiar and comfortable with this protocol for initial requests. However, it intelligently shifts the response path to a durable session. Essentially, the agent acknowledges the initial request via HTTP and then begins streaming its responses and ongoing updates over the established durable session. This hybrid approach leverages the strengths of both protocols.
This strategy aligns with evolving architectural patterns in popular development frameworks. Platforms like Vercel and TanStack are actively incorporating plug-in points for durable session providers. Ably’s Vercel AI SDK plug-in, for example, directly addresses this by enabling features such as multi-device session resumption, robust interruption support, and persistent storage, all contributing to a more resilient and user-friendly AI agent experience.
O’Riordan also expressed a strong opinion on the terminology developers should use, advocating for a move away from outdated concepts. "I feel strongly that people should not have to think about pub/sub," he stated. This sentiment suggests that while publish-subscribe (pub/sub) is a foundational pattern, the higher-level abstractions and solutions now available, like durable sessions, offer a more intuitive and powerful way to manage real-time data flows for complex applications, including those powered by AI. By abstracting away the complexities of pub/sub, Ably aims to allow developers to focus on building intelligent applications rather than managing the intricacies of message delivery.
The broader implications for AI infrastructure
The evolution of AI agents from simple query-response systems to sophisticated, long-running reasoning engines necessitates a corresponding evolution in the underlying communication infrastructure. Ably’s focus on durable sessions and its adaptation of human collaboration technologies for AI agents highlights a significant trend: the blurring lines between traditional real-time applications and AI-driven systems.
As AI agents become more integrated into daily workflows, the expectation for seamless, uninterrupted, and context-aware interactions will only grow. This will likely drive further innovation in areas such as state management, fault tolerance, and efficient data synchronization. The success of Ably’s AI Transport suggests that solutions which offer a pragmatic balance—leveraging existing protocols while introducing new capabilities for state persistence and real-time updates—will be highly valued.
The ability of AI agents to maintain context across multiple interactions and devices, to be interrupted and resumed without data loss, and to provide a consistent experience regardless of network conditions will be critical for their widespread adoption and effectiveness. Companies like Ably are at the forefront of building the invisible plumbing that makes these advanced AI capabilities possible, demonstrating that the foundational principles of robust real-time communication are as relevant as ever in the age of artificial intelligence. The ongoing development of specialized infrastructure, like durable sessions, is a testament to the rapid maturation of the AI ecosystem, moving beyond foundational models to the practical implementation of intelligent agents in real-world applications.
