Amsterdam – At the forefront of cloud-native innovation, KubeCon Europe 2026 in Amsterdam played host to a significant announcement from Microsoft: the unveiling of the Azure Kubernetes Application Network (App Net). This fully managed service mesh, built upon the robust foundation of Istio’s ambient mode, represents a strategic shift towards abstracting the complexities of service mesh technology, making it virtually invisible to end-users while enhancing network capabilities for burgeoning AI workloads. Mitch Connors, a principal software engineer at Microsoft and a respected Istio maintainer serving on the project’s Technical Oversight Committee, articulated the vision behind this development in an exclusive interview with The New Stack.
Connors, who joined Microsoft two years ago to spearhead service mesh operations before transitioning into a product management role, highlighted the core philosophy driving App Net: "Success for me looks like most people not knowing what a service mesh is, even though they’re using one." This sentiment underscores a broader industry trend to simplify and democratize complex infrastructure technologies, enabling development teams to focus on delivering business value rather than managing intricate network configurations. The Azure Kubernetes Application Network is poised to be a key enabler of this goal, particularly as the demands of AI workloads reshape the networking landscape.
The Evolution of Service Mesh: From Sidecars to Ambient Simplicity
The journey towards a more user-friendly service mesh has been marked by significant architectural evolution. Historically, Istio’s sidecar model, while powerful, presented operational challenges. "You run ‘helm upgrade Istio,’ none of your proxies have upgraded," Connors explained, illustrating the potential for divergence between control plane updates and data plane deployments. "You need to go and recreate all your applications, allow rollouts to start all over again in order to take advantage of that upgrade." This cumbersome process, often referred to as "day-two headaches," created friction and hindered widespread adoption.
Istio’s ambient mode, which reached general availability approximately two years prior to KubeCon EU 2026, represented a pivotal shift. By migrating encryption to a lightweight, per-node Rust proxy and decoupling Layer 7 features into independently upgradeable waypoint proxies, ambient mode eliminated the need for application restarts following control plane updates. This architectural refinement significantly improved the operational overhead associated with service mesh management.
However, Connors noted that even with these advancements, adoption faced hurdles. He revealed that "about 85% of ambient installations aren’t keeping up with CVE patches," indicating a persistent gap between the availability of advanced features and their practical implementation by users. This observation served as a catalyst for Microsoft’s decision to develop a fully managed solution.
The launch of Azure Kubernetes Application Network at KubeCon EU 2026 is a direct response to this challenge. Built on Istio ambient, the service offers mutual transportation layer security (mTLS) enabled by default across all managed Kubernetes clusters. The deliberate omission of the term "service mesh" in its name is a strategic marketing and product design choice. "We’ve met a lot of customers who say, ‘I don’t need service mesh. I just need a proxy that gives me mTLS,’" Connors stated. "So the product meets them there. And maybe once you’ve added all those features together, you’ll realize you’re on a service mesh, but it’s a very different experience than someone who adopted service mesh three years ago." This approach aims to onboard users based on immediate needs, such as enhanced security, and gradually introduce them to the broader capabilities of a service mesh as their understanding and requirements evolve.
Navigating the AI Revolution: New Demands on Network Infrastructure
The burgeoning field of Artificial Intelligence, particularly the widespread adoption of Large Language Models (LLMs) and other sophisticated AI workloads, is placing unprecedented demands on network infrastructure. Connors elaborated on how these new workloads necessitate a departure from traditional networking paradigms. "Traditional HTTP routing assumes each request costs roughly the same to serve," he observed. "That’s not even remotely true with an LLM, where one request might be the perpetual ‘hi’ message being sent to our LLM, and another request might be someone asking Copilot to explain service mesh for them, and that’s going to take a little bit."
The variability in request complexity and resource consumption inherent in AI workloads requires a more dynamic and intelligent networking layer. In response to these evolving needs, the Istio project is embracing a "two-speed" approach. This strategy involves collaborating with projects like Agent Gateway, a Linux Foundation initiative specifically designed for agentic traffic, developed by engineers with deep expertise in Istio.
"If you want cutting-edge A2A, MCP and all of the other alphabet soup of AI, you can use Agent Gateway APIs, and it’s just a clear signal that these are going to be an alpha experience," Connors explained. "We can’t promise you that they’re going to be around in the same shape in two years, but you’ve expressed interest in using the bleeding edge of technology, and so we’re going to support you in that as well." This flexible model acknowledges the rapid pace of AI development and the experimental nature of its associated protocols, while still providing a pathway for early adopters to leverage cutting-edge capabilities.
A key innovation in this area is the Gateway API’s inference extension. This feature integrates a small LLM directly into the network layer, acting as a token estimator to assess the complexity of incoming requests upfront. The estimated token usage then informs rate limiting and resource allocation across the cluster, with actual token consumption in responses providing further feedback for dynamic adjustments. Azure Kubernetes Application Network is shipping with this inference extension, and integration with the broader Agent Gateway is slated for upstream inclusion.
Beyond traffic management, Connors sees a critical role for the service mesh in AI governance. As organizations increasingly utilize LLMs, ensuring compliance and security becomes paramount. "Platform teams offer approved LLM endpoints, but can’t enforce that users stick to them," he noted. "We want to actually inspect the body of the request and say, this is an LLM request. It needs to be going to an approved LLM service." This capability allows for granular policy enforcement at the network level, preventing unauthorized access to sensitive AI models or the misuse of corporate resources.
Addressing Multi-Cluster Complexity and GPU Scarcity
The expansion of AI workloads, particularly those requiring significant computational power, often necessitates multi-cluster deployments and efficient resource utilization. Ambient multi-cluster support has emerged as a critical area of development within Istio over the past year, and it is a foundational prerequisite for Azure Kubernetes Application Network. "Without a consistent root of trust across clusters, any traffic that goes between clusters effectively just bypasses all of the network safety that you’ve set up," Connors emphasized. Establishing secure and consistent communication across distributed environments is essential for maintaining data integrity and security.
The scarcity of GPU resources further amplifies the importance of flexible networking. Connors highlighted that "GPU capacity is often available but not in the right region." The service mesh, through its ability to manage inter-service communication seamlessly, empowers organizations to deploy GPU-intensive AI workloads in regions where capacity is readily available, irrespective of the physical location of the data or the end-users. This geographical flexibility, coupled with robust network security, allows for optimized resource utilization and cost-effectiveness.
Despite the advancements and the growing importance of service mesh technologies, Connors acknowledged that a significant portion of the Kubernetes ecosystem remains untapped. "Sixty percent of Kubernetes clusters still don’t run any service mesh," he stated. This statistic underscores the persistent perception of complexity associated with service meshes.
"We earned the reputation for being very complicated," Connors admitted candidly. "We were very complicated. And I don’t blame a user who tried service mesh five years ago from saying, ‘I don’t want anything with the word Istio put on my cluster at any point in time.’ Trust is hard to earn back, but we’re working to do that." Microsoft’s strategy with Azure Kubernetes Application Network is precisely aimed at rebuilding that trust by offering a managed, simplified experience. The goal is to appeal to a broader audience, including those who may not possess extensive cloud-native vocabulary but understand their needs from a "tactile business perspective."
The strategic bet with Azure Kubernetes Application Network is that by rendering the service mesh invisible and fully managed, organizations will embrace its core security benefits, such as mTLS, even if they are not initially focused on the full spectrum of service mesh capabilities. As AI continues to drive innovation and impose new demands on network infrastructure, this approach positions Microsoft to capture a significant segment of the market and guide users towards more sophisticated networking solutions as their requirements evolve. The future of enterprise networking, as envisioned by Microsoft and exemplified by App Net, is one where underlying complexity is seamlessly managed, allowing businesses to harness the power of advanced technologies like AI with greater ease and confidence.