At Google Cloud Next in Las Vegas, a pivotal discussion unfolded regarding the seismic shifts occurring within the financial discipline of cloud cost management, commonly known as FinOps. The accelerating integration of Artificial Intelligence (AI) into enterprise operations is forcing a reevaluation of established FinOps practices, necessitating a faster adaptation than was observed during the initial widespread adoption of cloud computing.
The conversation, hosted by The New Stack Makers, featured insights from Roi Ravhon, co-founder and CEO of Finout, a company specializing in FinOps solutions, and Pathik Sharma, who leads cloud FinOps at Google Cloud. Their dialogue highlighted the critical need for FinOps to mature at an unprecedented pace, driven by the unique economic models and operational complexities introduced by AI technologies, particularly large language models (LLMs).
Token Economics and the Accelerated Pace of AI FinOps
A central theme of the discussion was the stark contrast in the evolutionary timeline for FinOps in the cloud era versus the AI era. "We need to do the same thing we did for cloud to AI, but we’re doing it in a year," stated Roi Ravhon, encapsulating the urgency of the situation. Cloud computing’s integration into enterprise infrastructure spanned roughly a decade, allowing FinOps to develop and mature organically. In contrast, AI’s rapid ascent into production environments demands a compressed adaptation period, estimated by Ravhon to be around one year.
This accelerated evolution is largely driven by the fundamental economic principles governing AI operations, which challenge the long-held assumptions of traditional FinOps. Ravhon pointed to a critical divergence: despite a general trend of decreasing token prices from AI providers, the overall AI costs for enterprises are escalating. This counterintuitive trend is attributed to the increased computational demands of newer, more sophisticated AI models. For instance, the release of advanced reasoning models by companies like Anthropic and OpenAI around the time of the discussion meant these models were engaging in more extensive "thinking" processes, consuming more tokens to complete the same tasks.
Adding another layer of complexity is the inherent variability in AI costs. Unlike the more predictable pricing structures in traditional cloud services, the cost of running the same AI prompt can fluctuate significantly. "You ask the same question twice, and you get different token usage for everything," Ravhon observed, questioning the scalability of such unpredictable expenditure. This unpredictability is quickly eroding the patience of Chief Financial Officers (CFOs). Ravhon noted a shift in sentiment from an initial phase of "unlimited budgets" focused on innovation to a current emphasis on Return on Investment (ROI).
Strategic Cost Optimization in the AI Landscape
Pathik Sharma of Google Cloud echoed this sentiment, advocating for a more judicious approach to AI resource utilization. He shared an analogy he frequently uses with customers: "Don’t reach for Thor’s hammer when you don’t need it." This metaphor underscores the importance of selecting the appropriate AI model for a given task, rather than defaulting to the most powerful and expensive option. Sharma recounted instances where customers were using advanced models like Gemini Pro for simple tasks such as email summarization or drafting. He highlighted that Google’s smaller and more cost-effective Gemini Flash model is often sufficient for these use cases.
The core of FinOps in the AI era, according to Sharma, is not about burdening individual employees with the technical nuances of model selection. Instead, it involves establishing an "orchestration layer" that intelligently routes each request to the most economical model capable of fulfilling it reliably.
Sharma further elaborated on the multifaceted nature of AI expenditure, extending beyond LLM API costs. The total cost of AI implementation encompasses significant investments in GPUs and TPUs (which remain in high demand and short supply), compute resources for model training and inference, storage for vast datasets, and the considerable organizational costs associated with AI adoption. Citing recent research from Stanford University, Sharma emphasized that for every dollar invested in tangible AI technology, companies may spend up to ten dollars on intangible aspects such as process redesign, reskilling, and organizational transformation.
A key strategy for mitigating these costs involves deploying smaller AI models closer to the end-user. Sharma cited his personal experience installing Gemma, Google’s compact open-source model, on his smartphone. This on-device model, weighing under 4 GB, is capable of performing tasks like summarization, OCR, and translation locally. This demonstrates that not all AI-driven operations necessitate the use of large, resource-intensive frontier models.
Agentic FinOps: Balancing Automation with Determinism
The discussion also delved into the potential and pitfalls of using AI agents for FinOps tasks. Ravhon characterized FinOps as a discipline focused on addressing numerous "small problems that you need to fix." In large enterprises with thousands of developers, this translates into a continuous stream of right-sizing recommendations and anomaly detections related to redundant infrastructure. Individually, these issues might seem minor, but collectively, they represent significant financial leakage.
While the traditional approach involved a combination of increased headcount and cultural initiatives to foster cost consciousness among engineers, the advent of AI agents presents a new temptation: to delegate these tasks entirely to LLMs. However, Ravhon cautioned against this approach, arguing that FinOps is a "partially deterministic problem." He explained that relying solely on LLMs is problematic because tasks like right-sizing and anomaly detection have inherent mathematical thresholds and deterministic logic. LLMs, he noted, can sometimes exhibit a tendency to "convince themselves that they’re right when they want to be right," leading to inaccurate or unreliable outcomes.
Ravhon proposed an architectural framework for agentic FinOps that prioritizes determinism. This approach involves building agentic solutions like assembling Lego bricks: the foundational, deterministic components remain intact, while the agentic layer acts as a binder. "If you want detection, detection is deterministic. Don’t try to reinvent the wheel," he advised. Conversely, tasks such as data enrichment and contextual analysis are well-suited for AI agents. Crucially, any potentially destructive actions, such as server termination, must be preceded by deterministic checks or a human approval step to ensure safety and accuracy.
Sharma offered a complementary perspective, drawing parallels between onboarding an AI agent and integrating a new Site Reliability Engineer (SRE). This involves establishing clear standards, defining scoped permissions, and developing playbooks to guide decision-making processes. Using the example of Kubernetes right-sizing on Google Kubernetes Engine (GKE), Sharma clarified that the objective is not to instruct an LLM to "fix my Kubernetes." Instead, the agent should be provided with the same data an SRE would use: golden signals, request and limit metrics, observability data from the past 30 days, and peak p99 vCPU utilization. The agent’s output would then be a recommendation presented as a pull request for the application owner’s review and approval. This process, Sharma emphasized, builds immediate trust, as the recommendation’s origin and context are transparent.
Foundational Principles for Newcomers to FinOps
When questioned about the best starting point for individuals new to FinOps, both Ravhon and Sharma pointed towards community resources rather than specific vendor solutions. "Sign up to the FinOps Foundation," Ravhon strongly recommended. He articulated that FinOps is fundamentally an organizational challenge, and simply acquiring a FinOps tool will not resolve the underlying issues.
Ravhon stressed the sequential nature of addressing FinOps: culture change must precede the adoption of tools. This involves fostering cross-team accountability, cultivating a cost-conscious mindset among engineering teams, and reframing cloud spend as a strategic investment. "Only when you understand that you need a tool to continue scaling, this is the time you need to talk to Finout or an equivalent tool," he concluded.
Sharma concurred with this perspective, drawing on the widely recognized principle from the Spider-Man franchise: "With great power comes great responsibility." He asserted that everyone involved in managing infrastructure holds significant influence and must approach their responsibilities with a value-driven mindset, rather than solely focusing on cost. By prioritizing value, efficiency, accountability, and governance naturally follow.
The insights shared at Google Cloud Next underscore a critical juncture for FinOps. The rapid integration of AI necessitates a proactive and strategic approach to cost management, demanding innovation and adaptation from both technology providers and enterprise users alike. The journey from cloud FinOps to AI FinOps is compressed but essential for sustainable and responsible technological advancement.
