The debate surrounding the role of AI in software development continues to polarize the industry, with some predicting widespread job displacement and others dismissing AI agents as mere novelties. At the forefront of this evolving discussion is ClickHouse, a company that has moved from initial skepticism to the systematic integration of AI coding agents into its development workflow. This journey, particularly throughout 2025, highlights the rapid maturation of AI tools and the nuanced understanding required to leverage them effectively. ClickHouse’s experience offers a compelling case study on how engineering teams are navigating the complexities of AI adoption, moving beyond simple copy-pasting to sophisticated, agent-driven development processes.
The Three Tiers of AI-Assisted Development
To understand the evolution of AI integration, it is helpful to categorize the ways developers interact with AI tools. ClickHouse identifies three distinct levels:
-
Level 1: Copy-Pasting from Chat Interfaces: This foundational level, prevalent since 2023, involves developers posing questions to AI models in separate chat interfaces and manually integrating the generated code snippets into their development environments. While still valuable for initial exploration and rapid prototyping, this method is increasingly being superseded by more integrated solutions. Its primary utility lies in brainstorming and understanding new concepts or syntaxes.
-
Level 2: Integrated Agents in CLI or IDE: This represents the current sweet spot for many development teams, including ClickHouse. These agents operate directly within the developer’s command-line interface (CLI) or integrated development environment (IDE). They possess the capability to read existing codebases, execute commands, modify files, initiate builds, run tests, and even commit changes. Developers can hand-hold the agent for complex tasks, while routine operations are delegated, significantly streamlining day-to-day workflows. This level fosters a collaborative dynamic where the AI acts as an intelligent assistant.
-
Level 3: Autonomous Agents in Isolated Environments: This advanced tier involves sophisticated multi-agent systems operating in feedback loops within isolated environments. These systems are designed for spec-driven development and can orchestrate complex workflows autonomously. While promising, the tooling for this level is still in its nascent stages, and the reliability of long-term autonomous loops can be inconsistent. ClickHouse has seen early deployments in production, but acknowledges the ongoing need for refinement.
The rapid advancement of AI models means that conclusions drawn from early experiences in 2025 might be outdated. What might have seemed like a toy six months ago could now be a critical productivity enhancer.
The Catalyst for Change: Claude Opus 4.5 and Beyond
ClickHouse’s initial stance towards AI agents, particularly for its core C++ codebase, was one of caution. For much of 2025, agents like early versions of Claude Code proved useful for less complex tasks, such as generating JavaScript boilerplate or simple Python scripts. However, they struggled to navigate the intricacies of ClickHouse’s large C++ repository. Even at their October 2025 engineering offsite, a significant portion of the team had not yet embraced agents for substantial development work, citing sporadic successes but a lack of systematic integration.
The turning point arrived in November 2025 with the release of Claude Opus 4.5. This iteration marked a significant leap in the agent’s ability to handle complex codebases. The author of the original piece recounts a personal shift from skepticism to active adoption, beginning with small, precisely defined C++ tasks. The agent’s capabilities quickly expanded to debugging by analyzing CI logs and implementing minor features, consistently exceeding expectations. This pivotal moment transformed agents from experimental tools into viable components of daily development on a large C++ project. This period, therefore, marks 2025 as the "year of the tools," setting the stage for anticipated "productivity gains" in 2026.
Proven Use Cases: Where AI Agents Excel at ClickHouse
The widespread adoption of AI agents at ClickHouse is not a theoretical exercise; it is driven by demonstrable value in several key areas:
-
Boilerplate and Integration Tasks: Repetitive modifications to build systems, configuration updates across multiple files, complex software installations (such as JDK), and the creation of Kubernetes manifests are tasks where AI agents consistently outperform humans. Their ability to execute these tasks without fatigue and with a reduced error rate makes them ideal for such monotonous yet crucial work. For teams just beginning their AI integration journey, these are the most accessible and immediately beneficial applications.
-
Resolving Merge Conflicts: AI agents have proven remarkably adept at resolving merge conflicts, achieving success rates near 100%. The "agent does, you review" paradigm ensures higher code quality. Reviewing code generated by an AI is often more effective than self-reviewing code that was just typed, as it allows for a more objective assessment of the changes.
-
Code Review Augmentation: While initially exploring third-party solutions like GitHub Copilot and Cursor’s bugbot, ClickHouse ultimately developed its own AI-powered review bot. This custom solution leverages the Copilot CLI with specific review instructions, delivering consistently surprising quality. Human reviewers can now dedicate their expertise to architectural considerations, while the AI bot efficiently identifies resource leaks, race conditions, and edge-case errors. This division of labor optimizes the review process.
-
Rectifying Flaky Tests: ClickHouse’s continuous integration (CI) pipeline is a high-volume environment, processing millions of tests daily across hundreds of commits and pull requests. The team’s commitment to investigating every test failure, rather than muting or retrying, had historically led to an overwhelming backlog. In early 2026, AI agents became instrumental in tackling this challenge. The author, with AI assistance, submitted approximately 700 pull requests addressing test failures and CI infrastructure issues. This dramatically reduced daily findings from around 200 to a mere 3-5 per 10 million test runs. Furthermore, the deployment of two autonomous agents has enabled the continuous discovery of edge cases, making this single use case a justification for the entire investment in AI agents.
-
Bug Investigation: AI agents demonstrate proficiency in analyzing logs, formulating hypotheses, and responding to prompts. However, their capacity to generate plausible yet incorrect hypotheses presents a significant challenge, demanding careful human oversight. The effectiveness of agents in bug investigation is heavily dependent on the engineer’s judgment. Experienced Site Reliability Engineers (SREs) can leverage AI to reach correct conclusions faster, while less experienced engineers might be led astray by confidently presented false leads. A notable example involves a complex concurrency bug that had eluded three human developers for an extended period. This bug was ultimately resolved by an AI agent (Opus 4.6) with a single-line code change after approximately an hour of reasoning, accompanied by a comprehensive explanation and test cases.
Recommendations for AI Integration
The journey at ClickHouse offers valuable insights for other organizations looking to adopt AI coding agents. The core recommendation is to treat AI as a tool of thought, not a replacement for thinking. This implies a strategic approach that leverages AI to augment human capabilities rather than seeking to automate human cognitive processes entirely.
The practical takeaways from ClickHouse’s experience can be summarized as follows:
- Start with Level 2 Integration: Focus on integrating AI agents directly into the CLI and IDE. This offers a tangible productivity boost with manageable complexity.
- Identify High-Value, Repetitive Tasks: Begin by applying AI agents to areas like boilerplate code generation, configuration management, and routine maintenance.
- Leverage AI for Code Review and Testing: Utilize agents to enhance code quality through automated review and to tackle the persistent challenge of flaky tests.
- Emphasize Human Oversight in Complex Scenarios: For bug investigation and complex problem-solving, ensure experienced engineers guide and validate AI-generated insights.
- Continuous Evaluation and Adaptation: The AI landscape is dynamic. Regularly assess the performance of AI agents and adapt integration strategies as new models and tools emerge.
- Invest in Customization: Where off-the-shelf solutions fall short, consider developing custom AI tools tailored to specific project needs, as ClickHouse did for its code review bot.
- Foster a Culture of Experimentation: Encourage engineers to experiment with AI agents and share their findings, creating a learning environment.
The Road Ahead: Towards True Autonomy
The integration of AI agents at ClickHouse is far from complete. The company is actively exploring further applications, including:
- Automated Triage of Bug Reports: Utilizing agents to categorize and prioritize incoming bug reports, streamlining the initial response process.
- Automatic Reversion of Problematic Changes: Implementing AI systems to detect and automatically revert code changes that introduce regressions or performance degradation.
- Agentic Testing of New Features: Deploying AI agents to rigorously test new features from various perspectives, uncovering potential issues before release.
- Continuous Analysis of Workload Performance: Employing AI for ongoing monitoring and analysis of system workloads to identify and address performance bottlenecks proactively.
The most ambitious frontier remains Level 3: genuinely autonomous coding loops. ClickHouse views the development of robust and reliable multi-agent systems for complex software development as the primary focus for the upcoming year.
The era of justifiable skepticism regarding AI-assisted coding is drawing to a close. The capabilities of AI models have advanced significantly, and the supporting tooling has matured to a point where practical, impactful integration is no longer a distant prospect but a present reality. The productivity gap between engineering teams that effectively leverage AI agents and those that do not is poised to widen considerably. For skilled engineers who are adaptable and open to embracing new technologies, this represents a critical juncture, offering an opportunity to redefine the boundaries of software development and significantly enhance their professional impact. The future of coding is increasingly intertwined with intelligent automation, and proactive adoption is key to staying ahead in this rapidly evolving technological landscape.
