A friend of mine, a CTO at a growing SaaS company, recently shared her struggles onboarding junior developers. Her team is tackling a critical legacy modernization project, and while AI-powered tools have helped her new hires ramp up faster, challenges have emerged. These junior developers, relying on AI for code suggestions, sometimes miss how their changes affect the broader system. One AI-assisted solution even caused a critical system failure when it overlooked a subtle dependency. Her story will sound familiar to many leaders navigating the complexities of integrating AI into legacy software projects.

‍

AI Productivity Meets Legacy Challenges

It’s not news that AI has become a transformative force in software development, including modernization projects. Recent studies highlight its growing role in improving productivity and quality, especially in complex environments. For example, a study of GitHub Copilot’s use in enterprise settings found that AI-assisted developers completed 26% more tasks on average, increased weekly code commits by 13.5%, and experienced faster iteration cycles, all while maintaining code quality. However, the latest DORA report shifts the narrative, reporting a 7.2% decline in software delivery stability when AI is introduced into the development process.

‍

This discrepancy raises an important question: How and why does AI simultaneously enhance productivity and code quality but hinder delivery stability? For CIOs and CTOs, the answer lies in the context—specifically, understanding the basic contours and relationships of a codebase writ large, and the nuances and challenges in the  subsystem where work is being done, a skillset that, for now, is uniquely human. In this blog, we’ll explore the dual impact of AI in coding and how leaders can navigate these complexities to modernize legacy software effectively.

‍

The Productivity Gains of AI in Software Development

AI has proven itself as a valuable tool for increasing developer productivity. The GitHub Copilot study, conducted with over 4,800 developers across leading organizations, illustrates this point with compelling data:

‍

• Productivity Boost: Developers using AI completed 26% more tasks on average.
• Faster Iteration: AI users saw a 38.4% increase in code compilation frequency, enabling more agile development cycles.
• Code Quality Maintained: AI-assisted teams produced output that met the same high standards as human-generated code.

‍

These benefits are particularly pronounced among junior developers, who saw productivity gains of up to 40%. By offloading repetitive coding tasks to AI, these developers could focus on higher-value activities, accelerating their ramp-up and bridging the skill gap with senior colleagues.

‍

The implications for legacy software modernization would appear to be significant. Modernizing legacy systems often requires considerable developer time and resources. By leveraging AI, teams should be able address technical debt more efficiently, iterate faster, and free up bandwidth to focus on innovation. However, for organizations significantly burdened by outdated technology stacks, the temporary boost in productivity is having unforeseen consequences to the health of the codebase.

‍

The Delivery Stability Conundrum

The DORA report’s finding that AI usage reduces software delivery stability by 7.2% is not surprising, given the rapid increases in production from junior developers. Delivery stability measures an organization’s ability to deploy reliable software consistently, a cornerstone of DevOps maturity. A decline in this metric suggests that AI usage creates great code, but misses the big picture of how that code fits into the puzzle as a whole.

‍

The complexity of legacy codebases requires larger scale systems thinking. While AI excels at generating code and identifying specific patterns, it lacks the human intuition needed to fully understand the interconnectedness of large, complex codebases. This can lead to:

‍

1. Overconfidence in AI-Generated Code: Developers may accept AI-generated solutions without fully comprehending their implications, leading to unforeseen integration issues.
2. Blind Spots in Testing: AI-driven tools often focus on isolated code snippets, potentially missing edge cases or interactions across different parts of the system.
3. Acceleration in the Wrong Direction: Faster iteration cycles can inadvertently amplify existing flaws in the system, introducing new bugs or exacerbating technical debt.

‍

For organizations modernizing legacy systems, these risks are magnified. Legacy software often includes fragile, undocumented code that AI tools can easily break when not taking the whole context of an application into account.. The introduction of AI without proper safeguards can destabilize systems already operating at the edge of reliability.

‍

Bridging the Gap: Context Is Key

The dichotomy between increased productivity and reduced stability underscores the importance of context. To harness AI effectively, CIOs and CTOs must ensure their teams understand the full codebase and development pipeline. Here’s how:

‍

1. Adopt a Comprehensive Codebase Understanding

Legacy systems are notorious for their complexity. Before integrating AI, teams must ensure practitioners using AI have a general understanding of the codebase as a whole. This foundational understanding ensures that AI-generated solutions align with the system’s overall architecture. This is especially true for those junior developers who are moving faster and are more likely to ship local solutions that cause global problems.

‍

2. Maintain a Comprehensive Testing Suite

By identifying and addressing issues early in the development lifecycle, robust testing practices help ensure that new features integrate seamlessly without disrupting existing systems. Testing safeguards against bugs, performance bottlenecks, and security vulnerabilities, enabling teams to release stable, high-quality software at a consistent pace. 

‍

3. Re-think the role of junior developers

With basic coding tasks deliverable by AI, the opportunity and need for junior engineers moves up a level of abstraction where they will need to consider the correctness of a code change against the details of the system they are working on and the context of their business writ large. Ensure your junior devs have line of sight into the complexity of your codebases as well as how the solution itself supports the business. 

‍

4. Modernize Incrementally

When modernizing legacy software, start with small, manageable projects where AI can deliver quick wins. Gradual adoption allows teams to learn from early implementations, refine processes, and build confidence in AI tools without jeopardizing stability.

‍

The Path Forward for CIOs and CTOs

AI has the potential to revolutionize software development, offering unprecedented productivity gains and paving the way for legacy software modernization. However, these benefits come with challenges that must be addressed through a balanced approach.

‍

Human context is required to ensure understanding the nuances of your codebase so you can unlock AI’s full potential while safeguarding delivery stability. This means adopting robust testing practices, shifting the role of the junior developer, and approaching modernization projects with a clear, phased strategy.

‍

My friend's experience highlights a truth we can’t ignore: while AI is an incredible asset, it’s not a substitute for human judgment, especially in the nuanced world of legacy modernization. By pairing AI tools with strong human oversight and mentorship, organizations can strike the right balance—empowering junior developers, avoiding costly missteps, and fostering long-term stability and innovation. For CIOs and CTOs, the goal is clear: leverage AI strategically, but never lose sight of the value of human context and expertise.

‍

---

‍

Michael Carlson

Partner @Stride AI Labs 

‍