TL;DR

• Over 80% of large enterprises use AI for legacy code analysis and documentation.
• Real-world code generation accuracy ranges from 25–34%, presenting a challenge for many organizations.
• Strategies involve incremental refactoring that combines legacy data vectorization with human developer verification.

Current Status

As of 2026, over 80% of large enterprises have integrated AI-based modernization tools into their workflows. Companies use Large Language Models (LLMs) to analyze execution paths of outdated code. They also use these models to generate missing documentation. AI agents support cloud-native transitions by identifying dependency relationships within legacy systems.

Technical limitations persist. LLMs show 84–89% accuracy in specific benchmarks. However, real-world class-level code generation accuracy remains at the 25–34% level. Problem-solving rates on SWE-bench reached approximately 50%. Errors occur in complex architectures. About 51% of adopting organizations reported low accuracy as a major issue.

Companies are focusing on the vectorization of legacy data to address technical debt. They convert old documentation and code into vector formats for AI processing. This process lays the foundation for data migration. System migration can proceed while preserving existing business logic.

Analysis

AI agents can understand legacy architectures. They trace execution paths and identify dependencies faster than human developers. This helps in understanding systems when original personnel have departed. AI acts as a bridge to transfer knowledge from past systems.

The 25–34% accuracy in practical applications warrants caution. Delegating entire system refactoring to AI can carry significant risk. Code suggested by AI agents might omit exception handling. It could also create security vulnerabilities. AI is likely better suited as an analytical assistant than an independent executor.

Practical Application

IT decision-makers can adopt AI in stages. First, build a knowledge base by vectorizing existing code and documentation. Priority should be given to creating an environment where agents understand the context.

An incremental transition is recommended over a full replacement. Organizations can start by decoupling low-risk modules into a Microservices Architecture. Generated code should undergo developer reviews and unit testing. Verification costs for AI-written code should be considered during budgeting.

FAQ

Q: Can AI analyze specific legacy languages such as MUMPS or ALC? A: Specific benchmark figures for these languages are not confirmed. General models show higher performance in mainstream languages. The accuracy of special language analysis requires further verification.

Q: How do architectural understanding and code generation differ? A: Architectural understanding identifies calling relationships and data flows. Code generation creates working programs based on that understanding. AI is proficient at identifying structures, but code generation accuracy requires improvement.

Q: Should adoption be postponed if there are accuracy issues? A: It can be more appropriate to adjust the scope of utilization. Prioritize analysis, documentation, and test case generation. This can increase modernization speed while managing risk.

Conclusion

AI agents are useful tools for legacy system analysis. While many enterprises have adopted them, low accuracy remains a challenge. Companies can leverage AI's analytical capabilities. They should subject the output to rigorous verification processes. Success can depend on AI's ability to re-implement complex business logic precisely.

참고 자료

• 🛡️ Source
• 🏛️ Beyond Synthetic Benchmarks: Evaluating LLM Performance on Real-World Class-Level Code Generation - arXiv