Qodo’s fully autonomous agent tackles the complexities of regression testing

Code is constantly evolving in the software development process and requires constant testing for quality and maintainability. This is the basis of regression testing, where existing tests are re-run to ensure that modified code continues to work as intended.

However, regression testing can be time-consuming and complex, and can often be neglected in favor of other priorities.

Dig (formerly CodiumAI) says it can alleviate process headaches with today’s release of its new fully autonomous AI regression testing agent, Dig Cover. Its agent creates verification suites to ensure that software applications essentially behave. The 2 and a half year old startup announced its new tool on AWS re:Inventwhere she also presented herself as a finalist in the AWS Unicorn Tank competition.

“We’re moving to a place where AI doesn’t just write code — it helps solve most of the developer workload by proving that the code is working properly,” Qodo CEO Itamar Friedman told VentureBeat.

Supporting the next big leap in software development

Dig explained earlier this year on VentureBeat Transform that it is gradually approaching AI agents – it is taking over competitors such as Devin which offer more complex apartments. The Israeli startup offers a number of small agents that solve specific tasks within software development workflows.

Qodo Cover is the latest of these. The fully autonomous agent it analyzes the source code and performs regression tests to verify it as it changes during its life cycle. The platform ensures that each test passes, and increases the amount of code it covers – leaving only those that meet all three criteria.

Enterprise developers are estimated to spend only an hour a day writing code; Friedman emphasized that the rest of their time goes to key tasks like testing and inspection. “However, many companies rush to generate code with AI, focusing on one hour while ignoring the rest of the equation.”

Traditional testing approaches simply don’t scale, he noted, which may halt the next leap in software development where AI can reliably generate 80% or more of high-quality code. “Just as hardware verification revolutionized chip manufacturing a few decades ago, we are now at a similar inflection point with software. When 25% or more of the code is generated by AI, we need new paradigms to ensure reliability.”

Face hugging approved

Hugging Face’s recently received a pull request generated fully autonomously by Qodo Cover, demonstrating its ability to generate production quality tests. PyTorch image model repository. Pull requests are a means of quality control in software development, allowing collaborators to propose and review changes before they are integrated into the codebase. This can keep bad code and bugs out of the main codebase to ensure quality and consistency.

Hugging Face’s adoption validates Qodo’s offering and exposes it to more than 40,000 projects in the popular machine learning (ML) repository.

“Accepting a contribution to a large open source project is a signal that AI agents are starting to perform at the level of professional developers when it comes to understanding complex code bases and maintaining high quality standards,” Friedman said. “It’s a glimpse into how software development will evolve.”

Qodo Cover is built on an open-source project that Qodo launched in May. This project was based on TestGen LLMa tool developed by Meta researchers to fully automate testing. To overcome the problems with tests generated by large language models (LLMs), the researchers set out to answer specific questions:

• Does the test compile and run correctly?
• Will the test increase code coverage?

Once these questions are verified, it’s important to perform a manual investigation, Friedman writes blog post. This includes the question:

• How well is the test written?
• What value does it actually add?
• Does it meet any additional requirements?

Users provide several inputs to Qodo Cover, including:

• The source file of the code to be tested
• Existing test suite
• Coverage report
• Command to build and run suites
• Code coverage goals and the maximum number of iterations to run
• Additional context options and prompts

Qodo Cover then generates more tests of the same style, validates them with the runtime environment (ie, do they build and pass?), assess metrics such as increased code coverage, and update existing test suites and coverage reports. This is repeated until the code reaches a coverage threshold or maximum number of iterations.

It gives developers full control and provides progress reports

The Qodo agent can be deployed as a comprehensive tool that analyzes full repositories to identify gaps and inconsistencies and extend test suites. Or it can be created as a GitHub action that automatically creates pull requests to design tests for newly changed code. Qodo emphasizes that developers retain full control and have the ability to review and selectively accept tests. Each pull request also includes detailed coverage progress reports.

Qodo Cover supports all popular AI models, including GPT-4o and Claude 3.5 Sonnet. The company claims to deliver high-quality results in more than a dozen programming languages, including JavaScript, TypeScript, C++, C#, Ruby, Go, and Rust. It is designed to integrate with Qodo Merge, which checks and processes download requests, and the Qodo Gen coding tool.