Automated Test Case Generation With LLMs

IBM's ASTER uses LLM prompting guided by lightweight program analysis to generate high-coverage, natural unit tests for Python and Java

ASTER incorporates software mocking methods to manage external dependencies like database transactions and web service calls

Smaller models like Granite-34B and Llama-3-8B perform nearly as well as larger models, offering cost and privacy advantages for enterprise environments

Over 70% of surveyed developers at IBM preferred ASTER-generated tests, finding them more usable and comprehensible 

ASTER received the Distinguished Article Award at the 2025 International Conference on Software Engineering (ICSE) for its innovative approach

Plans include refining testing models to reduce LLM costs, expanding ASTER to more programming languages and testing levels

ASTER addresses the issues with a four-step process: static analysis, LLM-guided test generation, postprocessing and refinement, and coverage augmentation