COMMUNICATION
Evaluating LLM-Based Test Refactoring with a Behavior-Preserving Composite Metric
- Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering : 412-414
Lien de l'article :
https://doi.org/10.1145/3774748.3795657
Discipline :
Informatique et sciences de l'information
Auteur(s) :
Wendkûuni C. Ouédraogo, Yinghua Li, Xueqi Dang, Paweł Borsukiewicz, Xin Zhou, Anıl Koyuncu, Jacques Klein, David Lo, Tegawendé F. Bissyandé
Auteur(s) tagués :
BISSYANDE T. François D'Assise
Renseignée par : BISSYANDE T. François D'Assise
Résumé
Large Language Models (LLMs) are increasingly used to refactor automatically generated unit tests, which often achieve high coverage but suffer from poor readability and maintainability. Evaluating such refactorings remains challenging, as existing metrics either penalize beneficial edits or overlook readability. We introduce CTSES, a composite metric combining CodeBLEU, METEOR, and ROUGE-L to balance semantics, readability, and structure. Experiments on 5,000+ refactorings from Defects4J and SF110 show that CTSES reduces false negatives and provides more interpretable evaluation.
Mots-clés
GPT-4o, Mistral-Large-2407, across, Defects4J, SF110