On Interpreting the Effectiveness of Unsupervised Software Traceability with Information Theory
I tiakina i:
| I whakaputaina i: | arXiv.org (Dec 6, 2024), p. n/a |
|---|---|
| Kaituhi matua: | |
| Ētahi atu kaituhi: | , , |
| I whakaputaina: |
Cornell University Library, arXiv.org
|
| Ngā marau: | |
| Urunga tuihono: | Citation/Abstract Full text outside of ProQuest |
| Ngā Tūtohu: |
Kāore He Tūtohu, Me noho koe te mea tuatahi ki te tūtohu i tēnei pūkete!
|
MARC
| LEADER | 00000nab a2200000uu 4500 | ||
|---|---|---|---|
| 001 | 3142374882 | ||
| 003 | UK-CbPIL | ||
| 022 | |a 2331-8422 | ||
| 035 | |a 3142374882 | ||
| 045 | 0 | |b d20241206 | |
| 100 | 1 | |a Palacio, David N | |
| 245 | 1 | |a On Interpreting the Effectiveness of Unsupervised Software Traceability with Information Theory | |
| 260 | |b Cornell University Library, arXiv.org |c Dec 6, 2024 | ||
| 513 | |a Working Paper | ||
| 520 | 3 | |a Traceability is a cornerstone of modern software development, ensuring system reliability and facilitating software maintenance. While unsupervised techniques leveraging Information Retrieval (IR) and Machine Learning (ML) methods have been widely used for predicting trace links, their effectiveness remains underexplored. In particular, these techniques often assume traceability patterns are present within textual data - a premise that may not hold universally. Moreover, standard evaluation metrics such as precision, recall, accuracy, or F1 measure can misrepresent the model performance when underlying data distributions are not properly analyzed. Given that automated traceability techniques tend to struggle to establish links, we need further insight into the information limits related to traceability artifacts. In this paper, we propose an approach, TraceXplainer, for using information theory metrics to evaluate and better understand the performance (limits) of unsupervised traceability techniques. Specifically, we introduce self-information, cross-entropy, and mutual information (MI) as metrics to measure the informativeness and reliability of traceability links. Through a comprehensive replication and analysis of well-studied datasets and techniques, we investigate the effectiveness of unsupervised techniques that predict traceability links using IR/ML. This application of TraceXplainer illustrates an imbalance in typical traceability datasets where the source code has on average 1.48 more information bits (i.e., entropy) than the linked documentation. Additionally, we demonstrate that an average MI of 4.81 bits, loss of 1.75, and noise of 0.28 bits signify that there are information-theoretic limits on the effectiveness of unsupervised traceability techniques. We hope these findings spur additional research on understanding the limits and progress of traceability research. | |
| 653 | |a Software | ||
| 653 | |a Datasets | ||
| 653 | |a Source code | ||
| 653 | |a System reliability | ||
| 653 | |a Performance evaluation | ||
| 653 | |a Information retrieval | ||
| 653 | |a Effectiveness | ||
| 653 | |a Software reliability | ||
| 653 | |a Debugging | ||
| 653 | |a Entropy (Information theory) | ||
| 653 | |a Links | ||
| 653 | |a Software engineering | ||
| 653 | |a Information theory | ||
| 653 | |a Machine learning | ||
| 653 | |a Software development | ||
| 653 | |a Predictive maintenance | ||
| 700 | 1 | |a Rodriguez-Cardenas, Daniel | |
| 700 | 1 | |a Poshyvanyk, Denys | |
| 700 | 1 | |a Moran, Kevin | |
| 773 | 0 | |t arXiv.org |g (Dec 6, 2024), p. n/a | |
| 786 | 0 | |d ProQuest |t Engineering Database | |
| 856 | 4 | 1 | |3 Citation/Abstract |u https://www.proquest.com/docview/3142374882/abstract/embedded/6A8EOT78XXH2IG52?source=fedsrch |
| 856 | 4 | 0 | |3 Full text outside of ProQuest |u http://arxiv.org/abs/2412.04704 |