Why Do Developers Engage with ChatGPT in Issue-Tracker? Investigating Usage and Reliance on ChatGPT-Generated Code
Guardado en:
| Udgivet i: | arXiv.org (Dec 10, 2024), p. n/a |
|---|---|
| Hovedforfatter: | |
| Andre forfattere: | , |
| Udgivet: |
Cornell University Library, arXiv.org
|
| Fag: | |
| Online adgang: | Citation/Abstract Full text outside of ProQuest |
| Tags: |
Ingen Tags, Vær først til at tagge denne postø!
|
MARC
| LEADER | 00000nab a2200000uu 4500 | ||
|---|---|---|---|
| 001 | 3142734190 | ||
| 003 | UK-CbPIL | ||
| 022 | |a 2331-8422 | ||
| 035 | |a 3142734190 | ||
| 045 | 0 | |b d20241210 | |
| 100 | 1 | |a Das, Joy Krishan | |
| 245 | 1 | |a Why Do Developers Engage with ChatGPT in Issue-Tracker? Investigating Usage and Reliance on ChatGPT-Generated Code | |
| 260 | |b Cornell University Library, arXiv.org |c Dec 10, 2024 | ||
| 513 | |a Working Paper | ||
| 520 | 3 | |a Large language models (LLMs) like ChatGPT have shown the potential to assist developers with coding and debugging tasks. However, their role in collaborative issue resolution is underexplored. In this study, we analyzed 1,152 Developer-ChatGPT conversations across 1,012 issues in GitHub to examine the diverse usage of ChatGPT and reliance on its generated code. Our contributions are fourfold. First, we manually analyzed 289 conversations to understand ChatGPT's usage in the GitHub Issues. Our analysis revealed that ChatGPT is primarily utilized for ideation, whereas its usage for validation (e.g., code documentation accuracy) is minimal. Second, we applied BERTopic modeling to identify key areas of engagement on the entire dataset. We found that backend issues (e.g., API management) dominate conversations, while testing is surprisingly less covered. Third, we utilized the CPD clone detection tool to check if the code generated by ChatGPT was used to address issues. Our findings revealed that ChatGPT-generated code was used as-is to resolve only 5.83\% of the issues. Fourth, we estimated sentiment using a RoBERTa-based sentiment analysis model to determine developers' satisfaction with different usages and engagement areas. We found positive sentiment (i.e., high satisfaction) about using ChatGPT for refactoring and addressing data analytics (e.g., categorizing table data) issues. On the contrary, we observed negative sentiment when using ChatGPT to debug issues and address automation tasks (e.g., GUI interactions). Our findings show the unmet needs and growing dissatisfaction among developers. Researchers and ChatGPT developers should focus on developing task-specific solutions that help resolve diverse issues, improving user satisfaction and problem-solving efficiency in software development. | |
| 653 | |a Debugging | ||
| 653 | |a Data analysis | ||
| 653 | |a Large language models | ||
| 653 | |a Sentiment analysis | ||
| 653 | |a Chatbots | ||
| 653 | |a User satisfaction | ||
| 653 | |a Software development | ||
| 700 | 1 | |a Mondal, Saikat | |
| 700 | 1 | |a Roy, Chanchal K | |
| 773 | 0 | |t arXiv.org |g (Dec 10, 2024), p. n/a | |
| 786 | 0 | |d ProQuest |t Engineering Database | |
| 856 | 4 | 1 | |3 Citation/Abstract |u https://www.proquest.com/docview/3142734190/abstract/embedded/6A8EOT78XXH2IG52?source=fedsrch |
| 856 | 4 | 0 | |3 Full text outside of ProQuest |u http://arxiv.org/abs/2412.06757 |