PICOT questions and search strategies formulation: A novel approach using artificial intelligence automation
Salvato in:
| Pubblicato in: | Journal of Nursing Scholarship vol. 57, no. 1 (Jan 2025), p. 5 |
|---|---|
| Autore principale: | |
| Altri autori: | , , |
| Pubblicazione: |
Blackwell Publishing Ltd.
|
| Soggetti: | |
| Accesso online: | Citation/Abstract Full Text Full Text - PDF |
| Tags: |
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
MARC
| LEADER | 00000nab a2200000uu 4500 | ||
|---|---|---|---|
| 001 | 3165158164 | ||
| 003 | UK-CbPIL | ||
| 022 | |a 1527-6546 | ||
| 022 | |a 1547-5069 | ||
| 022 | |a 0363-2792 | ||
| 022 | |a 0743-5150 | ||
| 024 | 7 | |a 10.1111/jnu.13036 |2 doi | |
| 035 | |a 3165158164 | ||
| 045 | 2 | |b d20250101 |b d20250131 | |
| 084 | |a 37455 |2 nlm | ||
| 100 | 1 | |a Gosak, Lucija, MSc, RN |u Faculty of Health Sciences, University of Maribor, Maribor, Slovenia | |
| 245 | 1 | |a PICOT questions and search strategies formulation: A novel approach using artificial intelligence automation | |
| 260 | |b Blackwell Publishing Ltd. |c Jan 2025 | ||
| 513 | |a Journal Article | ||
| 520 | 3 | |a Aim: The aim of this study was to evaluate and compare artificial intelligence (Al)based large language models (LLMs) (ChatGPT-3.5, Bing, and Bard) with humanbased formulations in generating relevant clinical queries, using comprehensive methodological evaluations. Methods: To interact with the major LLMs ChatGPT-3.5, Bing Chat, and Google Bard, scripts and prompts were designed to formulate PICOT (population, intervention, comparison, outcome, time) clinical questions and search strategies. Quality of the LLMs responses was assessed using a descriptive approach and independent assessment by two researchers. To determine the number of hits, PubMed, Web of Science, Cochrane Library, and CINAHL Ultimate search results were imported separately, without search restrictions, with the search strings generated by the three LLMs and an additional one by the expert. Hits from one of the scenarios were also exported for relevance evaluation. The use of a single scenario was chosen to provide a focused analysis. Cronbach's alpha and intraclass correlation coefficient (ICC) were also calculated. Results: In five different scenarios, ChatGPT-3.5 generated 11,859 hits, Bing 1,376,854, Bard 16,583, and an expert 5919 hits. We then used the first scenario to assess the relevance of the obtained results. The human expert search approach resulted in 65.22% (56/105) relevant articles. Bing was the most accurate Al-based LLM with 70.79% (63/89), followed by ChatGPT-3.5 with 21.05% (12/45), and Bard with 13.29% (42/316) relevant hits. Based on the assessment of two evaluators, ChatGPT-3.5 received the highest score (M=48.50; SD =0.71). Results showed a high level of agreement between the two evaluators. Although ChatGPT-3.5 showed a lower percentage of relevant hits compared to Bing, this reflects the nuanced evaluation criteria, where the subjective evaluation prioritized contextual accuracy and quality over mere relevance. Conclusion: This study provides valuable insights into the ability of LLMs to formulate PICOT clinical questions and search strategies. Al-based LLMs, such as ChatGPT-3.5, demonstrate significant potential for augmenting clinical workflows, improving clinical query development, and supporting search strategies. However, the findings oversight. also highlight limitations that necessitate further refinement and continued human Clinical Relevance: Al could assist nurses in formulating PICOT clinical questions and search strategies. Al-based LLMs offer valuable support to healthcare professionals by improving the structure of clinical questions and enhancing search strategies, thereby significantly increasing the efficiency of information retrieval. | |
| 653 | |a Language | ||
| 653 | |a Relevance | ||
| 653 | |a Accuracy | ||
| 653 | |a Information retrieval | ||
| 653 | |a Retrieval | ||
| 653 | |a Subject heading schemes | ||
| 653 | |a Nurses | ||
| 653 | |a Search strategies | ||
| 653 | |a Evidence-based practice | ||
| 653 | |a Chatbots | ||
| 653 | |a Efficiency | ||
| 653 | |a Clinical outcomes | ||
| 653 | |a Skills | ||
| 653 | |a Artificial intelligence | ||
| 653 | |a Scripts | ||
| 653 | |a Health care | ||
| 653 | |a Decision making | ||
| 653 | |a Evidence-based nursing | ||
| 653 | |a Librarians | ||
| 653 | |a Medical personnel | ||
| 653 | |a Large language models | ||
| 653 | |a Clinical nursing | ||
| 653 | |a Data mining | ||
| 653 | |a Subjectivity | ||
| 653 | |a Health services | ||
| 653 | |a Evaluation | ||
| 653 | |a Automation | ||
| 653 | |a Strategies | ||
| 653 | |a Questions | ||
| 653 | |a Human-computer interaction | ||
| 653 | |a Language modeling | ||
| 700 | 1 | |a Stiglic, Gregor, PhD, BSc |u Faculty of Health Sciences, University of Maribor, Maribor, Slovenia | |
| 700 | 1 | |a Pruinelli, Lisiane, PhD, MSc, RN |u College of Nursing and College of Medicine, University of Florida, Gainesville, Florida, USA | |
| 700 | 1 | |a Vrbnjak, Dominika, PhD, MSc, RN |u Faculty of Health Sciences, University of Maribor, Maribor, Slovenia | |
| 773 | 0 | |t Journal of Nursing Scholarship |g vol. 57, no. 1 (Jan 2025), p. 5 | |
| 786 | 0 | |d ProQuest |t Sociology Database | |
| 856 | 4 | 1 | |3 Citation/Abstract |u https://www.proquest.com/docview/3165158164/abstract/embedded/L8HZQI7Z43R0LA5T?source=fedsrch |
| 856 | 4 | 0 | |3 Full Text |u https://www.proquest.com/docview/3165158164/fulltext/embedded/L8HZQI7Z43R0LA5T?source=fedsrch |
| 856 | 4 | 0 | |3 Full Text - PDF |u https://www.proquest.com/docview/3165158164/fulltextPDF/embedded/L8HZQI7Z43R0LA5T?source=fedsrch |