Imagen de Portada

A Machine Learning-Based Approach to Enhancing Label Aggregation in Crowdsourcing

Guardado en:
Publicado en:ADCAIJ : Advances in Distributed Computing and Artificial Intelligence Journal vol. 14 (2025), p. e32757-e32778
Autor principal: Suyal, Himanshu
Otros Autores: Singh, Avtar
Publicado:
Ediciones Universidad de Salamanca
Materias:
Machine learning
Similarity
Datasets
Labels
Annotations
Bayesian analysis
Crowdsourcing
Statistical inference
Reliability
Supervised learning
Effectiveness
Acceso en línea:Citation/Abstract
Full Text - PDF
Etiquetas: Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
Resumen:Crowdsourcing is the most effective means of obtaining labelled data for supervised machine learning. However, the varying expertise of crowd workers often results in noisy annotations. While traditional label aggregation methods attempt to handle label noise, they typically overlook the relationships between different data instances. Moreover, crowdsourced datasets often experience class imbalance, wherein predominant classes eclipse minority classes, hence exacerbating label accuracy issues. This paper proposed a Reliability-Weighted Bayesian Label Aggregation (RWBLA) to overcome the above challenges. First, the K-nearest neighbours (KNN) method is used to improve the label set for each instance by augmenting labels from its closest neighbours, resulting in multiple noisy label sets. Next, it improves the aggregation by assigning weights to the neighbouring labels based on worker reliability, adaptive distance, and label similarity for each instance. In addition, worker reliability is determined dynamically based on the neighbourhood information, and to handle the imbalance issue, label similarity is modified. In the end, a weighted Bayesian inference method is used to infer the correct label for each instance. The performance of the proposed approach is evaluated on 20 synthetics and three real-world crowdsourcing datasets. It shows that RWBLA consistently surpasses eight baseline label aggregations, improving aggregation accuracy by 3 % to 13 %. Moreover, in analyses utilizing imbalanced real-world crowdsourcing data, RWBLA surpassed state-of-art aggregation algorithms by 2 % to 6 %, underscoring its efficacy in situations with minor class imbalances.
ISSN:2255-2863
DOI:10.14201/adcaij.32757
Fuente:Advanced Technologies & Aerospace Database