A transformer model for boundary detection in continuous sign language
Spremljeno u:
| Izdano u: | Multimedia Tools and Applications vol. 83, no. 42 (Dec 2024), p. 89931 |
|---|---|
| Izdano: |
Springer Nature B.V.
|
| Teme: | |
| Online pristup: | Citation/Abstract Full Text - PDF |
| Oznake: |
Bez oznaka, Budi prvi tko označuje ovaj zapis!
|
| Sažetak: | Sign Language Recognition (SLR) has garnered significant attention from researchers in recent years, particularly the intricate domain of Continuous Sign Language Recognition (CSLR), which presents heightened complexity compared to Isolated Sign Language Recognition (ISLR). One of the prominent challenges in CSLR pertains to accurately detecting the boundaries of isolated signs within a continuous video stream. Additionally, the reliance on handcrafted features in existing models poses a challenge to achieving optimal accuracy. To surmount these challenges, we propose a novel approach utilizing a Transformer-based model. Unlike traditional models, our approach focuses on enhancing accuracy while eliminating the need for handcrafted features. The Transformer model is employed for both ISLR and CSLR. The training process involves using isolated sign videos, where hand keypoint features extracted from the input video are enriched using the Transformer model. Subsequently, these enriched features are forwarded to the final classification layer. The trained model, coupled with a post-processing method, is then applied to detect isolated sign boundaries within continuous sign videos. The evaluation of our model is conducted on two distinct datasets, including both continuous signs and their corresponding isolated signs, demonstrates promising results. |
|---|---|
| ISSN: | 1380-7501 1573-7721 |
| Digitalni identifikator objekta: | 10.1007/s11042-024-19079-x |
| Izvor: | ABI/INFORM Global |