Attending to Topological Spaces: The Cellular Transformer
Na minha lista:
| Publicado no: | arXiv.org (May 23, 2024), p. n/a |
|---|---|
| Autor principal: | |
| Outros Autores: | , , , , , , , |
| Publicado em: |
Cornell University Library, arXiv.org
|
| Assuntos: | |
| Acesso em linha: | Citation/Abstract Full text outside of ProQuest |
| Tags: |
Sem tags, seja o primeiro a adicionar uma tag!
|
MARC
| LEADER | 00000nab a2200000uu 4500 | ||
|---|---|---|---|
| 001 | 3059654581 | ||
| 003 | UK-CbPIL | ||
| 022 | |a 2331-8422 | ||
| 035 | |a 3059654581 | ||
| 045 | 0 | |b d20240523 | |
| 100 | 1 | |a Ballester, Rubén | |
| 245 | 1 | |a Attending to Topological Spaces: The Cellular Transformer | |
| 260 | |b Cornell University Library, arXiv.org |c May 23, 2024 | ||
| 513 | |a Working Paper | ||
| 520 | 3 | |a Topological Deep Learning seeks to enhance the predictive performance of neural network models by harnessing topological structures in input data. Topological neural networks operate on spaces such as cell complexes and hypergraphs, that can be seen as generalizations of graphs. In this work, we introduce the Cellular Transformer (CT), a novel architecture that generalizes graph-based transformers to cell complexes. First, we propose a new formulation of the usual self- and cross-attention mechanisms, tailored to leverage incidence relations in cell complexes, e.g., edge-face and node-edge relations. Additionally, we propose a set of topological positional encodings specifically designed for cell complexes. By transforming three graph datasets into cell complex datasets, our experiments reveal that CT not only achieves state-of-the-art performance, but it does so without the need for more complex enhancements such as virtual nodes, in-domain structural encodings, or graph rewiring. | |
| 653 | |a Datasets | ||
| 653 | |a Neural networks | ||
| 653 | |a Performance prediction | ||
| 653 | |a Transformers | ||
| 653 | |a Topology | ||
| 700 | 1 | |a Hernández-García, Pablo | |
| 700 | 1 | |a Papillon, Mathilde | |
| 700 | 1 | |a Battiloro, Claudio | |
| 700 | 1 | |a Miolane, Nina | |
| 700 | 1 | |a Birdal, Tolga | |
| 700 | 1 | |a Casacuberta, Carles | |
| 700 | 1 | |a Escalera, Sergio | |
| 700 | 1 | |a Hajij, Mustafa | |
| 773 | 0 | |t arXiv.org |g (May 23, 2024), p. n/a | |
| 786 | 0 | |d ProQuest |t Engineering Database | |
| 856 | 4 | 1 | |3 Citation/Abstract |u https://www.proquest.com/docview/3059654581/abstract/embedded/J7RWLIQ9I3C9JK51?source=fedsrch |
| 856 | 4 | 0 | |3 Full text outside of ProQuest |u http://arxiv.org/abs/2405.14094 |