Theoretical and Natural Science
- The Open Access Proceedings Series for Conferences
Theoretical and Natural Science
Vol. 31, 07 March 2024
Open
Access | Article
Convergence of polarized self-attention with consistent rank Chinese text classification
* Author to whom correspondence should be addressed.
Abstract
Utilizing the powerful feature extraction capabilities of deep learning, a text classification algorithm with multi-dimensional and high-domain adaptability is designed in this study. This method enhances the model’s understanding of topics and content by incorporating the Polarized Self-Attention (PSA) module, which strengthens the spatial structure and semantic features of textual information. The loss function is redesigned to assign smaller losses to misclassifications of neighboring categories, allowing the model to optimize classification accuracy while learning hierarchical structural information between categories. Finally, experimental verification is conducted on a publicly available news dataset, demonstrating improved results in text classification achieved by the proposed algorithm.
Keywords
Text Classification; Attention Mechanism; Loss Function
References
1. LIU P F, QIU X P, HUANG X J. Recurrent neural networkfor text classification[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence, 2016: 2873-2879.
2. KIM Y. Convolutional neural networks for sentence classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014: 1746-1751.
3. Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in neural information processing systems, 2017, 30.
4. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). BERT: Bidirectional Encoder Representations from Transformers. arXiv preprint arXiv:1810.04805.
5. Syaiful Imron, Setiawan, E. I., Joan Santoso, & Mauridhi Hery Purnomo. (2023). Aspect Based Sentiment Analysis Marketplace Product Reviews Using BERT, LSTM, and CNN.
6. Wang Y, Huang M, Zhu X, et al. Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of the 2016 conference on empirical methods in natural language processing. 2016: 606-615.
7. Chen J, Hu Y, Liu J, et al. Deep short text classification with knowledge powered attention[C]//Proceedings of the AAAI conference on artificial intelligence. 2019, 33(01): 6252-6259.
Data Availability
The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. Authors who publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open Access Instruction).
- Volume Title
- Proceedings of the 3rd International Conference on Computing Innovation and Applied Physics
- ISBN (Print)
- 978-1-83558-317-3
- ISBN (Online)
- 978-1-83558-318-0
- Published Date
- 07 March 2024
- Series
- Theoretical and Natural Science
- ISSN (Print)
- 2753-8818
- ISSN (Online)
- 2753-8826
- DOI
- 10.54254/2753-8818/31/20241133
- Copyright
- 07 March 2024
- Open Access
- This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited