The proliferation of online violence has given rise to a spate of malignant incidents, necessitating a renewed focus on the identification of cyberbullying comments. Text classification lies at the heart of efforts to tackle this pernicious problem. The identification of cyberbullying comments presents unique challenges that call for innovative solutions. In contrast to traditional text classification tasks, cyberbullying comments are often accompanied by subtle and arbitrary expressions that can confound even the most sophisticated classification networks, resulting in low recognition accuracy and effectiveness. To address this challenge, a novel approach is proposed that leverages the BERT pre-training model for word embedding to retain the hidden semantic information in the text. Building on this foundation, the BiSRU++ model which combines attentional mechanisms is used to further extract contextual features of comments. A multi-task learning framework is employed for joint training of sentiment analysis and cyberbullying detection to improve the model's classification accuracy and generalization ability. The proposed model is no longer entirely reliant on a sensitive word dictionary, and experimental results demonstrate its ability to better understand semantic information compared to traditional models, facilitating the identification of potential online cyberbullying comments.

Bidirectional Encoder Representations from Transformers (BERT) has become a paradigm shift in natural language processing (NLP). This scientometric study analyzes a curated dataset of 200 highly-cited Scopus publications to map the intellectual landscape and research trajectories catalyzed by BERT and other transformer models. The study discloses a quick evolution from foundational architectural innovations and pre-training paradigms to widespread domain adaptation, rigorous model optimization for efficiency, and critical examination of model capabilities and societal effects. The literature shows BERT's role as a foundational model, successfully implemented in diverse fields such as biomedicine, education, and software engineering, while simultaneously spurring substantial research into compression, quantization, and efficient inference. A parallel and influential strand of studies emerged concentrated on “BERTology”, giving the linguistic knowledge and biases encoded within these models, and addressing ethical concerns regarding their deployment. The study synthesizes these developments, presenting how BERT not only set new performance benchmarks but also built a new paradigm for transfer learning and spurred a self-critical research community, ultimately paving the way for the present era of large language models.