BBC‐DRL: A Reinforcement and Deep Learning‐Based Cybersecurity Knowledge Graph Construction Method

ABSTRACT

The increasingly sophisticated and dynamic nature of cyberattacks poses substantial challenges to modern cybersecurity. This study investigated core technologies for constructing cybersecurity knowledge graphs, with particular emphasis on Named Entity Recognition (NER). Existing NER studies primarily focus on enhancing the BERT–BiLSTM–CRF (BBC) model. However, simply fine‐tuning hyperparameters or modifying network architectures often leads to performance saturation, resulting in limited accuracy and suboptimal entity extraction. Moreover, the inherent flexibility of NER introduces ambiguity, because open‐domain contexts frequently contain out‐of‐vocabulary entities. To address these limitations, we proposed the BBC_DRL model. Building upon domain ontology, it integrated deep reinforcement learning (DRL) into the deep learning (DL) framework and employed an agent driven by the proximal policy optimisation (PPO) algorithm to adaptively adjust labels, thereby improving both accuracy and robustness, particularly in entity disambiguation. Experimental evaluations demonstrated the effectiveness of the proposed approach, which achieved F1‐scores of 91.42%, 94.78% and 94.95% on the CyNER, DNRTI and CoNLL2003 datasets, respectively, and outperformed current state‐of‐the‐art NER models.