MARATTO

article · Security and Communication Networks

Research on Network Security Situational Awareness Based on Crawler Algorithm

202233 citationsOpen accessUniversity of Ghana

In plain language

Network security situational awareness evaluates current or potential cyber attacks to determine the security status of a target system. To enhance data availability for situational awareness, a network security event database was built using the Scrapy web crawler framework to collect records from security event websites and the China Computer Network Intrusion Prevention Center vulnerability database. A text-based analysis tool was developed to clean and process textual security event information, establishing an event processing solution. Experimental tests demonstrated that the resulting database contains 43,848 data entries. Compared to traditional algorithms, this method increased data capacity by 12.79% and 29.33%, while reducing data reading time by 63.5% and 87.2%.

Key takeaways

  • A dedicated network security event database of 43,848 entries was built using web crawling and vulnerability data.
  • A text-based processing tool was designed to clean and analyse textual information from security events.
  • The crawler-based system increased database capacity by 12.79% and 29.33% relative to traditional algorithms.
  • Data reading times were reduced by 63.5% and 87.2% compared to baseline methods.

Why it matters

Tracking global information security requires timely insight into emerging threats. Gathering vulnerability records from across the web manually is slow and resource-intensive. Automated web crawling paired with text processing enables organisations to build larger security event databases more rapidly, accelerating the speed at which security operators can access threat intelligence and assess potential system risks.

Commercialisation angle

The software offers an applied and tested prototype for threat intelligence aggregation and data cleaning. Potential users include security operations centres, enterprise network defence teams, and cybersecurity software providers seeking faster ingestion of vulnerability reports. While the system demonstrates improved read times and database capacity across 43,848 records, the abstract does not indicate whether it is packaged for immediate commercial integration.

AI-generated from the published abstract. Always read the original work before citing.

Abstract

Network security situation awareness is a critical basis for security solutions because it displays the target system’s security state by assessing actual or possible cyber-attacks in the target system. Aiming at the security and stability of global information flow, this paper studies the perception and measurement of the overall situation of network security. Through the Scrappy web crawler framework, data were collected from several Zhiming network security event websites, and based on the vulnerability database of China Computer Network Intrusion Prevention Center, the network security event database was designed and established, which enriched the data of situational awareness research. This study investigates the analysis and processing of network security events, a crucial parameter in the stage of security insight and perception, and builds and implements a text-based network security event analysis tool. By designing a network security event analysis tool based on text processing, the data cleaning of network security time text information is completed, and a set of network security event processing solutions with high applicability and comprehensiveness are formed. Statistical experimental results show that the network security event database built based on the crawler algorithm contains 43,848 pieces of data, which increases the capacity by 12.79% and 29.33% compared with the traditional algorithm, and reduces the reading time by 63.5% and 87.2%.

Research topics

  • Network Security and Intrusion Detection
  • Anomaly Detection Techniques and Applications
  • Advanced Malware Detection Techniques

Read the original research

This page summarises published work. The authoritative version sits with the publisher.

DOI: 10.1155/2022/3639174

Is something wrong with this record? Report it or request removal.

Discussion

Discuss this research

Have you built on this work, tried to replicate it, or seen it applied in practice? Share what you know. Verified researchers and MARATTO™ domain experts can open a discussion, and any member can reply. Contributions are reviewed before they appear.

No discussion yet. Open the first thread.