Signature Analysis Mathematical Model of Network Traffic and Experimental Evaluation of Its Functioning Efficiency

Relevance. In modern attack detection systems (ADSs), signature analysis algorithms are key components in the process of analyzing network traffic. Their widespread use in implementation of attack detection rules is due not only to the ease of their configuration and search speed, but also to the ability to detect attacks with zero false positives. This is achieved by specifying such sets of special rules (signature rules) that uniquely identify a specific type of attack. The development and optimization of models and algorithms for constructing such rules is an urgent task, since its solution allows increasing the level of protection of network resources from attacks.

The purpose of the research is to improve the operating efficiency of ADSs built on the basis of signature analysis. Methods used. The research is based on the application of provisions from the theories of sets and information retrieval, as well as parallel and network programming techniques. The subject is models and algorithms for signature search of attacks in network traffic, the object is signature ADSs.

Novelty. The paper presents a mathematical model of signature analysis of network traffic, which differs from known analogs in the universality of the representation of signature rules and support for multi-level processing of both individual packets and network data flows; an assessment of the effectiveness of the software implementation of this model is performed. The universality of the representation of signature rules is achieved due to the possibility of their expansion with new rules regardless of their internal implementation and without the need to reconstruct the original model. Multi-level processing of packets and network data flows by signature rules is ensured by the IP defragmentation and TCP reassembly algorithms developed and integrated into the model.

Practical significance. The result of the experiment shows that the developed network traffic analyzer demonstrates performance that is 1,5 times superior in terms of promptness and resource consumption to other open source ADSs. Thus, the developed model can be used in constructing an effective ADS.

1. Kumar S., Spafford E.H. A Pattern Matching Model for Misuse Intrusion Detection. Proceedings of the 17th National Computer Security Conference, NIST, 11–14 October 1994, Baltimore, USA, vol.1. 1994. p.11–21.

2. Branitskiy A. Software Ways for Enhancing the Effectiveness of the Network-Based Signature Attack Detection Systems. Proceedings of the VIIth International Conference on Infotelecommunications in Science and Education, 28 February – 1 March 2018, St. Petersburg, Russian Federation, vol.1. St. Petersburg: Saint-Petersburg State University of Telecommunications Publ.; 2018. p.118–123. (in Russ.) EDN:XSUFGX

3. Gowrison G., Ramar K., Muneeswaran K., Revathi T. Efficient context-free grammar intrusion detection system. International Journal of Innovative Computing Information and Control. 2011;7(8):4779–4788.

4. Kazachkin D. S., Gamayunov D. Y. Network traffic analysis optimization for signature-based intrusion detection systems. Proceedings of the Spring/Summer Young Researchers’ Colloquium on Software Engineering. 2008;2. DOI:10.15514/SYRCOSE-2008-2-5

5. Kruegel C., Toth T. Using Decision Trees to Improve Signature-Based Intrusion Detection. Proceedings of the 6th International Symposium on Recent Advances in Intrusion Detection, RAID 2003, 8–10 September 2003, Pittsburgh, USA. Lecture Notes in Computer Science, vol.2820. Berlin, Heidelberg: Springer; 2003. p.173–191. DOI:10.1007/978-3-540-45248-5_10

6. Ilgun K., Kemmerer R. A., Porras P. A. State transition analysis: A rule-based intrusion detection approach. IEEE Transactions on Software Engineering. 1995;21(3):181–199. DOI:10.1109/32.372146

7. Kumar S., Spafford E.H. A Software Architecture to Support Misuse Intrusion Detection. Proceedings of the 18th National Information Security Conference, NIST, 10–13 October 1995, Baltimore, USA, vol.1. 1995. p.194–204.

8. Zhang X., Wu T., Zheng Q., Zhai L., Hu H., Yin W., et al. Multi-Step Attack Detection Based on Pre-Trained Hidden Markov Models. Sensors. 2022;22(8):2874. DOI:10.3390/s22082874

9. Lunt T.F., Tamaru A., Gillham F. A Real-Time Intrusion Detection Expert System (IDES). Final Technical Report for SRI Project 6784. 1992.

10. Lindqvist U., Porras P.A. eXpert-BSM: A host-based intrusion detection solution for Sun Solaris. Seventeenth Annual Computer Security Applications Conference, 10–14 December 2001, New Orleans, USA. IEEE; 2001. p.240–251. DOI:10.1109/ACSAC.2001.991540

11. Lindqvist U., Porras P.A. Detecting computer and network misuse through the production-based expert system toolset (P-BEST). Proceedings of the 1999 IEEE Symposium on Security and Privacy, 14–14 May 1999, Oakland, USA. IEEE; 1999. p.146–161. DOI:10.1109/SECPRI.1999.766911

12. Kim H.J., Choi J. Recommendations for Responding to System Security Incidents Using Knowledge Graph Embedding. Electronics. 2023;13(1):171. DOI:10.3390/electronics13010171

13. Wang Y., Jere S., Banerjee S., Liu L., Shetty S., Dayekh S. Anonymous Jamming Detection in 5G with Bayesian Network Model Based Inference Analysis. Proceedings of the 23rd International Conference on High Performance Switching and Routing, HPSR, 06–08 June 2022, Taicang, China. IEEE; 2022. p.151–156. DOI:10.1109/HPSR54439.2022.9831286

14. Almseidin M., Al-Sawwa J., Alkasassbeh M., Alweshah M. On detecting distributed denial of service attacks using fuzzy inference system. Cluster Computing. 2023;26(2):1337–1351. DOI:10.1007/s10586-022-03657-5

15. Braden R., Borman D., Partridge C. RFC 1071: Computing the Internet Checksum. 1988. URL: https://dl.acm.org/doi/pdf/10.17487/RFC1071 [Accessed 22.08.2025]

16. Fall K.R., Stevens W.R. TCP/IP illustrated, vol.1. Addison-Wesley Professional, 2012. 1008 p.

17. Laraba A., François J., Chrisment I., Chowdhury S.R., Boutaba R. Detecting Multi-Step Attacks: A Modular Approach for Programmable Data Plane. NOMS 2022-2022 IEEE/IFIP Network Operations and Management Symposium, 25–29 April 2022, Budapest, Hungary. IEEE Press; 2022. p.1–9. DOI:10.1109/NOMS54207.2022.978993.

18. Madhloom J.K., Noori Z.H., Ebis S.K., Hassen O.A., Darwish S.M. An Information Security Engineering Framework for Modeling Packet Filtering Firewall Using Neutrosophic Petri Nets. Computers. 2023;12(10):202. DOI:10.3390/computers12100202

19. Branitskiy A.A. Algorithms for parallel search of template substrings in the implementation of signature rules of ADSs. Regional Informatics and Information Security: Conference Proceedings, 1–3 November 2017, St. Petersburg, Russian Federation, vol.4. St. Petersburg: Saint-Petersburg Society for Informatics, Computer Engineering, Communications and Control Systems Publ.; 2017. p.210–212. EDN:XTONUL

20. Thota K.K., Raj R.J.R. Efficient Regular Expression Matching and Hardware-Accelerated Finite Automata Pattern Recognition in NIDS. Proceedings of the 6th International Conference on Recent Trends in Advance Computing, 14–15 December 2023, ICRTAC, Chennai, India. IEEE; 2023. p.349–353. DOI:10.1109/ICRTAC59277.2023.10480760

21. Branitskiy A.A. Detection of Anomalous Network Connections Based on Hybridization of Computational Intelligence Methods. Ph.D. Dissertation. St. Petersburg: St. Petersburg Federal Research Center of the Russian Academy of Sciences Publ.; 2018. 305 p. (in Russ.) EDN:OQGUHC

22. Febrita R.E., Hakim L., Utomo A.P. The Implementation of Machine Learning for Optimizing Network-Based Intrusion Detection in the Snort Application. Proceedings of the 6th International Seminar on Research of Information Technology and Intelligent Systems, ISRITI, 11–12 December 2023, Batam, Indonesia. 2023. p.141–147. DOI:10.1109/ISRITI60336.2023.10467566

23. Makanju A., LaRoche P., Zincir-Heywood A.N. A Comparison Between Signature and Machine Learning Based Detectors. URL: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=16e92def7fca8e41894e875f2eb9d4118a4d09df (Accessed 22.08.2025)