Adapting big data analytic techniques and machine learning to establish anti-intrusion attacks system / Mostafa Mohamed Shendi ; Supervised Hatem Elkadi , Mohamed Khafagy

Thesis (M.Sc.) - Cairo University - Faculty of Computers and Artificial Intelligence - Department of Information Systems

Security, availability, and performance are becoming more frequently requested and sophisticated. Traditional solutions cannot protect the organization{u2019}s assets or keep their services running and secure from different cyber-attacks. These solutions need to focus more on customer needs and satisfaction. Organizations need to perform real-time analysis on a massive amount of data from various types to discover anomalous fragments within a reasonable response time. Businesses can widen the scale of processed data, accelerate threat detection speed, keep their services up and running by monitoring the servers{u2019} status, predict failure before it happens, and increase customer satisfaction by providing efficient service on time. Processing the massive amount of the system{u2019}s log files using relational database technology has been facing a bottleneck. Traditional data analysis models have difficulties defeating these attacks since they consume too much time analyzing different logs from different devices simultaneously. To analyze such massive information sets, we need a parallel processing system and a reliable data storage mechanism. Big Data is the solution to overcome these issues. Big Data analytics plays a significant role in analyzing and correlating large volumes of disparate and complex data from different sources in different formats. In this thesis, we highlight the characteristics of Big Data and present a review of log file analysis in a Big Data environment as a first step towards getting the maximum benefits of big data in logs analytics. We propose a security information and event management model to provide real-time analysis of security alerts generated by applications, hardware, network and provide reports for compliance purposes. We applied real-time big data processing and machine learning to detect anomalous traffic

Issued also as CD