Thesis (Ph.D.) - Cairo University - Institute of Statistical Studies and Research- Department of Computer and Information Science

In traditional computer architecture, operating systems (OSs) are responsible for managing systems resources, handling processes requests to access the resources, and maintaining data privacy through process isolation. However, this architecture broadens the attack surface to include the whole software stack. This thesis exploits machine virtualization to provide substantially stronger information security guarantees against information leakage attacks than that traditionally offered by physical (non-virtualized) computer systems. System administration-related advantages of machine virtualization are viewed as valuable security-related advantages that are exploited to reduce systems{u2019} exposure to security threats. This thesis presents four contributions. Firstly, a novel OS-independent information security approach called Virtualized Anti-Information Leakage (VAIL) to address information leakage attacks by malicious software and insiders, even after decryption of sensitive files using untrusted machines. The idea is based on combining machine virtualization with cryptography and system call monitoring to achieve the intended objective. Secondly, a novel approach called VAIL Disordered-Bitstring Provider (VDBP) to generate pseudo-random bitstrings. It is one of VAIL modules. It is essential to support VAIL{u2019}s cryptography-related operations, and salt user-supplied passwords. Thirdly, a novel approach called VAIL System Call Monitor (VSCM) to detect and thwart previously unknown code injection attacks. It is one of VAIL modules. It intercepts and verifies CreateProcess() system call invocations from a monitored process. In case an unknown executable is detected in the first parameter of a call, this indicates its maliciousness. In response, VSCM encrypts that parameter value to render the call invalid, thereby thwarting adversaries{u2019} attacks by preventing the OS from loading and executing the new malicious child process. Fourthly, a novel covert channel that is specific to virtual machine monitors (VMMs); it is called VMM memory reclamation-based covert storage channel

Issued also as CD