Abstract
One technique of improving computer security is to test an executable for presence of malicious code without running the executable. The present disclosure enables such detection of malicious code by leveraging the observation that system calls (syscalls) are a main pathway for exploits, since syscalls are an important way for a program to interact with an operating system kernel. The disclosure describes techniques to compute a control flow graph for the executable comprising only syscalls. A number of independent control flows are produced from such a control flow graph. Graph analysis/matching techniques are applied to detect exploit patterns in these syscall graphs, e.g., based on matching against known syscall exploit sequences for different vulnerabilities. In this manner, a potentially malicious executable is detected and can be isolated without exposing a computer system to damage.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Desnos, Anthony; Petrova, Elena; Boulgakov, Alexandre; Neal, Richard; and Mithra, Zubin, "Flow-graph analysis of system calls for exploit detection", Technical Disclosure Commons, (June 21, 2018)
https://www.tdcommons.org/dpubs_series/1271