VMProtect의 역공학 방해 기능 분석 및Pin을 이용한 우회 방안

초록

Commercial obfuscation tools (protectors) aim to create difficulties in analyzing the operation process of software by applyingobfuscation techniques and Anti-reversing techniques that delay and interrupt the analysis of programs in software reverse engineeringprocess. In particular, in case of virtualization detection and anti-debugging functions, the analysis tool exits the normal execution flowand terminates the program. In this paper, we analyze Anti-reversing techniques of executables with Debugger Detection and ViralizationTools Detection options through VMProtect 3.5.0, one of the commercial obfuscation tools (protector), and address bypass methods usingPin. In addition, we predicted the location of the applied obfuscation technique by finding out a specific program termination routinethrough API analysis since there is a problem that the program is terminated by the Anti-VM technology and the Anti-DBI technologyand drew up the algorithm flowchart for bypassing the Anti-reversing techniques. Considering compatibility problems and changes intechniques from differences in versions of the software used in experiment, it was confirmed that the bypass was successful by writingthe pin automation bypass code in the latest version of the software (VMProtect, Windows, Pin) and conducting the experiment. Byimproving the proposed analysis method, it is possible to analyze the Anti-reversing method of the obfuscation tool for which the methodis not presented so far and find a bypass method.

키워드