In recent years, millions of new malicious programs are produced by Pa mature industry of malware production. These programs have tremendous challenges on the signature-based anti-virus products and pose great threats on network and information security. Machine learning techniques are applicable for detecting unknown malicious programs without knowing their signatures. In this paper, a
) method is developed to detect malwares with a two-layer framework. The
) are employed to identify whether the programs perform any malicious functions according to the API-calls of the programs. The
) is applied to detect malwares according to the low level function identification. The
method is compared with many classical classification algorithms with comprehensive test datasets containing 16135 malwares and 1800 benign programs. The experiments demonstrate that the
method outperforms other algorithms in terms of detection accuracy.