The Drive for Awareness
In November of 2016, PC Pitstop, makers of PC Matic, commissioned a test with AV Comparatives focusing on ransomware and virus detection. This test was commissioned in an effort to stress the importance of detection rates. PC Matic chose to commission the test because several anti-virus (AV) companies choose not to participate in public testing. However, these companies continue to be successful. Therefore, suggesting buyers are not purchasing their security solutions based on malware detection rates. This is concerning. The primary purpose for a security program is to prevent a malware attack. Yet, buyers do not know how well certain products can do this, because several companies choose to forego third-party detection testing.
It is time the security industry begins to drive an emphasis on detection rates. By doing so, every security solution provider will be driven to do better. Thus, the consumers will be better protected. If products are not being tested for accuracy, how can buyers possibly know if it’s capable of protecting their data? They cannot.
AV Comparatives Conducts Involuntary Malware Detection Test
In the test, PC Matic commissioned with AV Comparatives in November of 2016, 18 security solutions were tested. Those results can be found here. The results of the initial test brought in hundreds of comments from not only PC Matic users, but users of other security solutions as well.
Several users of other AV solutions requested PC Matic commission another test with AV Comparatives. This time, to include additional security programs. PC Matic took this feedback and did just that.
Another involuntary test was commissioned with AV Comparatives, this time including 28 AV solutions. PC Matic also requested AV Comparatives test not only ransomware and virus samples, but polymorphic ransomware samples and false positives as well. For those who may be unfamiliar, polymorphic ransomware is a ransomware variant that has the ability to change, or morph, by altering the code within the sample. Consequently, this form of ransomware has the ability to alter itself every minute, potentially even every second. False positives are unknown files. Therefore, they have not been tested and deemed safe or malicious.
The Results
The test included 120 polymorphic ransomware samples, 1,000 recent ransomware samples, and 4,000 other malware variants. As stated above, this test was involuntary, meaning none of the security solutions knew they were going to be tested, with the exception of PC Matic. None of the AV programs, including PC Matic, knew when they would be tested. During the testing process, the tests were run “On Execute” with the internet connection enabled. Each sample was run through a script and executed, and then analyzed to see if the AV product in question was able to properly detect the sample.
Multiple security solutions tested well, including Microsoft. Historically, Microsoft’s free AV program, Windows Defender was far from effective; testing dead last against several leading AV solutions in 2013. Today, its detection is better than some purchased security solutions.
What PC users must understand is, if they are going to pay for a security solution, they must pay for the best.
PC Matic earned perfect scores in polymorphic ransomware, recent ransomware, and other malware detection. Its false positive rate was 96.67%. This is due to PC Matic’s whitelisting methodology. Therefore, anyone who uses the traditional blacklisting methodology, which allows any unknown files to execute, will score 100%.
The chart below allows readers to hover over each category to see how their security solution scored in all four areas.
