4000 VIRUSES, 1000 RANSOMWARE SAMPLES, 100% DETECTION
As you may recall from the Q3 2016 PC Pitstop Newsletter our CEO, Rob Cheng, mentioned the need for consumers and IT Pros alike to focus on detection rates when choosing their anti-virus solution. Therefore, PC Pitstop partnered with AV Comparatives to create a comprehensive and superior detection rate test that, hopefully, will focus purchase decisions on detection rates. The test overcomes many of the flaws of prior public tests.
The test is involuntary, that is no participant was aware when the test was being performed. Products were downloaded from each of the web sites, and run in default mode. The current model between the testers and the testees is financial. The testees pay the testers and submit their products.
Since it is involuntary, the test can be comprehensive permitting the public test results for both Malwarebytes and Webroot.
A criticism of AV testing are the small sample sizes. AV Test and AV Comparatives typically employ samples sizes measured in the hundreds which is a contrast to the Virus Bulletin, which has sample sizes in the tens of thousands. This test bridged the gap with a sample size of 5000. None of the samples were known to Virus Total at the time of the test. No vendor, including PC Pitstop, provided samples for the test.
The samples were categorized into ransomware and non-ransomware samples. In addition to the first comprehensive test, this is one of the first tests to analyze ransomware detection rates. The tests were run “On Execute” with the internet connection enabled. Each sample was run through a script and executed, and then analyzed to see if the AV product in question was able to properly detect the sample.
This is the first of many tests that we plan to develop to further the discussion of detection rates in the security industry. Our new test is not perfect, but it is ground-breaking in many ways, and we believe that it will force the industry to improve detection rates to thwart the threat of ransomware. We’re planning to do more involuntary tests with AV Comparatives, if you’d like to stay up to date when new results are released click here to sign up for our weekly newsletter.
You can read the full PC Pitstop fourth quarter newsletter here.