How independent antivirus tests work
Antivirus marketing is full of percentages. Here is how the independent labs behind those numbers actually test, and how to read their results without being misled.
Who the testers are
Three independent organisations publish regular, detailed results for consumer security products:
- AV-TEST (Magdeburg, Germany) tests consumer products for Windows, macOS and Android every couple of months. It scores Protection, Performance and Usability separately.
- AV-Comparatives (Innsbruck, Austria) runs a long-term “Real-World Protection Test”, malware protection tests, performance tests and false-positive tests. It publishes periodic summary reports.
- SE Labs (London, UK) focuses on realistic attack scenarios and publishes quarterly reports on home and business products.
Vendors often pay to take part in or license the results of these programmes. The labs publish their methods, which is what makes the results comparable. Always read the methodology section.
Protection: can it stop real attacks?
Protection tests expose each product to real-world threats, such as malicious websites and emailed attachments, and to reference sets of recently discovered malware. The question is whether the product stops the attack at any stage: before download, on download, or when the file runs. Good tests use fresh samples, because detecting a threat that has circulated for months is much easier than detecting one seen for the first time.
Performance: does it slow you down?
Performance tests time common tasks, such as copying files, installing and launching applications, downloading and opening websites, on a machine with and without the product installed. The difference shows the overhead. Results depend on the test hardware, so compare products within the same test round rather than across different ones.
Usability: does it cry wolf?
A product that blocks harmless software is not only annoying but risky. Users who see frequent false alarms learn to click “allow” without thinking. False-positive tests check how often legitimate programs and websites are wrongly flagged.
How to read a result
- Check the date and version. A result from last year says little about today’s product.
- Look at all three categories. Top protection with many false alarms may suit you less than slightly lower protection with none.
- Prefer consistency. A product that scores well round after round, at more than one lab, is a safer bet than a one-off headline.
- Be wary of rounded marketing claims. “100% protection” usually describes one test set in one round, not a guarantee.
- Remember the free options. Built-in products such as Microsoft Defender appear in the same tests, so you can compare them directly.
Sources
- AV-TEST, test procedures and results: av-test.org
- AV-Comparatives, test methods: av-comparatives.org
- SE Labs, public reports: selabs.uk