A Shock to the System

When it comes to installation sizes, smaller is actually better, as long as essential features like detection aren’t compromised in order to reduce footprint, and we at ESET like to think that’s a trade-off we manage rather well.

With all due respect to our colleagues and competitors at Symantec, their products, on the other hand, have not generally been noted for their tiny footprint. We know that they have long been aware of this, and that they have been making a determined effort to address that situation. So when Passmark published some performance benchmark figures (installation size, scanning speed and so on) that indicated that Norton Internet Security 2009 did very well indeed on installation size, we weren’t entirely surprised.  What did surprise us, though, was that:

• Norton Internet Security 2009 was apparently smaller than ESET Smart Security: after all, compact installations is (just) one of the things we do very well.
• Even stranger, ESET Smart Security as tested by Passmark was apparently more than six times the size we think it is.

So we did a little testing in our own labs. Sure enough, when we reproduced the Passmark test methodology on Microsoft Windows Vista with default settings , ESET Smart Security weighed in at 368MB rather than its true installation size of 68MB. As you’re probably well aware, Vista comes with a feature called System Restore which rolls back program files and registry keys to a previous “good” state when something goes wrong. In fact, System Restore goes back as far as Windows ME (Millennium Edition). That’s a very useful safety net, but there’s a price. All the data collected from the monitoring of changes to the system and application files has to be stored somewhere locally in the form of Restore Points. It turns out that Passmark defines installation size as “the difference between the snapshot of the Disk Space […] before and after the product is installed on the system“. This means, of course, that the extra space taken up by the Restore Points is calculated as part of the installation size. So we tried the same test on Vista and XP, with System Restore turned on and with it turned off, and the results were very interesting:

Well, the third result for ESET Smart Security makes sense: System Restore on Vista is more complex than it was on previous versions of Windows, generating more files and much larger Restore Points, so even with System Restore active,it's smaller than the Norton Internet Security 2009 installation. But the interesting thing is that the Symantec product doesn't vary.  

So the true size of an ESET Smart Security installation, as shown by the results with System Restore turned off, is less than half the size of the Symantec installation. However, since Norton Internet Security 2009 bypasses Microsoft Windows’ System Restore feature so that no Restore Point is created (unless the user creates a restore point manually), there’s no way to tell this from the Passmark report.

This would not be a problem if the user were aware of the issue, and therefore able to make an informed choice as to whether to create (or not to create) a restore point. Since the user is not prompted to create a Restore Point manually, however, it is definitely a problem if he assumes that System Restore has been invoked and is there to help him to recover from a failed installation.

We’re not suggesting intentional malpractice here, of course. But as far as we know, Norton Internet Security 2009 is the only product out of the ten tested against benchmark #7 (installation size)in the Passmark test  that bypasses System Restore. So the fair and accurate way of running this particular test would have been to disable System Restore on the test system for every vendor tested. (Or to create a Restore Point manually for Norton Internet Security 2009, which on Vista would increase its size to 445MB.)

As it happens, Norton Internet Security 2009 and ESET Smart Security tied for first place in the Internet Security Products category. So we wouldn’t dream of embarrassing Symantec by pointing out that if that particular test had been done more accurately, the extra point would have put our product into first place all on our own. ;-)

There’s plenty of information about System Restore around on Technet and so on, but here are a few good places to start:

• Microsoft - Use System Restore to Undo Changes if Problems Occur
• Microsoft - How to turn off and turn on System Restore in Windows XP
• Microsoft - System Restore: frequently asked questions
• Symantec – Disabling or enabling Windows XP System Restore
• Wikipedia – entry on System Restore

Andrea Kokavcova, Aryeh Goretsky, David Harley
ESET LLC