Comparative Testing and Swimming the Channel

Greetings, friends, fans and foes. I know it’s been a while, but I’ve been travelling, with intermittent connectivity: first the Infosecurity expo in London, then the CARO and AMTSO workshops in Budapest, then the EICAR conference in Berlin. This week I’ve been at the Channel Expo in Birmingham (the one in the UK, that is) – I get to all the glamorous places, expecially the ones that begin with a "B".

Channel sales isn’t something I know a lot about: while anti-malware people are generally acknowledged to be greedy, unscrupulous low-life bottom-feeders profiting from the misfortunes of others, I’ve spent most of my career in AV research as a customer, and am still acquiring the taste for human blood that is apparently a prerequisite for working in this industry. That doesn’t mean, though, that I don’t appreciate the hard work of the people in sales and marketing whose labours bring in the cash that allows me to live in the lap of luxury here in the home counties (that’s the South of England, for our USian readers).

Seriously, guys, I learned a lot about the business side of this industryfrom ESET UK’s presentations on the services they offer to resellers, and I’d have considered signing up myself if I wasn’t such a hopeless sales person.  (This might also be a good point at which to thank our partners in Budapest for an interesting and useful discussion during my recent visit.)

That wasn’t what I was there for, though. I was there to deliver a presentation in the Technology Threatre on comparative testing. (Bet you didn’t expect that!) Which was interesting in itself: afterwards, I found myself exchanging views with a couple of people who were already resellers, and someone who’s in the process of setting up a testing lab in the UK at the moment. Which takes me neatly on to the subject of AMTSO (the Anti-Malware Testing Standards Organization). Yes, again…

As I’ve mentioned before, one of the most interesting (well, to me…) aspects of AMTSO’s current work has been the setting up of a Review Analysis Board. In brief, the principle is that the Board can consider requests to have a test/review evaluated by a group of suitably qualified individuals within AMTSO: basically. we’ll analyse tests to see whether it’s conformed with the good practice guidelines already published on the web site. It’s taken a while to select suitable participants and establish the basic mechanisms for requesting and carrying out a review – this is definitely a job that needs to be done right, and that does take time. However, those mechanisms were agreed by the membership at the Budapest meeting, and it’s likely that the first review swill be made public sooner rather than later.

It’s probably inevitable that some testers will see this as a threat: however, I’d rather see it as a positive step towards improving testing practice globally, and it looks like testers are starting to think proactively about getting their methodologies reviewed independently. Speaking purely personally, I’d much rather be involved with helping testers that way than with "going after" bad testers with a big stick shouting "You didn’t do it right!". But I guess we’ll have to see how it all plays out.

Meanwhile, the documents approved at Budapest are now up on the AMTSO web page for public viewing, including the Review Analysis Process documentation..

Director of Malware Intelligence

Author David Harley, ESET

Follow us

Copyright © 2017 ESET, All Rights Reserved.