The Race to Zero

General

24

The Race to Zero contest is being held during Defcon 16 at the Riviera Hotel in Las Vegas, 8-10 August 2008.

The event involves contestants being given a sample set of viruses and malcode to modify and upload through the contest portal. The portal passes the modified samples through a number of antivirus engines and determines if the sample is a known threat. The first team or individual to pass their sample past all antivirus engines undetected wins that round. Each round increases in complexity as the contest progresses.
 
There are a number of key ideas we want to get across by running this event:
1. Reverse engineering and code analysis is fun.

2. Not all antivirus is equal, some products are far easier to circumvent than others. Poorly performing antivirus vendors should be called out.

3. The majority of the signature-based antivirus products can be easily circumvented with a minimal amount of effort.

4. The time taken to modify a piece of known malware to circumvent a good proportion of scanners is disproportionate to the costs of antivirus protection and the losses resulting from the trust placed in it.

5. Signature-based antivirus is dead, people need to look to heuristic, statistical and behaviour based techniques to identify emerging threats

6. Antivirus is just part of the larger picture, you need to look at controlling your endpoint devcies with patching, firewalling and sound security policies to remain virus free.

We are not creating new viruses and modified samples will not be released into the wild, contrary to the belief of some media organisations

Above all we want the contestants to have fun!
————————————————

The website also has a graphic that show the contest is given a sample and then modifies it to evade detection. This is script kiddie 101.

The key ideas they claim to want to get across do not warrant creating new malware.

1) Reverse engineering and code analysis is fun.

If that’s what you want to teach, then grab some samples and start reverse engineering them and analyzing them. Modification is not required to get the idea across.

2. Not all antivirus is equal, some products are far easier to circumvent than others. Poorly performing antivirus vendors should be called out.

Gee, Virus Bulletin and other organizations have been testing anti-virus products for more than a decade and proving that not all antivirus is equal. If it was everyone would buy the cheapest one. A statistically miniscule sample set is not enough to make broad conclusions about anti-virus products.

Retrospective testing can easily show which the poorly performing products are without creating any new malware.

3. The majority of the signature-based antivirus products can be easily circumvented with a minimal amount of effort.

Old news to any informed person. There are plenty of new variants out there every day to prove the point without making more.

4. The time taken to modify a piece of known malware to circumvent a good proportion of scanners is disproportionate to the costs of antivirus protection and the losses resulting from the trust placed in it.

Now there’s real idiocy. The losses are caused by malware and some hackers, not by trust placed in anti-virus products. Creating new variants, which are of course, new threats, does not teach defense in depth. The person appears to be arguing against using anti-virus at all, which of course bad advice for most users. It displays a complete ignorance of the principals of defense in depth.

5. Signature-based antivirus is dead, people need to look to heuristic, statistical and behaviour based techniques to identify emerging threats

Well here’s some old news for you. It doesn’t take modifying malware to understand that signature based scanning was never designed as protection against emerging threats. A stupid contest won’t prove what is already known.

6. Antivirus is just part of the larger picture, you need to look at controlling your endpoint devcies with patching, firewalling and sound security policies to remain virus free.

And here they contradict what they said in item 4.

Finally they add a foot note:

"We are not creating new viruses and modified samples will not be released into the wild, contrary to the belief of some media organisations

Above all we want the contestants to have fun! "

Yes modifications are new viruses. Mistakes do happen and the modified variants can get let loose into the wild. The organizers have zero control over the participants to ensure that the modifications will not be released and publically posted.

The statement "Above all we want the contestants to have fun!" truly shows the point is not about any of items one through six.

Next they have the rules of engagement…

——————-
Rules of Engagement
The following rules apply to all contetants:

* 1. Contestants can work in teams of up to 4 people
* 2. Modified virus samples must be functionally the same as the original
   You can modify mutexes, filenames, process names, IP addresses, etc as long as the code functions the same <Randy’s note> Script kiddie tricks
* 3. Modified malcode samples must still exploit the vulnerability it was intended for
  Samples of vulnerable software will be provided to contestants to test their exploits against
* 4. Modified samples will not be submitted to antivirus vendors unless authorised by contest participants <Randy’s note> This contradicts their FAQ which I will get to
* 5. Race to Zero staff may analyse virus submissions to draw conclusions/trends, etc
* 6. Techniques used to perform mutations will not be submitted to antivirus vendors without contestants approval but may be used during our post-contest round-up presentation
* 7. Judges decision is final, no correspondence will be entered into unless beer is supplied
——————-

Frequenty Asked Questions

Q: Won’t this simply encourage the creation of new malware?
A: No, we don’t believe so. The process that will be undertaken by contestants is already happening 24 hours a day, 7 days a week worldwide and it would be naive to think otherwise. It is because of this that we want to analyse how difficult a suitably motivated attackers’ task is to circumvent widely deployed AV defences.

<Randy’s note> Of course they are encouraging the creation of new malware. Creating new malware is required to win the contest. The argument that others are doing it already is like saying that because other people steal it is ok to have a theft contest. If analysis of the difficulty is desired then look at what’s already been done.

Q: Will the samples generated for the contest be given to AV vendors?
A: We very much hope so, but this is down to each contestant to decide. We are optimistic that contestants will give us permission to pass on their modified samples to the AV vendors that want them, but it is not something we are able to demand of them. All samples, including those submitted to AV vendors will be securely deleted from the Race to Zero systems after the contest analysis is complete.

<Randy’s note> Actually requiring samples to be submitted to AV vendors is something that could be part of the rules if the organizers had a wee little bit of spine.

Q: Is this an attempt to undermine the AV vendors?
A: Certainly not. Part of doing security research is tackling questions that may at first appear highly controversial. We feel that there are legitimate questions for us to investigate about the techniques that could be used by attackers. By researching into these areas we hope to be able to bolster the defences against malware that will be available in addition to AV. We are not saying AV has no value, or that people should turn off their AV protection.

<Randy’s note> Here is a great contradiction. They claim to hope to bolster defenses, but will not require samples and techniques to be shared with people who are actually writing products to do just that.

Q. What do you mean by signature-based AV?
A. Almost all AV engines today work at a level higher than just blacklisting samples. They have a heuristic component to them which looks for routines common to a family of malware. They may be able to unpack the sample and analyse the underlying code so that if you were to repack the virus with a different packer it would still be detected. In the end though they are still looking for particular patterns or signatures.

<Randy’s note> They clearly do not understand the variety of heuristic approaches in use.

Why not a contest to write a better anti-virus program? Well, they are squarely aimed at script kiddies and not at anyone with the technical skill required to write an even mediocre anti-virus program. Lacking the skills to build they encourage destruction.

A pretty lame script kiddie contest quite honestly.

Randy Abrams
Director of Technical Education
ESET LLC

Author ESET Research, ESET

  • Brian

    Great post Randy, this was the most on point response on Race to Zero that I’ve seen. Its easy to blame the AV vendors (not the hackers) because you know where to find them.

  • solcroft


    Gee, Virus Bulletin and other organizations have been testing anti-virus products for more than a decade and proving that not all antivirus is equal. If it was everyone would buy the cheapest one. A statistically miniscule sample set is not enough to make broad conclusions about anti-virus products.

    Virus Bulletins and other organizations are inherently flawed because their results fail to coincide with reality – on the contrary, they only provide the spiel that marketing divisions want to hear. If antivirus products are really as excellent as Virus Bulletin and other organizations claim they are, then this contest will only be positive publicity for the antivirus industry, and they have nothing to be worried about.


    Old news to any informed person. There are plenty of new variants out there every day to prove the point without making more.

    That’s the whole point of this contest – the publicity. Surely antivirus vendors have nothing against increasing the numbers of such informed persons?


    Now there’s real idiocy. The losses are caused by malware and some hackers, not by trust placed in anti-virus products. Creating new variants, which are of course, new threats, does not teach defense in depth. The person appears to be arguing against using anti-virus at all, which of course bad advice for most users. It displays a complete ignorance of the principals of defense in depth.

    Partly incorrect. Losses are caused by malware AND an inappropriate level of trust placed in anti-virus products. To take road safety as an example: people will become more careful when they learn that seat belts won’t save them from every accident. There is no argument in abolishing antivirus software; it is the antivirus industry who is putting the words into the mouth of others. Rather, by raising publicity about the facts, more people will become aware of factors in computer security and explore alternative, complementary approaches to keep themselves safe. I don’t see how this can be a bad thing.


    Of course they are encouraging the creation of new malware. Creating new malware is required to win the contest. The argument that others are doing it already is like saying that because other people steal it is ok to have a theft contest.

    Just because the contest involves writing malware does not mean it encourages writing malware. Going by your logic, Formula 1 races encourage people to speed recklessly while driving.


    Here is a great contradiction. They claim to hope to bolster defenses, but will not require samples and techniques to be shared with people who are actually writing products to do just that.

    There is no contradiction. The contest hopes to bolster defenses by generating publicity and creating awareness among the general public. Antivirus vendors have repeatedly claimed that there is nothing new to be learned from these “script kiddies” – why then are they suddenly so interested in acquiring the samples?

  • Randy Abrams

    The fact that the organizers who make the rules say that they can’t share the samples without permission, when they in fact can make the rules is very telling about their honesty level.

    There is zero need to create no malware to prove the point that defense in depth is required no matter what security product you use.

    Randy Abrams
    Director of Technical Education
    ESET LLC

  • solcroft

    Randy,

    Your goals seem to have shifted from arguing that the contest is pointless and unnecessary, to complaining that you don’t like the rules.

    You are correct that the point doesn’t need to be proven, because it already is. However, awareness of this point among the general public is the key issue. For them to safely protect themselves, they need to learn that marketing claims by antivirus vendors aren’t necessarily true – such as the slogans proclaimed by the Flash applet on ESET’s home page.

  • Randy Abrams

    My goals haven’t changed at all. I am not complaining that I don’t like the rules. I am simply pointing out the dishonesty in the assertions of the organizers. It is simply ludicrous to think this contest will be noticed by the general public. They don’t even know of defcon. The vast majority of the general public doesn’t even know what a bot or a botnet is. This really will be a complete unknown to the general public.

    That the contest is pointless and that the organizers are not being straight forward are not mutually exclusive facts. Talk about one aspect and the other as well does not constitute a change of goals or opinions.

    Randy Abrams
    Director of Technical Education
    ESET LLC

  • eidolon sniper

    Randy,

    I will have to agree with you that this contest will be largely unknown to the public, which I think is a shame.

    If we take that premise into account, however, your arguments then fall apart. For a pointless contest that will supposedly go unnoticed and achieve absolutely nothing, it seems to be receiving quite an amount of vitriol from the antivirus industry.

  • Randy Abrams

    The argument doesn’t fall apart at all. It is a pointless contest that creates new malware that most likely will get out into the public. Most of the malware goes unnoticed by the general public too, but it doesn’t mean it isn’t a problem. Your argument is like the head of Sony BMG saying that most people don’t know what a rootkit is so they shouldn’t worry about them :)

    Randy Abrams
    Director of Technical Education
    ESET LLC

  • eidolon sniper

    That’s where you start passing off your own assumptions as facts. Your whole argument rests on the premise that these viruses will be released into the wild.

  • solcroft


    They don’t even know of defcon. The vast majority of the general public doesn’t even know what a bot or a botnet is.

    And that’s the exact problem. The general public have no idea of the extent of the problem; all they hear each day are the ads from antivirus vendors and delusional tests like Virus Bulletin proclaiming how antiviruses detect “100%” of viruses in the wild. And when someone actually tries to raise awareness, the antivirus companies try to shoot them down, fearful that their monopoly on the propaganda will be broken.

    It is interesting to note that the previous post on this blog claims concerns that the public will be misled by the hype and misinformation created by this contest, while you are rushing to dismiss it as insignificant. Perhaps you guys could actually sit down and agree on what to spin to the public first, before you go out and make conflicting arguments?

  • ZimmerBlot

    It appears you are looking at an older versions of the site as the current text on the site is different than that which you quote. You may also want to read the motivations section which talks around the points you comment on, perhaps more clearly:

    –(http://www.racetozero.net/motivations.html)

    Beyond the format of a public contest nothing being proposed for the Race to Zero is really that new. The AV arms race has been going on for a long time, and the ability of attackers to modify code to bypass AV detection is widely known.

    So if that is the case, why did we decide to organise Race to Zero, what were our motivations?

    As we all know security is not a point solution but a continually evolving process, and as such it is vitally important that defensive measures taken by businesses and individuals can keep up with the evolving range of threats. It is also a sad truth that security afforded to an entity is directly (though not soley) related to the security spend available to that entity. How much to spend, and where best to spend it is a question tackled by security professionals worldwide everyday. With that in mind we are interested in gathering data to aid research into quantifying a number of areas from the perspective of an attacker:

    -= the real world difficulty of avoiding detection by the different classes of AV
    -= the associated costs in both time and money
    -= the level of skill required
    -= the techniques which are successfully able to avoid triggering threat identification

    Quantifying how much an attacker must invest to circumvent the defences that a defender has invested in is a key part of being able to evaluate where best to place security spend to gain the most benefit. Race to Zero is one way in which we as researchers can proactively answer these and other questions, while at the same time challenging some of the best minds available in the security community. Race to Zero will help to illustrate clearly the level of sophistication that AV avoidance techniques have achieved, in addition to indicating if new complementary anti-malware techniques are needed to combat such threats. In a nutshell we are interested in researching the relative costs involved in the AV ‘arms race’ in both a quantitative and qualitative way

    It is also hoped that the open and public nature of Race to Zero will stimulate full and frank discussions across the information security community around the difficulties of detecting increasingly sophisticated malware; in addition to discussion around ways to improve detection techniques.

    We hope to be able to give a presentation of findings from Race to Zero at DefCon, a paper has been submitted but a decision on it has not yet been made. Following the contest, when further analysis has been conducted, a technical paper will be publicly released.

    At the conclusion of the contest when everybody realises the world hasn’t ended (and perhaps admitted to themselves that it was all overhyped from the start?) – maybe people will spend time enough to analyse the issues that have been raised, as well as the data that has been gathered to see if there are things that can be learnt about how to deal with the out of control arms race that is the current situation with malware it all its forms.

    It really seems the AV vendors have their backs against the wall, and if the contest is as worthless as you make it out to be why are all the AV vendors public relations people making so much fuss about it? Surely if it were so worthless it would be best to just give it as little publicity as possible?
    You appear to have spent time picking holes in the contests apparent contradictions, but appear to be blind to your own and that of the other AV vendors who are attacking this with such vitriol.

  • Arun

    Randy,

    I’m quite disappointed by the reaction of the AV companies with regards to this contest. If you don’t like what you see, ignore it. The rhetoric like the one in your post above seem to suggest that you are somehow afraid of such events (as if the script kiddies and malware for profit writers actually need an event to peddle their wares).

    I’m a NOD32 user, have been for a while, and am quite happy with the performance of the product. But reading your blog post seems to suggest that you (ESET and the AV industry in general) are out of ideas and are somehow apprehensive about such “contests” proving such a case.
    I am quite willing to give you the benefit of the doubt (and hoping as well) that, that isn’t the case, as it will be a dark prospect for the average consumer indeed. But posts such as the above where you give these organisers more credit than they’re worth suggests otherwise.

    Keep up the good work in the fight against malware!

  • David Harley

    I don’t think anyone in the research community would object to spreading the message that AV isn’t the 100% solution. Security – even the anti-malware bit of it – needs to be multi-layered. Not only does it require other defensive measures, it requires the end user to take some responsibility for their own actions, and there is -still- no absolute guarantee of total protection.

    However, this contest is sending a number of other messages that are rather more controversial and in some instances downright misleading.

  • Randy Abrams

    > eidolon sniper Says:
    > That’s where you start passing off your own assumptions as
    > facts. Your whole argument rests on the premise that these
    > viruses will be released into the wild.

    The fact is that an AV company has to treat them as if they will be. the fact is that the organizers do not require providing samples because they really don’t care about mitigating risk.

    The fact is that you take a tiny bit of what I say and call it a “whole assumption”

    Randy

  • Randy Abrams

    Hi Arun,

    > Arun Says:

    > I’m quite disappointed by the reaction of the AV
    > companies with regards to this contest.

    Therein lies the misconception. It isn’t the AV companies who are complaining at all. It isn’t the sales or marketing departments at the AV companies. It is *individuals” in the research community who, incidentally, are more often at odds with our own marketing departments that in agreement, who are complaining. The people actually trying to do something about the problem to the best of our abilities are complaining about the people creating more malware for shameless hype and self promotion, rather than trying to help the cause. I’ll post another blog and explain a bit more about both the AV industry and a correct approach the race to zero organizers could have taken. granted, it would take more work and they would have to raise the bar on the skill level required to enter the contest.

    > I’m a NOD32 user, have been for a while, and am quite
    > happy with the performance of the product.

    I’m happy to hear that. We all like to have our work appreciated.

    > But reading your blog post seems to suggest that you
    > (ESET and the AV industry in general) are out of ideas
    > and are somehow apprehensive about such “contests�
    > proving such a case.

    I often prove the case when I am teaching about antivirus and security. I actually tell people what the “in the wild” term means and the small nature of the test set involved in a VB 100.

    Off to wrtie the next blog now :)

    > Keep up the good work in the fight against malware!

    Thanks!

    Randy Abrams
    Director of Technical Education
    ESET LLC

  • Randy Abrams

    >ZimmerBlot Says:

    > It appears you are looking at an older versions of the
    > site as the current text on the site is different than
    > that which you quote. You may also want to read the
    > motivations section which talks around the points you
    > comment on, perhaps more clearly:

    Thanks for the information. I have not revisited the site recently.

    On the site under the rules section it is stated:

    >4. Modified samples will not be submitted to antivirus
    > vendors unless authorised by contest participants

    You actually think it is responsible to promote the creation of new malware? You know as well as I do, and most security professionals do how easy it is to modify code to become undetected. what do you think the motive of a person creating undetected malware who doesn’t want an AV company to have a sample is?

    Requiring that all samples be submitted to AV companies is certainly something the organizers could require without compromising the “motivation” of the contest.

    And no, I don’t think that submitting samples then make the contest good or ok, but I also recognize that I’m not going to stop this exercise in lunacy either.

    Randy Abrams
    Director of Technical Education
    ESET LLC

  • http://anti-virus-rants.blogspot.com kurt wismer

    @solcroft:
    “There is no argument in abolishing antivirus software; it is the antivirus industry who is putting the words into the mouth of others.”

    sorry, but to the layperson “anti-virus is dead” (which appears, or at least appeared on their site) means exactly that… ‘anti-virus is broken, it doesn’t work, you shouldn’t be paying for it and you shouldn’t be using it’ – that is what a layperson would take away from the phrase “anti-virus is dead” and that is the message a layperson would take away from this contest…

  • @kurt wismer

    @kurt wismer

    That isn’t the aim of the contest at all, though I’m sure that little fact won’t stop people from creating and imposing their little preconceptions.

    Even if it was – let’s assume that for the sake of argument – would it be bad or inaccurate? Would it be such a despicable thing if a few people stood up and took notice, and thought to themselves, “Hey, maybe there are better ways to beat the virus problem, and I’d like to find out more about them”? Here Mr Abrams displays no compunction against heaping ridicule on the possibility of the contest influencing people to not use layered defenses – I have to wonder if he shares that same ridicule for the general populace at large who have no layered defenses to speak of; the antivirus is their one and only layer. Let us make no mistake about what Mr Abrams really meant; his concern was not about people being influenced away from layered defenses at all (because they never had any to begin with).

  • solcroft

    @kurt wismer

    That isn’t the aim of the contest at all, though I’m sure that little fact won’t stop people from creating and imposing their little preconceptions.

    Even if it was – let’s assume that for the sake of argument – would it be bad or inaccurate? Would it be such a despicable thing if a few people stood up and took notice, and thought to themselves, “Hey, maybe there are better ways to beat the virus problem, and I’d like to find out more about themâ€?? Here Mr Abrams displays no compunction against heaping ridicule on the possibility of the contest influencing people to not use layered defenses – I have to wonder if he shares that same ridicule for the general populace at large who have no layered defenses to speak of; the antivirus is their one and only layer. Let us make no mistake about what Mr Abrams really meant; his concern was not about people being influenced away from layered defenses at all (because they never had any to begin with).

  • Randy Abrams

    > Let us make no mistake about what Mr Abrams really meant;
    > his concern was not about people being influenced away from
    > layered defenses at all (because they never had any to begin
    > with).

    On the contrary. I try to teach people to adopt a layered defense approach, regardless of whether they currently have it or not.

    If you don’t know what I mean, and you obviously don’t, then ask. I’m not hard to find.

    One is a layer and the Race to Zero crowd is encouraging people to take away that layer while I and other responsible professionals try to encourage them to add layers.

    Randy Abrams
    Director of Technical Education
    ESET LLC

  • solcroft

    Oh COME on Mr Abrams. Are you seriously saying that people are going to strip themselves of all security when they find out about this contest? And that it is you and other responsible security professionals – as opposed to simple common sense, which I am sure many of these people possess – who will save the day?

  • solcroft

    Not only do you take it upon yourself to dictate what the contest aims are, you also deem yourself fit to decide for the public what they will do when they learn of this contest. I wonder if any average Joe Schmoes reading this blog are feeling insulted by the insinuation that they are idiots and fools to be herded this way and that, and be told by self-proclaimed responsible security professionals what they should think, like children being lectured by parents.

  • Randy Abrams

    Actually, I clearly do not take it upon myself to dictate what the contest aims are. This is a blog, not a text book. Most people recognize that people express opinions in a blog. I would guess that this is probably obvious to the vast majority of blog readers.

    I also do not decide what the public will do. I make observations about what some people will do. If I meant that everyone would do the same thing I would have clearly said so. To try to twist the words to read “everyone”, or even “most people” is clearly disingenuous. I think the “average” Joe Schmoes are far more than smart enough to realize this.

    If you stop to think about how many people fall for phishing schemes, and that even then it is a very small portion of the people who receive them, you would easily understand that there will actually be a significant number of people who believe that they should ditch their AV. I don’t care if they buy ESET or download a free AV product, I believe they do need to have the protection.

    Perhaps if childish contests weren’t sponsored, parental lectures wouldn’t be needed :)

    Randy Abrams
    Director of Technical Education

  • solcroft

    You can’t really have it both ways, Mr Abrams. There are people who will draw the wrong conclusions from this contest, and then there are people who will draw the right ones. Of course, it’s up to one’s own arbitrary definition of how many there will be from each group, pretend that it will be true, and then argue accordingly to one’s own premise. But what you cannot do is focus only on the former group, yet dismiss the latter by claiming that the public won’t know of this contest. That is selective logic in action – only the stupid people will find out about the contest, but the reasonable people, who will benefit from it, won’t.

  • Yousuf

    @ Solcroft…

    An upfront suggestion is for you to please share your knowledge (instead of here, in such blogs) on a website aptly URLed http://www.computersecurityrealities.com

    This way, your expertise will be there for all to see, read and avail benefit from.

    Especially for those who aren’t into computers and stuff…like me or you…or Randy for example.

    Knowledge should be shared. Randy is doing his part at ESET.

    And many others, like you and Randy contribute their efforts.

    Instead of pushing him or anybody else, lets invent something new.

    I know that you have understood my point of view. Just think with a proactive positivity.

    Regards,

Follow Us

Automatically receive new posts via email:

Delivered by FeedBurner

3 articles related to:
Hot Topic
29 May 2008
ESET Virus Radar

Archives

Select month
Copyright © 2014 ESET, All Rights Reserved.