Linux/Cdorked.A: New Apache backdoor being used in the wild to serve Blackhole

Last week, our friends at Sucuri sent us a modified version of an Apache webserver redirecting some of its requests to the infamous Blackhole exploit packs. Sucuri has published a blog post on this attack. (You can find more about Blackhole here.)

Our analysis of this malware, dubbed Linux/Cdorked.A, reveals that it is a sophisticated and stealthy backdoor meant to drive traffic to malicious websites. We urge system administrators to check their servers and verify that they are not affected by this threat. Detailed instructions to perform this check are provided below. (Update 5/1/2013: An improved tool coded in C replaced the Python script we originally published.)

In fact, Linux/Cdorked.A is one of the most sophisticated Apache backdoors we have seen so far. Although we are still processing the data, our Livegrid system reports hundreds of compromised servers. The backdoor leaves no traces of compromised hosts on the hard drive other than its modified httpd binary, thereby complicating forensics analysis. All of the information related to the backdoor is stored in shared memory. The configuration is pushed by the attacker through obfuscated HTTP requests that aren’t logged in normal Apache logs. This means that no command and control information is stored anywhere on the system.

Technical analysis of Linux/Cdorked

Here we provide the first technical analysis of Linux/Cdorked, which seems to be affecting hundreds of webservers right now. In the Linux/Cdorked binary all the important or suspicious strings are encrypted. As shown in the following image, a function is responsible for decrypting the strings on demand with a static XOR key.

The version of Linux/Cdorked that we have analyzed contains a total of 70 strings that are encoded this way. As shown in the following screenshot, the key used for encoding the data is 27A4E2DADAF183B51E3DA7F6C9E6239CDFC8A2E50A60E05F.

As mentioned before, Linux/Cdorked does not write any files on the disk. Instead, it allocates around six megabytes of shared memory to keep its state and configuration information. This memory block, a POSIX shared region of memory (shm), is used by all Apache subprocesses but can also be accessed by any other process since the malware authors didn’t limit its permission.  The following screenshot shows the (read, write for everyone) permission rights assigned to the shared memory region.

There are two ways the attacker can control the behavior of the backdoored server: through a reverse connect shell or through special commands, all of them are triggered via HTTP requests.

The Linux/Cdorked.A backdoor

The HTTP server is equipped with a reverse connect backdoor that can be triggered via a special HTTP GET request. It is invoked when a request to a special path is performed with a query string in a particular format, containing the hostname and port to connect. The client IP of the HTTP dialog is used as a key to decrypt the query string as a 4 byte XOR key. Additionally, IP specified in X-Real-IP or X-Forwarded-For headers will override the client IP as the XOR key. This means we can craft a X-Real-IP header that will in effect be a “\x00\x00\x00\x00” key. Query string also needs to be hex-encoded before sending.

While the shell is used by the attacker, the HTTP connection creating it is hung (the backdoor code does not implement forking). This implies that malicious shells can be found if one has access to the server and checks for long-running HTTP connections. On the other hand, the HTTP request does not appear in Apache’s log file due to the way the malicious code is hooked into Apache.

Redirection in Linux/Cdorked.A

When redirecting a client, the malware adds base64 encoded string to the query containing information like the original visited URL and whether or not the request was originally to a javascript file so the server could provide the right payload.

An example redirection looks like:

Location: hxxp://dcb84fc82e1f7b01. xxxxxxgsm.be/index.php?j=anM9MSZudmNiaW11Zj1jY3
Zja3FqdSZ0aW1lPTEzMDQxNjE4MjctMzYwNDUzNjUwJnNyYz0yMzImc3VybD13d3cuaW5mZWN0ZWRzZXJ2
ZXIuY29tJnNwb3J0PTgwJmtleT0xM0Q5MDk1MCZzdXJpPS9mb3J1bS93Y2YvanMvM3JkUGFydHkvcHJvdG
9hY3Vsb3VzLjEuOC4yLm1pbi5qcw==

After decoding, the following parameters appear:

js=1&nvcbimuf=ccvckqju&time=1304161827-360453650&src=232&surl=www.infectedserver
.com&sport=80&key=13D90950&suri=/forum/wcf/js/3rdParty/protoaculous.1.8.2.min.js

The “surl” parameter shows the infected host and the “suri” indicates what the original requested resource was.

After the redirection, a web cookie is set on the client so it is not redirected again. This cookie is also set if a request is made to a page that looks like an administration page. The backdoor will check if the URL, the server name, or the referrer matches any of the following strings : ‘*adm*’, ‘*webmaster*’, ‘*submit*’, ‘*stat*’, ‘*mrtg*’, ‘*webmin*’, ‘*cpanel*’, ‘*memb*’, ‘*bucks*’, ‘*bill*’, ‘*host*’, ‘*secur*’, ‘*support*’.  This is probably done to avoid sending malicious content to administrators of the website, making the infection harder to spot. The following screenshot shows part of the code responsible for handling the web cookie.

A few other conditions must be met before redirection happens; for example, a check is done for the presence of the Accept-Language, Accept-Encoding, and Referrer header.

Other Linux/Cdorked.A commands

We found 23 commands in Linux/Cdorked.A that can be sent to the server via a POST to a specially crafted URL. The request must also contain a cookie header starting with “SECID=”. The query string value must hold 2 hex encoded bytes that are encrypted with the client IP, using the same technique as the shell. The SECID cookie data will be used as arguments to some of the commands. We believe that the URLs to redirect clients are sent to the backdoor using this method. The redirection information will be stored encrypted in the allocated shared memory region. We also believe that the conditions for redirection are set this way, for example, a white list of user agents to redirect can be preconfigured and a black list of IPs to avoid redirection.

This is the complete list of commands found in the binary we have analysed:  ‘DU’, ‘ST’, ‘T1′, ‘L1′, ‘D1′, ‘L2′, ‘D2′, ‘L3′, ‘D3′, ‘L4′, ‘D4′, ‘L5′, ‘D5′, ‘L6′, ‘D6′, ‘L7′, ‘D7′, ‘L8′, ‘D8′, ‘L9′, ‘D9′, ‘LA’, ‘DA’.

Finally, some information about the status of the backdoor is returned in the ETag HTTP header, as shown in the screenshot below. We are still investigating the purpose of each of the commands and will publish our results as soon as the analysis is completed. In short, they all either add content to, or remove it from, the configuration in the shared memory region.

Linux/Cdorked.A Remediation

As previously mentioned, the permissions on the shared memory allocation are loose. This allows other process to access to memory. We have made a free tool to allow systems administrators to verify the presence of the shared memory region and dump its content into a file (you can download dump_cdorked_config.c or copy and paste the code from the listing below).

We also recommend using debsums for Debian or Ubuntu systems and `rpm –verify` for RPM based systems, to verify the integrity of your Apache web server package installation. (However, remember to temper this advice with the reality that the package manifest could have been altered by an attacker.) Checking for the presence of the shared memory is the recommended way to make sure you are not infected. We would be interested in receiving any memory dumps for further analysis.

At the time of writing, the ESET Livegrid monitoring system is showing hundreds of webservers that seem to be affected by this backdoor with thousands of visitors being redirected to malicious content. We will publish more information on the scale and complexity of this operation in the days to come.

The following researchers contributed to this report: Olivier Bilodeau, François Chagnon, Alexis Dorais-Joncas, Sebastien Duquette and Marc-Étienne Léveillé.

Resources:

SHA1 of the analyzed binary: 24e3ebc0c5a28ba433dfa69c169a8dd90e05c429

Code for the shared memory dump tool: dump_cdorked_config.c

// This program dumps the content of a shared memory block
// used by Linux/Cdorked.A into a file named httpd_cdorked_config.bin
// when the machine is infected.
//
// Some of the data is encrypted. If your server is infected and you
// would like to help, please send the httpd_cdorked_config.bin
// and your httpd executable to our lab for analysis. Thanks!
//
// Build with gcc -o dump_cdorked_config dump_cdorked_config.c
//
// Marc-Etienne M.Léveillé <leveille@eset.com>
//

#include <stdio.h>
#include <sys/shm.h>

#define CDORKED_SHM_SIZE (6118512)
#define CDORKED_OUTFILE "httpd_cdorked_config.bin"

int main (int argc, char *argv[]) {
    int maxkey, id, shmid, infected = 0;
    struct shm_info shm_info;
    struct shmid_ds shmds;
    void * cdorked_data;
    FILE * outfile;

    maxkey = shmctl(0, SHM_INFO, (void *) &shm_info);
    for(id = 0; id <= maxkey; id++) {
        shmid = shmctl(id, SHM_STAT, &shmds);
        if (shmid < 0)
            continue;

        if(shmds.shm_segsz == CDORKED_SHM_SIZE) {
            // We have a matching Cdorked memory segment
            infected++;
            printf("A shared memory matching Cdorked signature was found.\n");
            printf("You should check your HTTP server's executable file integrity.\n");

            cdorked_data = shmat(shmid, NULL, 0666);
            if(cdorked_data != NULL) {
                outfile = fopen(CDORKED_OUTFILE, "wb");
                if(outfile == NULL) {
                    printf("Could not open file %s for writing.", CDORKED_OUTFILE);
                }
                else {
                    fwrite(cdorked_data, CDORKED_SHM_SIZE, 1, outfile);
                    fclose(outfile);

                    printf("The Cdorked configuration was dumped in the %s file.\n\n", CDORKED_OUTFILE);
                }
            }
        }
    }
    if(infected == 0) {
        printf("No shared memory matching Cdorked signature was found.\n");
        printf("To further verify your server, run \"ipcs -m -p\" and look");
        printf(" for a memory segments created by your http server.\n");
    }
    else {
        printf("If you would like to help us in our research on Cdorked, ");
        printf("please send the httpd_cdorked_config.bin and your httpd executable file ");
        printf("to our lab for analysis at leveille@eset.com. Thanks!\n");
    }
    return infected;
}

Author Pierre-Marc Bureau, ESET

  • http://www.facebook.com/walkerbert Bert Walker

    I miss DOS, and the pre-hack-war era.

  • David Warren

    I get an invalid Syntax error (Python 2.4) on line 34.

    • Stephen Cobb

      Check the other comments, that may fix it.

      • David Warren

        Sorry Stephen, you can’t see comments until you post yours. Thank you so much!

  • afbach

    Your code has:
    shmid = shmget(SHM_KEY, SHM_SIZE, 0o666)

    python complains – just “0666″ maybe?

    • Stephen Cobb

      So, 0o666 is the octal representation of the shared mem permissions (rw-rw-rw-), but doesn’t work with Python 2.5 and earlier. Try 438 instead of 0o666, or 0666.

  • Ricky Hignite

    I get an error:

    File “dump_cdorked_config.py”, line 34

    shmid = shmget(SHM_KEY, SHM_SIZE, 0o666)

    ^

    SyntaxError: invalid syntax

    • Stephen Cobb

      0o666 is the octal representation of the shared mem permissions (rw-rw-rw-), but doesn’t work with Python 2.5 and earlier.

      You can change the decimal representation (so use 438 instead of 0o666).

  • CodeWolf

    Is there an update on the findings or detection of this threat? Thanks!

  • londonbloke67

    Hi,

    I tried changing the 0o666 to declmal, ran the script again, and got..

    Traceback (most recent call last):
    File “./check_shared_memory.py”, line 15, in ?
    from ctypes import *
    ImportError: No module named ctypes

  • Zzz

    How is the backdoor installed in the first place? Care of share that angle?

    Fancy stuff after compromize is not as interesting, makes a sensational read though.

  • Mark

    Thanks for the great information. Is there any method to check for the backdoor using Python 2.4?

    On all the servers that we run using CentOS we get the following error message (after fixing the octal permissions as indicated):

    Traceback (most recent call last):
    File “dump_cdorked_config.py”, line 15, in ?
    from ctypes import *
    ImportError: No module named ctypes

    Thanks!

  • Sébastien Duquette

    Hi Mark,

    You need to install the ctypes library, it should be provided by the python-ctypes package on CentOS. Please note that we have released a new C program as the python script was not working will all variants of Linux/Cdorked, you can find it at the end of the article. You will need gcc to compile it, the compilation command is specified in the comment at the top of the file. Let us know if you have any issues.

    Thanks,

    Sébastien

  • Weng Fu

    Why does not the CIA track down the criminals who write these codes and either jail or execute them?

  • Stephen Cobb

    Mark – We have updated the code. There is now a C program that will work for you, and it detects more variants than the original python script.

  • Stephen Cobb

    We are working on it. Hope to have something new on Monday, if not sooner.

  • Stephen Cobb

    I take your point, and believe me, we are researching the method of compromise. Are they getting in via Plesk, cpanel, Drupal, WordPress? Are they paying off sysadmins?

    I would say the fancy stuff is not just about sensationalism, it is important to know that this malware is much harder to detect than older malware, relative to the level of management normally accorded to LAMP systems.

    • kamicc olo

      So, is there any information what do those thousands of compromised servers have in common? I hope it might narrow the search of way of compromisation easier…

  • dharleyatESET

    I’m not sure that sort of investigation is within the CIA’s remit.

    • http://k0nsl.org/blog k0nsl

      LOL, that’s true. But really, one have to admire the tenacity in creating applications such as this. I think it admirable.

  • Alessandro Forghieri

    I have written a perl version of the test script (which does not
    require non-core modules). As I have no way to test it on an infected
    server, I canot validate THAT case (but it should work as the
    translation is very straightforward). It can be downloaded form
    pastebin: http://pastebin.com/EittPYmK

  • http://twitter.com/timstoop Tim Stoop

    Hey ESET,

    Great work! We’ve used your code to create an easy to deploy Nagios/Icinga check for checking this. Especially useful for large webserver deployments. You can find info here: https://blog.kumina.nl/2013/05/checking-for-linuxcdorked-a/

    Hope this helps!

  • dharleyatESET

    Afraid not.

  • Malware Student

    Why are the X-Real-IP and the special path to get the reverse shell censored? It’s not clear what is the method to obtain them.

  • Valentino

    Hello ESET.

    I’m wondering if Hetzner was hit exactly by Linux/Cdorked.A:

    “The malicious code used in the backdoor exclusively infects the RAM. First
    analysis suggests that the malicious code directly infiltrates running Apache
    and sshd processes. Here, the infection neither modifies the binaries of the
    service which has been compromised, nor does it restart the service which has
    been affected.”

    Rif: http://pastebin.com/raw.php?i=rj8FA7pC
    Regards.

  • xRepinsSporx

    Only fix for the malware is reinstalling correct?

  • floyd

    I wrote a nagios plugin to check for, among others, the cdorked infection since it is too check all servers manually. Just a little helper, not the magic bullet, ofc ;)

    https://github.com/bernhardbrunner/nagios-check-infections

Follow Us

Automatically receive new posts via email:

Delivered by FeedBurner

26 articles related to:
Hot Topic
ESET Virus Radar

Archives

Select month
Copyright © 2014 ESET, All Rights Reserved.