Superfish: Lenovo goes on the bloatware offensive

Its been just under three weeks since February 19th, when Lenovo became entangled in a web of controversy over its preinstallation of Superfish’s Visual Search adware on some of its popular consumer laptops during last year’s holiday shopping season.

Previously, I have remained very quiet on this subject, not because of my position as ESET’s Distinguished Researcher or because I am a Microsoft MVP, but because for the past several years I have been volunteering in my spare time to help answer questions in Lenovo’s online support forum, as well as answering questions via social media. As such, I have been in a rather unusual position of being able to see what has been happening, from both the viewpoint of a computer manufacturer that had distributed adware, to that of a company dedicated to preventing it. It has also been a difficult position for me to be in: Over the past couple of weeks there have been numerous occasions where one party has shared information that would be very useful to the other, and I have been unable to inform the other party because it was shared in confidence

Last week, through having a foot in these two worlds, I had a unique opportunity to videoconference with Lenovo Group’s CTO Dr. Peter Hortensius for an hour. Based on those discussions, and following Lenovo’s new promise on Friday, I feel it is time to comment publicly on the issue.

Now, I am not going to delve into the technical details of this adware since my colleague Stephen Cobb has discussed the Superfish adware in detail in this blog post, and ESET’s knowledgebase team has done their usual excellent job here, so what I am going to do, instead, is take a look at Lenovo’s response.

Note: you can download a standalone ESET Superfish cleaner here.

Timeline

Initial reports of problems with Superfish Visual Search began to appear in Lenovo’s support forum during September and October of 2014. In January, Lenovo discontinued preloading Superfish onto consumer laptops as well as disabling the connections to servers it used. This information was shared by Lenovo in their initial statement on Thursday, February 19th.

On the same day, an interview with Dr. Hortensius was posted in the Wall Street Journal’s blog, in which he seemingly downplayed the risks associated with the adware, suggesting they were more “theoretical concerns” and then went on to explain that Lenovo would be doing more in the future to vet what software was preloaded on its computers. This comment ignited a firestorm of controversy amongst members of the security community, many of whom were concerned not so much with the adware per se, but that it injected the same trusted root certificate authority into every computer on which it was installed. Instructions on how to obtain the certificate manually appeared online on February 19th as well, and turned out to be from an SDK offered by another software vendor named Komodia. ESET added detection for the adware on this day, too.

On Friday, February 20th, Lenovo issued a second statement on Superfish, announcing that they were providing a tool to uninstall the adware, as well as working with both Microsoft and McAfee to provide automatic detection and remove of both the Superfish adware and its certificate. Source code that could be used to automate decryption using Superfish’s certificate began to appear online as well. As of today (March 9th), we have not seen that code weaponized into malware or attack tools.

Figure 1: Lenovo PSIRT advisory

Figure 1: Lenovo PSIRT advisory

The Lenovo PSIRT (Product Security Incident Response Team) also published Lenovo Product Security advisory LEN-2015-010, Superfish Vulnerability, which focused not on the characteristics of the Superfish adware but on the risk that its self-signed root certificate could lead to HTTPS traffic interception. In conjunction with this, Superfish Uninstall Instructions were published, giving an automatic tool to uninstall the software and its certificate, as well as manual instructions to do so. Lastly, Lenovo released the source code for its automated uninstall tool under the MPL public license.

On Monday, February 23rd, Lenovo issued a third statement, an open letter from Dr. Hortensius affirming that Lenovo had ceased doing business with Superfish, that it would not preload the adware again on computers, that Symantec had updated their software to detect and remediate it, and that Lenovo was working on a plan to address these issues in the future.

Figure 2: Lenovo's February 27th news release

Figure 2: Lenovo’s February 27th news release

On Friday, February 27th, Lenovo announced its promise for cleaner, safer PCs. In it, they announced they would begin changing their practices to preload only software necessary for the operation of the computer (the operating system, related software, device drivers, plus programs necessary to take advantage of hardware not supported by the operating system). However, third-party security software would still be preloaded onto systems, as well as programs which are a requirement for certain markets on a country-by-country basis.  Microsoft is not always the preferred browser nor is Google the top search engine in some regions of  the world, and Lenovo has to ship systems which reflect these realities. But most importantly, “adware” and “bloatware” would no longer be a part of the Lenovo PC experience.

Evaluating Lenovo’s Response

So, how well exactly did Lenovo do with their response? Through the fourth quarter of 2014, there was no response from any part of the Lenovo community, beyond comments from users in their forum ranging from suggesting the de-installation of any unwanted software to advocating the wiping of the drive and installing a clean copy of Windows, and so forth.

While it may seem that such comments could have served as some sort of warning to Lenovo that something was amiss, it is important to note that these types of complaints and responses are not unique to Lenovo or Superfish. A quick search of the support forums of Hewlett-Packard and Dell, two of Lenovo’s competitors, reveals similar complaints about their preloaded software:

Figure 3: Search results from Hewlett-Packard

Figure 3: Search results from Hewlett-Packard

 

Figure 4: Search Results from Dell

Figure 4: Search Results from Dell

It is reasonable to state that while some may consider Lenovo’s initial response lackluster, it is not particularly unique unusual, either. The problem with computers being preloaded is not unique to Lenovo, but rather an industry-wide problem, dating back many years. A discussion of the reasons behind it fall out-of-scope for this article, but I would refer you to two fascinating articles by journalist Ed Bott for more information, How a decade of antitrust oversite has changed your PC and Is it time to force PC makers to disclose how much they make from crapware?. The law of unintended consequences, as Bott called it, is indeed very longlived.

Untitled1

Figure 5: Lenovo timeline

Starting on February 19th, and over the course of the next eight days, Lenovo went from having seemingly fumbled its initial response, to issuing four press releases, revising its security advisory four times, and at least half-a-dozen changes to the source code for its automated removal tool, culminating in their February 27th pledge to clean up their PCs. Here are some of its highlights:

  • Lenovo has worked with partners to create tools and update anti-malware programs to eliminate the Superfish adware and trusted root certificate.
  • Beginning immediately with Windows 8 and going forwarding into Windows 10, Lenovo’s standard computer image will contain only the operating system and related applications (device drivers, software to make use of unique hardware not supported in the operating systems, security software and Lenovo’s own applications).
  • Lenovo will share information with customers about all preloaded software, explaining clearly explains what each one does.
  • Lenovo will continue to work with their user community and industry experts to ensure only appropriate applications are installed and that they provide the best user experience.

For a global company with over 50,000 employees, accomplishing all of these actions in eight days is impressive.

The Undiscovered Country

So, where does this leave Lenovo’s current and prospective customers?

Given the way the retail channel works, it is possible there may still be a few computers out there with the Superfish adware and trusted root certificate preloaded. Customers who buy one of these machines can remove them using a variety of means, ranging from following Lenovo’s own instructions to making use of anti-malware programs.

By now, almost all reputable anti-malware programs should be effective in removing the adware and certificate from affected computers, but it’s important to remember that if the operating system is restored from a recovery partition or from recovery media, removal will need to be repeated after the system has been restored. While this is unlikely to be a major problem since Superfish will be caught and removed as the system updates, it will be a recurring issue for as long as the recovery image contains the Superfish adware.

For businesses (or individuals) who buy business-grade systems, such as Lenovo’s Think-branded systems, this never was an issue: The Superfish adware was only preinstalled on some of Lenovo’s consumer laptop offerings. Business-grade systems, whether from Lenovo, Dell, Hewlett-Packard or other manufacturers, are marketed differently than their consumer lines. They tend to have features such as more rugged construction, easy-access chassis for quick repairs, support for remote monitoring and management tools, and have a minimum amount of third-party software preloaded on them. They also cost a bit more, too, because of these features.

Just as it is important to note the models of Lenovo computer that were affected, it’s also important to note that some of the affected consumer models shipped without it during the time-frame. In particular, Microsoft’s own online and retail stores sell what they call Signature Edition PCs, which consist of computers from a variety of manufacturers, including Lenovo. These are sold without any preloaded third-party software, and as a result may be faster than the same models when bought from the manufacturer because fewer programs automatically run when the computer is started. They may also be priced slightly higher, too. This varies, though, based on promotions and discounts.

For consumer customers who were considering systems from Lenovo and may now be having second thoughts, all is not lost for you: As Lenovo’s promise for clean PCs unfolds, you will be in a position to reap its benefits. While they will likely come preloaded with some third-party anti-malware software, if it is a brand you do not favor, you should have no trouble uninstalling and replacing it with software from your favorite security vendor.

Aryeh Goretsky, MVP, ZCSE
Distinguished Researcher


Did the last computer you purchased come with unwanted software on it? If so, what was it and how did you deal with it? What do you think of Lenovo’s response? Is it enough, or are there more steps you feel they should take? Let us know in your comments, below.

 

Author Aryeh Goretsky, ESET

  • Coyote

    “Its been just under three weeks since February 19th, when Lenovo became
    entangled in a web of controversy over its preinstallation of
    Superfish’s Visual Search adware on some of its popular consumer laptops during last year’s holiday shopping season.”

    Excellent place for an excellent pun (of course, in my mind, every place is an excellent place for a pun).

    Indeed, though, it is true that preinstallation is a problem, and old problem too, but I would argue that there are different types of preinstalled software, some of which is legit and some of which isn’t (particularly, if it isn’t something someone would willingly know about and install by choice, then it isn’t legit for it to be preinstalled, either). Call that a theory or otherwise up to debate – it probably is up to debate. But still, the point is there is a difference between bloat by feature (which is how I always view it even though a bloated feature list isn’t necessarily desired, it is different from what this subject is) and ‘bloat’ like Superfish is. Of course, I would argue that much of Windows is bloat but I am biased so I will not at all state more than that on that subject. What I will argue, though, is bloat is size and resources specific: after all, was monkey, brain, cih, kriz, etc. bloated ? No, they weren’t – they were compact and efficient. But the last two especially were very dangerous (none of them were adware, either, I might add, nor were they spyware or any other ware except malware).

    • Hello,

      Actually, I thought Brain was a little high overhead, but, then again, there wasn’t exactly a baseline to compare the first DOS PC virus against at the time. Thanks for sharing your comments with us.

      Regards,

      Aryeh Goretsky

      • Coyote

        Well the idea was more of its size. Of course, size isn’t exactly an indication of resource consumption (one simple command can drop numerous Unices to their knees if not properly configured and so can a very small, compact C program – it is efficient too albeit efficient at halting the system.. would take no more than 20 seconds to write, if not a fair bit less than that). And indeed, less of a baseline. In any case, I think the takeaway of my response was how the software (whatever it may be) is used. I guess you could argue that it is sort of like a trojan horse in that was superfish really what it is claimed to be? If you want to go further than that (which I would personally because it brings more to my point) some worms were designed to remove other worms. But question: is the second worm any more appreciated than the first? (One example of many though the name escapes me at this time) Some would argue no and certainly it is still unauthorised. And as for the part on bloat specifically, and comparing certain viruses of old: it was a combination as mentioned – size, efficient and while some were also slower than others (HLL viruses instead of LLL for instance [although not always a 1:1 assembly certainly can allow it]), that doesn’t mean they weren’t efficient at what they were doing (and they weren’t exactly bloated). More specifically, I was referring to bloat relative to features. Bloat doesn’t equate to malicious but then at the same time neither does small equate to harmless.

        • Hello,

          Oh, I definitely agree that size ≠ resource consumption. Back in the DOS era, I found out how to deadlock a system in 9 bytes. But, for a single-user OS running in real mode… that kind of rates a meh on the achievement scale. A fork bomb on UNIX would be more sophisticated.

          I do not think that retroviruses were appreciated, either. They often tended to introduce further problems into the system.

          As for Superfish (which is, after all the topic of this discussion), I must say that I have no particular insight into the actual decision process leading up to its selection by Lenovo for installation onto their consumer laptops. As for it being a trojan horse, I am not sure I would be willing to go that far. That implies some malign activity on the part of the creator, whereas I think the Superfish adware problem had more to do people working with technology to which they did not properly understand the risks from and subsequent consequences of.

          Again, to their credit, Lenovo started turning things around very, very quickly once that level of understanding was reached, but the reason it happened in the first place was because the understanding wasn’t there to start with.

          From my little corner of the computer world, I can imagine it involved sales, marketing or bizdev people from one company speaking with their counterparts at the other one, and they were probably less focused on discussing esoteric (to them) technologies like SSL interception than they were for what they’re paid to do, which is close the deal.

          If you have spent any time in a large- (or even medium-) sized corporation, than you probably have worked with those kinds of people before. And, to be honest, they’re needed. Engineers are usually pretty lousy at doing things like designing boxes and selling product.

          So, where does that leave us? Well, for right now, we’re kind of in a holding pattern. The real question is going to be when Windows 10 is shipping and we ask that question, and not just to Lenovo but all the other computer manufacturers out there.

          Regards,

          Aryeh Goretsky

          • Coyote

            Yes, a fork bomb is exactly what I refer to (for both the command line and the C program – C is because it is my favourite language, and is essentially the backbone to Unix, but others would work too). Yet it isn’t really sophisticated. As for deadlock a DOS system, here’s another one that would probably trip up a lot of people, especially if before the last part, you were to emulate what seems to be it booting up properly (this just out of my imagination right now, I might add): overwrite the boot code to issue a HLT instruction. Or if you wanted, to reboot the system or… you get the idea. Of course, you could do this with batch scripts (for DOS) which, while it wouldn’t confuse those with enough knowledge, it would make many wonder why the system boots up completely but at the very moment it finishes boot up, it reboots. If you’re okay describing it, what was the 9 bytes you thought of ?

            Another similar one to retroviruses and one might argue this is the same: I think it was the ramen noodle worm. Anyway, it patched your system for various problems and then it made it very obvious that you were breached (website defacement with name of worm?). Maybe I’m mixing examples up, though.

            Yes, you’re right: a trojan horse is typically malignant. But for what I was getting at, it works: it (Superfish) isn’t what it seems to be, or it does do what it claims but also releases (or does) something bad. Yes, I would also agree they probably did not understand the implications (but then when they were alerted last year, they should rather take it seriously and not wait it out). To that end, it is not a trojan horse. But in any case, it has rather bad implications. What would be nice, however, is if instead of having a removal tool (I assume you have to download it), they would issue an update that would remove itself (maybe it does this but if not it should, or if not maybe it could disable the part that creates the risks). But same as you, I don’t have insight in to how they did it, how their fix (I think they did write one ?) works and so I can’t really judge too much what they should or shouldn’t do; I can only say what would be better than simply having to download the fix oneself (problem: will every customer know? Obviously not).

  • katybugsdad

    The Yoga 2 Pro I recently bought (after 02/19/2015) has a built date of 01/27/2015 and still had the Superfish installed. Superfish does reappear with a full recovery (along with the full factory load of promo software & links).

    For recently sold systems that still contain Superfish in the recovery partition, does Lenovo plan to offer a clean(er) computer image to users to get this out of their recovery partition?

    Also, you stated that going forward “Lenovo will share information with customers about all preloaded software, explaining clearly explains what each one does.” Are there plans for Lenovo to provide this type of information retroactively for recently sold systems? I personally would like/appreciate information that would make it easier to decide what preloaded software is add-on and what is base/necessary.

    • Hello,

      I have not heard anything about a fix for the recovery partition or updated factory recovery media.

      Likewise, I have not heard anything about publishing lists of preinstalled softwarefor existing systems. Given the number of systems Lenovo ships and the variety of configurations they come in (each model has multiple configurations in each country), my feeling is this is something we probably won’t see.

      Regards,

      Aryeh Goretsky

Follow us

Copyright © 2017 ESET, All Rights Reserved.