An in-depth look at the Emsisoft scanner technology

An in-depth look at the Emsisoft scanner technology

140527_eek_icon_256x256Our team of developers here at Emsisoft works hard to deliver major improvements in the realm of scan speed and optimization. You may know how to use our scanners thanks to our video tutorials, but do you know how our scanner technology works? Let’s lift up the hood to learn about the underlying technology that drives all Emsisoft’s products. 

  1. Two scan engines are better than one
  2. Available scan methods
  3. Advanced scanner features
  4. Productivity features
  5. Cleaning infections

1. Two scan engines are better than one

Part of what gives our scanner its advanced detection power is its dual engine. We are committed to providing the best possible scanner technology, that’s why we built our software to be flexible enough to swap out third party engines as required.

You may remember that we switched our engine from Ikarus to Bitdefender in 2012. A powerful match between our technological developments and a keen eye on the future of cutting edge antivirus developments keeps us ahead of the curve.

Advanced signature-based detection

The engine we have built complements the second Bitdefender engine, and are combined seamlessly to maximize efficiency.

One of the ways we detect unwanted programs is through signature-based detection. What this means is that we search programs for their unique signatures, which are like fingerprints, and scan your computer for these threats.

Here at Emsisoft, most of our lab time is spent creating detection signatures for PUPs (potentially unwanted programs) and on custom malware removal code for specific infections. We ran some numbers earlier this year and discovered that more than 74% of the total detected PUPs are detected by our in-house built scan engine component.

Maximizing performance with a dual engine scanner

Having two engines means we are better equipped to provide new signatures for threats as quickly as possible — so quickly, that often times both vendors have signatures made for the same threat within the hour!

chart_engines_en

Two engines are more efficient than one

If you’re concerned about increased memory usage, never fear. We have these duplicate signatures cleaned out at regular intervals to keep memory usage low. 90% of the signatures created by Emsisoft’s engine are duplicates and are not used in malware detection.

And you don’t have to worry about time lost either: the files on your hard disk are only read once and then scanned by both engines. This ensures that there is no significant scan time loss, even though we use two engines. It’s no coincidence that our dual engine scanner works faster than many big brands with just one engine available!

So how does information translate to your own practical use?

Simple: all detections with an (A) postfix are from our own engine and those with (B) are from Bitdefender.

In a nutshell: We believe two engines are better than one, and we use our own technology to detect threats to your computer that might otherwise be missed. But we won’t compromise efficiency in this process — Emsisoft works to keep your memory clean and uncluttered, and to detect threats at optimal speeds. To see some numbers about the real power of the second scan engine in our products, please see this article from earlier this year.

2. Available scan methods

Here is a quick rundown of the available scan methods on Emsisoft Emergency Kit, as well as Emsisoft Anti-malware and Emsisoft Internet Security:

scanner_en

Three different scan types

Quick Scan

Quick scan quickly gives you an overview of any active infections on your computer. It does so by scanning through all running programs and their modules. Quick scan also completes something called a “trace scan.” Traces are known file and registry- paths of malware infection. In simpler terms, it’s an antivirus scan that looks for a trace left behind by malware in order to locate it.

Additionally, quick scan checks installed drivers for active rootkits. A rootkit is a type of malicious software that hides certain files or registry keys from normal methods of detection so that it continues to have access to your computer. We’ll discuss rootkits and how they work further when we review the custom scan features.

We recommend a quick scan for automated/scheduled scans after boot or on user logons. It generally takes about thirty seconds to complete, so you don’t have to worry about it interrupting your day!

Malware Scan

The malware scan is similar to the quick scan, but it scans files in all folders that are known to host active malware infections. Our scanner identifies about a hundred common areas where malware likes to strike. One advantage here is that malware is really predictable in where it chooses to install — but don’t think the Emsisoft team is complacent! Our analysis team is constantly moving forward to detect new common areas, and they’re able to update software within minutes to keep your Emsisoft applications up-to-date.

We recommend a malware scan as the default scan when you suspect an active infection on your system. The malware scan does not detect inactive malware files, but luckily inactive files are non-active threats. These files can simply be deleted with a stroke of a key, like the Word document for a meeting long past or an unflattering vacation photo.

Custom Scan

Since the default mode of custom scan is set to perform a complete, full scan, use this option if you want to do a very thorough malware search and scan all files on all drives of your computer. The custom scan takes a significant amount of time to complete, and it isn’t recommended for frequent or daily use. It’s the kind of scan you should run a few times a year to be absolutely sure nothing is hiding around on your computer.

3. Advanced scanner features

One of the great features of a custom scan is that you can control your scanning settings. If you look at the custom scan settings dialog you’ll see all of your options. Some of them are enabled by default and others aren’t. Knowledge is the key to knowing what you need and what you don’t, but we have the best options selected for the everyday user. We’ll detail the options below so that you can familiarize yourself with what may or may not be appropriate for your scanning needs.

customscan_en

Advanced scanner features

Rootkit scan

A normal file scan uses Windows APIs (Application Program Interface) to read files. Think of an API as the foundation for building software applications, that is built of routines and protocols.

Unfortunately, though using a Windows APIs may be optimal for speed and performance in certain regards, these APIs may be manipulated by rootkits.

What are rootkits?

Rootkits are like soldiers in camouflage. They blend into systems through a number of different means, and a very common way to do this is by modifying lists and tables that tell a system where to find code (this is called “hooking”).

When antivirus software accesses this list of available files, the rootkit has manipulated the list to skip a file — a malware file. Once a file is made invisible like this, it’s difficult for your malware scanner to find.

To find hidden rootkits, our scanner uses its own NTFS file system parser code when looking for rootkits. This code doesn’t rely on the common Windows APIs, which gives us an advantage over stealthy rootkits.

If the rootkit has camouflage, the Emsisoft scanner has super vision!

Cleaning rootkits

Cleaning rootkits properly is very tricky. Sometimes rootkits can even hide within certain regions of your computer’s hard disk, like the boot sector. Simply deleting these malicious files often results in an unbootable computer.

Our specialists help many victims of careless cleaning attempts by other anti-malware products, so we know firsthand how important it is to use a trustworthy source.

Rootkits generally require manual cleaning. Our scanner will tell you to consult our malware removal experts to clean rootkits. They will analyze and identify the type of rootkit affecting your system, and provide you with detailed, step by step instructions to remove it without risking the stability of your computer system.

You might wonder why all of the scans don’t rely on our own scanner: this is because reading directly from the file system (direct disk access) is generally very challenging and typically much slower than using windows APIs. If it were different, rest assured, we would use our own NTFS file system parser code for all scans.

Traces Scan

A trace scan (a scan that looks for traces that malware left behind) may be one of three types:

-File traces: These are known paths of executable files on the hard disk that are used exclusively by malware. These are essentially traces that exist alone in the hard drive, independent of any other program’s folders.

Example: C:\windows\explore.exe (may be mixed with exploreR.exe).

-Folder traces: These are similar to file traces, but exist inside the folder of other common applications, like a Google Chrome setting folder.

Example: c:\program files (x86)\PUP Folder\.

-Registry traces: These are entries in the system registry database that indicate a malware infection. A registry trace points to an infection inside the actual settings of the computer. These are the most dangerous traces, and the related virus may significantly slow down the speed of your computer.

Example: HKLM\Software\Windows\CurrentVersion\Run.

It’s important to note that if a malware trace is detected, it doesn’t necessarily mean that there is an active infection. It may well be leftovers from a previous, incomplete cleaning attempt. Trace infections tell you to be aware and to investigate.

Generally when there is an active infection, traces are typically found next to file findings. You can clean them at any time.

PUPs detection

For legal reasons, we can’t call all unwanted programs “malware” in our user interfaces. The term PUP was invented by the antivirus industry several years back, which stands for Potentially Unwanted Program. Generally, PUPs exist to get their creators some extra cash by displaying ads, changing your default search engine provider, or by collecting private data to sell to advertisers.

Emsisoft Anti-Malware - Potentially Unwanted Programs (PUPs)

Potentially Unwanted Programs (PUPs) detection

PUP detection must be enabled on first installation of our software. In Emsisoft Anti-Malware and Emsisoft Internet Security it can be enabled in the File Guard settings dialog afterwards.

Scan in compressed archives

Compressed archives are files that contain a number of other files and shrink their size. Some common examples are ZIP, RAR, or 7Z, but there are hundreds of other less known compressed archives. Even a program like EXE may actually be a self-extracting archive, meaning it contains other files (generally this is for more efficient data transferring).

A malware file that is wrapped inside an archive file can’t directly start from within a compressed archive, as it needs to be unpacked first. Because of this, archives aren’t typically considered dangerous on their own. As a result, many scanners exclude archives from scanning or limit archive scanning to file sizes of about 200 MB.

Unpacking archives is incredibly time consuming and takes up a lot of system resources. You may disable the archive scan feature if you already understand what’s happening within your own archives and know there isn’t a possibility of infection.

Scan in NTFS alternate data streams

In 1993, with the introduction of NTFS (New Technology File System) as the default file system of Windows NT (predecessor of 2000, XP, 7, 8, etc.), a new feature called Alternate Data Streams was introduced. Files were now able to store meta data in hidden layers.

Unfortunately, these streams can also be used to store other types of harmful data, like complete malware programs — and all within a 0 byte text file.

Fast forward to today, and a harmless looking file extension may contain dangerous code which can be started automatically via autorun registry keys.

When the NTFS Alternate Data Streams scan option is enabled, the scanner searches all data layers for hidden threats.

Using the file extension filter

With the file extension filter you can limit the number of scanned files based on their file type. Many file types cannot be used to host dangerous code, so many people might initially think it’s a waste of time to scan certain files.

For example, all executable Windows files start with the byte sequence “MZ” which tells the operating system that the file can be run by the computer. Checking these byte sequences (or “magic bytes”) is a reliable method, and almost as fast as simply checking the file extension itself.

But it’s important to note: there’s an important reason that this feature is actually disabled in the default settings. This is because the scanner doesn’t just look at the type of file extension by name, but looks for specific file type markers inside of the file. File extensions can be easily changed to fool a scanner, but the content can’t.

Direct disk access mode

As mentioned above, the scanner is able to search files that are hidden by active rootkits by utilizing our own NTFS file system parser instead of Windows APIs. The direct disk access mode allows the Emsisoft scanner to bypass security checks and to go directly to a file location to find protected malware.

The downside of this method is how immensely time consuming it can be. Therefore it should only be used for specific folders that may contain rootkits. There is not much to gain by using this feature to scan your whole disk, that’s why this option is disabled by default. Rootkit scan always uses the direct disk access mode feature, so rest assured that it’s automatically set to be utilized when necessary.

Performance settings

When viewing the scan area of the software, you’ll see a small “Performance settings” option below the 3 main scan methods. If you click on it, it opens a little popup with advanced features for tuning the scanner speed:

performancesettings_en (1)

Performance settings

– Processors

By default, all available processors of your CPU are used for scanning. Note that quad-core CPUs are usually displayed as 8 virtual processors. You may want to disable one or two of them if you are planning a long duration scan and need to run some heavy resource consuming program at the same time.

– Number of threads

Threads can be visualized like execution tasks that run in parallel. Imagine threads like roads that information takes to get to the core.

If the scanner were to be single-threaded (one thread limit) a file would be read from disk, then scanned, and then the next file would be read and scanned, and so on. Using multi-threading technology, each virtual processor can scan a file at the same time without interrupting the others.

By default, the number of threads is the number of available processors + 1. The reason for that is that one thread with low CPU requirements is used for reading the data from the hard disk (as parallel reading typically isn’t an option), and then the files are distributed across all processors for simultaneous scanning. This is the heaviest part for a CPU.

– Scan thread priority

By default, Windows defines which programs get which percentage of the overall available hardware resources (CPU time). But you may define a higher or lower priority for the Emsisoft scanner. Use a higher than standard value to make sure that scans are finished in the shortest time (even if other programs are running). Use a lower than standard value if your work relies on other programs that require higher priority. This is best if you don’t care how long the scan takes, as long as it doesn’t interfere with your work.

– Use advanced caching

Caching means that files that are proven to be safe are not scanned over and over again. For example, if a file has been on your computer for a very long time and has already scanned many times without any findings, it is very unlikely to be malicious. A smart logic estimates how likely it is that a file is safe and then skips it for further scanning.

4. Productivity features

Context menu scan in Explorer (Not available in Emsisoft Emergency Kit)

The web is teeming with trojans and spamware, just waiting to get inside of your system. But a context menu scan can act as a great preventative method to contracting viruses in the first place.

Emsisoft Anti-Malware and Emsisoft Internet Security come with a useful Windows Explorer integration that can save you a lot of time if you’re performing frequent scans. Just right-click on any file or folder in Explorer and select the option “Scan with Emsisoft” in the context menu to start your custom scan.

Emsisoft Commandline Scanner

Emsisoft Commandline Scanner

Commandline Scanner

A commandline scanner is best for professionals who don’t need a graphical user interface to perform their scans. If you’re unsure of what this means, don’t worry! This isn’t a program that you’ll need.

The Emsisoft Commandline Scanner is a complete commandline interface that includes all features of the Windows-based scanner. It’s primarily used for automated scans, initiated by other programs or scripts which require a return value for further processing. Learn more about the available parameters of the command line scanner here.

5. Cleaning infections

Detecting an active threat is just one part of the journey to a clean computer. Cleaning is actually a more difficult process than finding PUPs, because malware works hard to avoid extraction. Here are a couple of cleaning prevention mechanisms that malware uses to lodge itself in your computer:

– Lock the file

Some malware is able to lock a file. If a file is locked, it can’t be deleted. Locks can be achieved by ensuring a program is always running.

– Watchguards

This is an infection method in which malware comes in a pair of two programs. If you kill one program, the other will notice and re-start immediately. If you kill the second, the first one restarts, and so on.

– Hiding

As mentioned above, Rootkits manipulate system APIs to remain hidden. If a file can’t be seen it can’t be removed, now can it?

Autorun as system component

Some threats load themselves into programs that your operating system autoruns (automatically runs) when you start your computer. If you try to kill them, you’ll get the dreaded blue screen, and everything will stall. If you remove the autorun entry, the malware recovers instantly.

How Emsisoft cleans infections

eek_lifebTo cope with these malware tricks, we have developed our own sophisticated cleaning engine. It cleans about 100 locations in the registry and file system that can be abused to automatically load malware on system startup.

If a file is locked, our cleaning engine schedules the removal of the malware in question for the next system boot up through a method that disables malware from blocking removal once again. Additionally, our engine restores default values of a number of autorun locations that would render the system unusable when you just delete the malware entries. During removal, a quarantine copy of each threat is saved for later analysis or restoration (unless you select the “delete” option instead of “quarantine”).

So what does it mean when a file is in “quarantine”? It means that a file is wrapped in an encrypted, secure container file where it cannot do harm to other files and applications on your system. We always recommend using the quarantine feature, because there is a small chance that the file that was detected is harmless (a false positive), or that the file might be necessary for further investigation or forensics. You may delete quarantine files after a couple of weeks if it turns out that the file is in fact harmless.

Scanning and cleaning files on network shares

While it is possible to scan files on network shares that are located on other machines, we don’t recommend that at all. It might save you a bit of time to walk over to it, or remote connect to a target machine, but please be aware that scanning remote files has some serious limitations by design:

  • Memory, rootkit and trace scans are not possible, as they require operating system APIs that can only be accessed locally. You’re limited to scanning files via standard file reading procedures, which means no direct disk access mode is available either.
  • Cleaning is not possible at all, because removing a detected active malware file without removing its accompanying autorun entries would most likely crash the computer and leave it in an unbootable state.

Always scan and clean locally. If you want to avoid installation of our software for doing that, go for the Emsisoft Emerngency Kit scanner which is fully portable and doesn’t require any installation.

Whether you’re an antivirus expert or a casual internet browser, we hope this information will help you understand exactly how Emsisoft’s top of the line technology is working to protect your computer from malware.

Have a great, malware-free day!

  • John D Lord

    Many thanks Christian and Team for taking the time to send the information, now I can make some copies to give to clients, family and friends.
    Kind Regards from Australia.
    JD

  • Chiron

    Thanks for the great info! However I’m confident that you kept a couple of aces up your sleeve without telling: were I a malware manufacturer, I’d consider the above as very precious info too… ;-)

  • TheSeeker11

    Most interesting. Thanks!

  • Luis Perez Salto

    Impressive!….I licencensed your product many years ago and since then i have never had a malware problem. Congratulations for your service to normal people, who would be very easy victitims of cibercriminals if was not for honest defenders like you!

    • christopher pope

      yes , i have to say that i installed your program on my pc and its brilliant, top notch and glad i did , thanks for the information and for your brilliant program

  • Moreh Richard Fulk

    I am very grateful to one of your sales agents “Thomas Ott,” to whom has been helping me with my dilemma here living in the Philippines these past 2 1/2 years, on the purchase page of cleverbridge the purchase section has no offering for those American’s such as myself living here in the Philippines, here we must use only Peso. Thomas Ott, has helped me with many aspects of finding a resolution to my present situation, I in the past had been a faithful affiliate to the Emisoft company. This man has done me so right in making it possible to continue the protection to which only Emisoft’s customer service has always done for me, giving me the extreme best customer service I have ever had. Soon I will find a way, I pray here in the Philippines to get a dollar debit card so I may purchase the two licenses I need for my wife and I to remain protected permanently by Emisoft! Folks when it comes to protection of our PC, laptop, or cell, the only company on the Internet that I recommend to my family and friends is Emisoft Internet Security, as they are the very BEST!

    • Mariska

      Thank you for your kind words! We will pass them on :)

  • Philip

    Hood = part of a piece of clothing that can be pulled up to cover the top of the head. Did you know there is no official language in the U.S. but because of feature films we understand what they mean to say. I had a real problem with “a stroller”. someone who strolls in English right! But in the U.S. it means ) a small folding chair on wheels that a baby or small child sits in and is pushed around in. But I understand hood means lid and not hood. You know there are loads of them don’t you. “An American pitbull Staffordshire” Staffordshire is in England the county of Staffordshire where the dog was created through selective breeding. “An American bulldog” it’s in English Bulldog and they cross it with a St Bernard and pretend it is an old English Bulldog but they call it an American bulldog. America is a continent and not a country: the Asian continent the African continent the European continent and the American continent. There are so many words they get totally wrong! They get the meaning totally wrong!
    But I shouldn’t get too cocky should I or else this might happen:

    It was a good article anyway probably one of the best you have done.