Network based infection detection

A few days back, I attended a presentation of Prof. Dr. Nasir Memon of the Polytechnic University in Brooklyn, New York. The talk was about a new system that the university developed for detecting computers infected with malware.

Malicious software is normally detected and prevented using virus scanners, IDS systems, firewalls, etc. The problem with this is, that most systems use so-called signatures. Since malicious software nowadays gets stealthier every day, signature systems miss about 80% of the attacks. Professor Memon's team tried a different approach.

The approach is to focus on what happens on the network after systems get infected. Instead of checking the possible infected hosts (which can give incorrect information anyway if a rootkit is installed), the network traffic is analysed. Therefore, at several detection points in the network, so-called SynApps are placed. These applications receive all data from the network (for instance using a span-port of a core switch).

Because very much data is sent in a typical network, the data itself is not stored. Instead, the data (payload) in the network packets is hashed, and the hash is stored. Also some additional information of the network packets is stored, like the source and destination, the time, and the type of data (several techniques are used to find out if the data is for instance an MP3 stream of a video broadcast).

This way a compression of information of 1:50 is reached. This makes it possible to keep the data for a long time (weeks). Of course, the original data can not be restored from that, but it is possible to check if data is sent more than once, or sent around by a relaying system. This is done by checking if the hashed data is already in the database. Storing the data is done using Bloom filters, which makes it very efficient to query the data and to find patterns.

Patterns are for instance:

• Systems that are slowed down.
• Systems that reboot frequently.
• Relays that send incoming data to other systems.
• Systems that send data to IP addresses, that were never answered by the DNS system.
• etc.

These patterns can be combined in such a way that a top 10 of suspicious systems can be presented by the system. 

Data is kept for some weeks. This makes it possible to identify if a newly discovered exploit already infected some systems in the last weeks, and fix those systems.

The type of malicious software that can be detected this way are:

• botnets;
• spyware;
• trojans;
• rootkits;
• viruses, worms.

Professor Memon is searching for companies that are willing to install the system. His university can learn from it, and companies will gain a good malware detection system. Maybe something to consider for your company?

Non-functional requirements

Every system has requirements, that the system must comply to. Creating a list of these requirements for a new system is not a simple task. There are special applications on the market for documenting an correlating requirements for development- and test purposes.

Most requirements are functional requirements. They state what the system should do, like "After inserting a new customer order, the system must print a order confirmation".

Besides functional requirements, there are non-functional requirements. These requirements are sometimes called the "-abilities". Some examples of non-functional requirements are: 

• Availability
• Scalability
• Reliability
• Stability

Besides these -abilities the following are also non-functional requirements:

• Cost/ licensing
• Security
• Uptime
• Robustness
• Documentation
• etc

Users of systems usually don not state these requirements explicitly, but they do have expectations about them.

It is the task of the IT architect or requirements engineer to find these implicit requirements. This can be very hard. Things that are obvious to the customers or end-users, are not always obvious to others. Not to forget the non-functional requirements that system administrators have, like the existence of backup windows.

A large part of the budget of building the system can be determined by non-functional requirements ("The system obviously must work seamlessly with the existing systems" or "The website should always be available"). Therefore it is very important to quantify these requirements to make them explicit: How bad would it be if the website was not available for 5 minutes per day?" What if it will take $500.000 to satisfy this requirement? Is it still important then?

It is important to remember that the acceptance of a system is largely dependent of the implemented non-functional requirements. A website can be very beautiful and functional, but if loading the site (a non-functional requirement) takes 30 seconds, your customers are gone!

LEAP - Microsoft Lead Enterprise Architect Program

This year I will attend the LEAP 2008: Microsoft Lead Enterprise Architect Program.

The purpose of LEAP is to extend the knowledge and insight of Microsoft's software portfolio for the business market of experienced and aspiring IT architects. Not only the most important Microsoft technologies are discussed, but also the relationship between the technologies and the business issues that they can address. Apart from technology available today, Microsoft's vision, mission, strategy and roadmap are part of the program.

LEAP is not an architecture program, but a program for IT architects. It is meant to extend the knowledge of architects with relevant knowledge of Microsoft technologies.

The LEAP-program consists of five masterclasses. Each masterclass zooms in into a business theme, and the technologies available to implement the theme.

The program is ended with a week's visit to the Microsoft campus in Redmond, Washington. There, the future roadmap and the architectural challenges are discussed with- and presented by Microsoft architects.

I am not a Microsoft expert. My roots are mainly in the UNIX world. Therefore, this seems like a good opportunity to gain knowledge on Microsoft technologies. These technologies are present at virtually all clients I work for.

The program starts in September with a one day workshop every month.

The Redmond visit will take place in January 2008.

Through this site I will publish my experiences in the program. Here is a link of someone who attended the LEAP previously.

I am looking forward to it!

UPDATE: Here is a post on how it went...