Traffic Scanner - Hardware Accelerated IDS

Date: March 2008
Version: 1.1.2
Author: The Liberouter Project Team
Organization: CESNET, z.s.p.o.

Contents

1   Introduction

IDS (Network Intrusion Detection System) 1 Gbps Traffic Scanner is an integrated software/hardware tool capable of detecting unauthorized access to computer systems or networks and malicious network traffic such as viruses, Trojan horses and worms. Comparing with a common IDS tools like Snort, the main benefit of using Traffic Scanner is very high throughput (up to 3.2 Gbps). Another benefit is re-usability of your current IDS infrastructure (Snort, Log storage facility, Security analysis tools). Traffic Scanner will only reduce network traffic and export approximately one percent (malicious one) of all network traffic. This reduced network traffic can be processed by Snort without throughput limitation. Thanks a original Liberouter Linux driver Traffic Scanner behave like a ordinary NIC (Network Interface Card) so using it is absolutely clear.

1.1   Obtaining IDS package

IDS package is available via WWW download page:

You can also check latest news at our web pages:

For building custom IDS firmware please visit this WWW page:

1.2   Package structure

The IDS package contains firmware and software to run IDS on COMBO cards. The package also contains IDS installation system, README and RELNOTES.

/base       - source codes for software tools and drivers
   /mk                - build system
   /sys_sw
      /drivers/linux     - Linux kernel drivers
      /hwtools           - necessary hardware tools for COMBO6 card
      /lib*              - libraries necessary for other tools
      /projects/ids
         /config            - default configuration file
         /doc               - ids.rc file
         /src               - scripts and idsctl tool for using IDS
      /swtools/csxtool   - tool to handle COMBO6 XML files
/firmware   - card firmware (mcs files, cam_init.txt files)
ERRATA      - how to handle known problems
README      - this file
RELNOTES    - differences against previous release

2   Installation instructions

2.1   Supported hardware

Mother card Add-on card Description
COMBO6X   a PCI-X (64/66) card with XILINX FPGAs
COMBO-4SFPRO/1G add-on card with four SFP cages for GE interfaces
NOTE:more information about Combo cards you can get on http://www.liberouter.org/hardware.php

2.2   Supported PCI bridges

Traffic Scanner works with PCI bridges of firmware versions c610.04.X where X >= 4. To check version of PCI bridge run command from ids package:

$ csid -s
Board    : combo6x
Addon    : sfpro
Chip     : xc2vp20
LAN ports: 4
Firmware : ok
SW       : 0x1d510001
HW       : 0x00000001
Text     : IDS_1Gbps_Traffic_Scanner
PCI brver: c610.04.04 (2007/03/29 19:01)

Line beginning with PCI brver shows PCI bridge firmware version. For more information about PCI bridge firmware versions and upgrade or downgrade of PCI bridge firmware please see page http://www.liberouter.org/card_combo6x.php.

2.3   Installing COMBO6 cards

After plugging Combo card into your PCI slot, you should test connection between the card and your PC. We use lspci(8) utility for this purpose. lspci(8) is a utility for displaying information about all PCI buses in the system and all devices connected to them. For correct recognition of the Combo card you need update PCI ID Database used by lspci(8) or download pciutils-2.2.2 (program collection containing lspci(8)) or later. If the lspci(8) output contains at least one of the following lines (according to the type of installed Combo card), your Combo card is connected properly.

$ lspci -d 18ec:
03:01.0 Ethernet controller: Cesnet, z.s.p.o. COMBO6X (rev 01)
04:02.0 Ethernet controller: Cesnet, z.s.p.o. COMBO6 (rev 01)
NOTE:Numbers at the beginning of the line can be different.

2.4   Building package

Compilation and installation process is covered by the build system located in the /base/mk directory. For more details about required tools necessary to build package see base/mk/REQUIREMENTS file.

To build kernel drivers needed by IDS package you will need fully configured Linux kernel source tree. More installation and compilation details about kernel drivers are available in the README file inside base/sys_sw/drivers/linux directory.

pkgtool(1) helps to build, install and uninstall a distribution package.

$ tar -xzvf ids-XX.YY.ZZ.tgz
$ cd ids-XX.YY.ZZ/base
$ ./pkgtool --build
NOTE:Replace XX.YY.ZZ with the chosen package version number such as 1.0.0

Optionally you can define installation directory with the --prefix=path option. The --prefix option takes effect only during building package. If no prefix is set then /usr/local/liberouter path is used.

$ ./pkgtool --build --prefix=/usr/local/ids

2.5   Installing package

The tools and kernel drivers will be installed to the installation directory (by default /usr/local/liberouter).

# ./pkgtool --install

If you are using recent 2.6 kernels it might be wanted to use the sysfs/udev mechanisms to create device files. In this case please add the option --udev.

# ./pkgtool --install --udev

After successful installation some post-installation steps are required:

  1. Set PATH variable to point to the installed tools directory -> ${PREFIX}/bin

    $ export PATH=$PATH:/usr/local/liberouter/bin
    
  2. Add new enviroment variable CS_XML_INDEX_PATH to file /etc/profile and set it to point to design index directory -> ${PREFIX}/mcs/index

    $ export CS_XML_INDEX_PATH=/usr/local/liberouter/mcs/index
    
  3. Set /etc/ld.so.conf to point to the library directory -> ${PREFIX}/lib

    create file liberouter.conf with content /usr/local/liberouter/lib (by default or with ${PREFIX}/lib if you have used option --prefix during building process) in /etc/ld.so.conf.d directory:

    # echo "/usr/local/liberouter/lib" > /etc/ld.so.conf.d/liberouter.conf
    # ldconfig
    
  4. Set /etc/man.config file to point to the Traffic Scanner's man pages -> ${PREFIX}/man

Add MANPATH /usr/local/liberouter/man entry to the /etc/man.config file.
  1. The driver needs device files in the /dev/combosix/ directory. To create them, run the combo6devices script in the /base/sys_sw/drivers/linux directory:

    # ./combo6devices
    

    To run IDS as a non-privileged user set r/w permissions on /dev/combosix/0 device and ids.conf configuration file.

    You have to change the permissions of the device and configuration file so that the non-privileged user that will execute the IDS tools has read-write access. For example, you can create a new group, say combo-rw, add that user to this group and change the file group ownership and permissions as follows:

    # chgrp combo-rw /dev/combosix/0
    # chmod 664 /dev/combosix/0
    # chgrp combo-rw /etc/liberouter/ids.conf
    # chmod 664 /etc/liberouter/ids.conf
    

    Using recent 2.6 kernels it might be wanted to use the sysfs/udev mechanisms to create device files. For more details about udev see http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev.html

    Example udev rules are in combo6.udev.rules file located in ids package ids-1.1.2/base/sys_sw/drivers/linux. If you didn't use option --udev during package installation, copy this file to /etc/udev/rules.d directory and modify it (change group or permissions). Following line will create COMBO6X device files required by the Traffic Scanner probe:

    ``KERNEL=="combosix[0-9]*", NAME="combosix/%n", GROUP="combo-rw", MODE="0664"``
    
    NOTE:

    Depending on your version of udevd you must adjust syntax of your udev rule file. Older versions of udevd require '=' character to match KERNEL key e.g. KERNEL="combosix[0-9]*". The newer one require '==' characters to match KERNEL key e.g. KERNEL=="combosix[0-9]*".

  2. If you want to load IDS's kernel modules when the PC starts up, add following lines to the /etc/modules file.

    For COMBO6X card:

    combo6core
    combo6x
    combo6x_c6pcreth
    

    After this, you don't need to run idslkm script any more.

    NOTE:

    In some distribution (e.g. Red Hat Enterprise Linux 4) you have to use file /etc/rc.modules in place of /etc/modules file. This file contains commands to load kernel modules at boot time (it is actually script so it must be executable file), so such file for COMBO6X card will look like this:

    #!/bin/sh
    /sbin/modprobe combo6core
    /sbin/modprobe combo6x
    /sbin/modprobe combo6x_c6pcreth
    

    As mentioned, make this file executable by command:

    # chmod +x rc.modules
    
  3. Please copy (and rewrite any previous version) of liberouterpkg script, located in the package base directory into your standard binary directory (e.g. /usr/local/bin/). This enables using liberouterpkg script as standard system tool.

3   liberouterpkg tool

liberouterpkg script is a new tool covering new Liberouter's package installation framework which enables safe and easy package uninstalling or simple package switching (and using different project packages on the same PC). To display all available functions of the liberouterpkg script use --help option.

# liberouterpkg --help

liberouterpkg uses /etc/liberouter/packages.list configuration file that stores information about installed packages.

3.1   List Of Installed Packages

liberouterpkg can be used to display list of all installed Liberouter's packages

# liberouterpkg --list
# Installed packages:
ids-1.1.2

or to display currently active package (only one package can be active at the same time)

# liberouterpkg
ids-1.1.2

3.2   Uninstalling Package

liberouterpkg tool is able to safely uninstall any previously installed Liberouter's package (covered by new Liberouter's package installation framework with liberouterpkg) by --uninstall=<PKG_ID> option where <PKG_ID> is a package identification string (that can be displayed by --list option)

# liberouterpkg --uninstall=ids-1.1.2

4   Configuration

IDS configuration is based on configuration file ids.conf located in the /etc/liberouter/ directory.

Changes made in these file takes effect after running the ids script (described in the next chapter). If you don't wish to reboot firmware, give the option -r to the script to restart the design without rebooting firmware.

$ ids -r

4.1   IDS configuration file

Configuration file for IDS is placed in the /etc/liberouter/ directory:

ids.conf general settings for IDS's scripts

This file can be freely modified to redefine default values. Default installation copy of file is stored in ${PREFIX}/etc/liberouter (by default /usr/local/liberouter/etc/liberouter) directory.

During installation of the next IDS package version you will be inquired if you prefer to keep your own configuration files (with your modifications) or to rewrite it with our up-to-date version. So you don't have to be worry to lose your changes. Up-to-date file will be copied to the ${PREFIX}/etc/liberouter directory.

4.1.1   ids.conf

This file has the form of the list of variables with default values used by IDS's scripts. These are main groups of variables defined in this file:

IDS configuration

combo6x card parameters

device file pointing to the combo6x card

COMBO6_DEVICE="/dev/combosix/0"     # combo6x device file
firmware parameters

variables used to locate *.mcs files, e.g.:

FIRMWARE_PATH="/usr/local/liberouter/mcs"                 # firmware installation path
DEF_FIRMWARE=$FIRMWARE_QUICK                   # default firmware (loaded if
                                               # "ids" command without
                                               # parameters is used)
interface parameters

variables for setting up combo network interfaces, e.g.:

C6ETH0_IP="10.0.0.2"      # IP address of the first combo card interface
ids-daemon parameters

ids-daemon variable:

START_DAEMON="YES"        # start ids daemon ?
rc script parameters

path to binaries:

BINDIR="/usr/local/liberouter/bin"   # path to liberouter binaries

5   Using IDS

This procedure serves to single start IDS design - the IDS will not be automatically started again after rebooting PC. If you wish to start IDS anytime you will reboot the PC, please see section IDS init.d script.

After building and installing package including post-install steps, all IDS tools are available as any other system tool. There are two main scripts to start IDS. Both scripts use /etc/liberouter/ids.conf configuration file.

5.1   Loading kernel modules

idslkm script is used for loading/removing IDS kernel modules. There are two main options for this script. Option -l is used for loading IDS kernel modules and option -r is used for removing these modules. Script supports combo6x card. Detection of the card is performed by the lspci(8) utility.

When all kernel modules are loaded, csid(1) is able to detect type of plugged cards and provide more detailed information.

You should get following output on PC with installed

  • combo6x mother card and sfpro add-on card:

    $ csid
    combo6x sfpro xc2vp20
    
NOTE:Chips can be different on some cards, so don't worry about differing last part of the csid's output.

5.2   Setting up IDS

ids is main start up script for IDS. Script boots firmware files (according to FIRMWARE_PATH and DEF_FIRMWARE variables from the /etc/liberouter/ids.conf), configures combo card interface and starts/stops ids-daemon. Switch -S should be specified to avoid freezing of probe. More information about using ids script can be displayed by -h option. Man documentation is also available.:

$ ids -h
$ man ids

Useful usage examples:

$ ids -p          # print all available designs according to FIRMWARE_PATH
                  # variable
$ ids -p               
Available designs in /usr/local/liberouter/mcs/ids/ are:
1: bleeding_rules
2: p2p_search
3: quicktest
4: virus_search

$ ids -n <number> # load design number <number> from list printed by ids -p
$ ids -n 4        # load ``virus_search`` design

$ ids -c          # print current (last loaded) firmware

$ ids -f <path> -S <path>     # load design located in <path> directory, after each restart post restart script located in <path> is executed
$ ids -f /home/foo/mcs/foo_search/ -S /home/foo/ids/ids_snort_restart

5.2.1   Sample post restart script to restore Snort

This is sample post restart script used with ids and ids-daemon to restart Snort if probe detected frozen. Path to script is specified by -S switch:

#!/bin/bash
/usr/local/snort/bin/snort -i c6eth00 -c /usr/local/liberouter/mcs/ids/01_00/virus_search/snort.conf -l /home/ids/log/ -D
NOTE:It is important to set right permissions to script - executable flag.

5.3   Logging probe information

ids-daemon logs important events to syslog using the user facility. To have these log messages saved to a file, you must configure the syslog daemon appropriately. In the case of the classical Unix syslogd daemon, add the following line to /etc/syslog.conf:

user.*                  /var/log/user.log

Regarding to your Linux distribution see syslogd(8) and logrotate(8) documentation how to adjust log file rotation.

5.4   Typical usage

5.4.1   Start probe

To start the probe use:

# idslkm -l
$ ids -S /path/to/post_restart_script         # or ids -n X
NOTE:Commands which begin with # must be run by root.

After the execution of those scripts IDS is started and in use. There are four new network interfaces c6eth00 - c6eth03 in the system. Numbers in the interface names are set according to number of card plugged in the PC (the first digit - 0 in our example) and according to number of interface (second digit). ids-daemon is running and checking if probe is working correctly. Post restart script is set to /path/to/post_restart_script.

You can check that IDS is set up correctly by setting desired IP address to IDS interfaces (via ids.conf or ifconfig) and then ping(8) it from PC connected to IDS port.

Firmware can be checked by csid(1):

$ csid -s
Board    : combo6x
Addon    : sfpro
Chip     : xc2vp20
LAN ports: 4
Firmware : ok
SW       : 0x1d510001
HW       : 0x00000001
Text     : IDS_1Gbps_Traffic_Scanner

Detailed firmware description can be found at http://www.liberouter.org/ids.php - firmware architecture. HW number detected by csid -s reflects firmware version, e.g. HW : 0x00000001.

5.4.2   Shutdown probe

To shutdown the probe use:

$ ids -s
# idslkm -r
NOTE:Commands which begin with # must be run by root.

Probe is correctly shut down, ids-daemon is terminated, network interfaces are down and kernel modules are removed.

5.5   idsctl

This tool is used to control IDS. It prints detailed statistics, initializes probe and enables/disables input buffers and export options.

NOTE:For experienced users only. It is not necessary to use this tool to run IDS.

More information about using idsctl can be displayed by -h or --help options. Man documentation is also available.:

$ idsctl -h
$ idsctl --help
$ man idsctl

Usage examples:

$ idsctl -c status_all           # print statistics of all units

$ idsctl -c init -f cam_init.txt # initialize IDS and load cam_init.txt file
                                 # into CAM (content addressable memory)
$ idsctl -c ibuf_dis -u 2        # disable input buffer unit 2

$ idsctl -c export_all_en        # make IDS to pass all incoming packets up to
                                 # c6ethXX interface

$ idsctl -c export_nm_en         # make IDS to pass all not matched packets up to
                                 # c6ethXX interface

5.6   ids-daemon

Due to unknown bug occurring seldom in some designs, ids-daemon has been created. This bug causes probe to freeze.

ids-daemon periodically checks state of probe according to registers of main components. If probe is detected frozen, daemon correctly restarts probe within seconds and post restart script is executed if specified. This script is intended to restore run of applications using c6eth00 interface as Snort.

Events are logged by syslog into file /var/log/user.log.

Starting and terminating of ids-daemon is done transparently by script ids. When used without -D switch ids-daemon is started after booting firmware. Use with option -s will cause correct termination of ids-daemon.

See ids-daemon manual page for more information.

5.7   IDS init.d script

Startup scripts are used to start some service (or some script) at the machine's boot time. Our sample startup script is used to automatically start up the IDS (loads kernel modules, boots firmware, sets up network interfaces and starts ids-daemon) anytime your PC is starting up. Sample script is stored in the package directory structure in the base/sys_sw/projects/ids/doc directory as a ids.rc file. It is prepared for use in the SysV init system (tested on the Red Hat Linux distribution). Script is commented so you can get a lot of information directly from the script.

Startup script uses ids.conf file to set up network interfaces.

5.7.1   Running startup script at the boot time

  1. copy sample script to the /etc/rc.d/init.d/ or /etc/init.d/ directory and rename it to the ids

  2. run chkconfig(8) or update-rc.d(8) and add ids as new service

    # chkconfig --add ids
    or
    # update-rc.d ids defaults
    

    to verify, everything is right, you can try to find ids in the list of services

    $ chkconfig --list | grep ids
    or
    # update-rc.d ids defaults
    

    You can also use bum gui utility.

NOTE:Commands and directories are dependent on your Linux distribution.
  1. reboot your system

6   Quick test

This section describe how to verify Traffic Scanner functionality using a standard OS tools like ping, dnet, tcpdump etc. For a newbie users is recommended to try this simple test.

6.1   Network placement

Connect one of the four Traffic Scanner port to standard NIC card which will be used as a traffic generator. Traffic Scanner will be configured for searching packets which match one of these Snort rules.

alert icmp any any <> any any ( content:"George"; )
alert icmp any any <> any any ( content:"Thomas"; )
alert udp any any <> any any ( )
alert tcp any any <> 10.0.0.0/24 666 ( content:"Michael"; )
alert tcp any any <> any 1024 ( content:"William"; )

Traffic Scanner configuration file which corresponds to this ruleset is called quicktest. Configuration can be easily loaded by ids utility.

quicktest is default firmware set in ids.conf file, so if you are using original config file, you only need to run command ids without parameters or follow this example.

Example:

# ids -p
Available designs in /usr/local/liberouter/mcs/index/IDS_1Gbps_Traffic_Scanner are:
1: bleeding_rules
2: p2p_search
3: quicktest
4: virus_search
# ids -n 3 # Load quicktest configuration

6.1.1   PC2 with NIC card

Configure IP address for eth0 interface and create record in arp table. After that you will be able to send icmp packets to Traffic Scanner by ping command.

Example:

# ifconfig eth0 10.0.0.1 netmask 255.255.0.0
# arp -s 10.0.0.2 00:00:00:00:00:01
$ ping 10.0.0.2

6.2   Functionality verification

6.2.1   Processing incoming packets

  1. Run ping 10.0.0.2 on PC2
  2. Run idsctl -c status_all on PC1. You will see that counter of received packets is being incremented.

Example:

# idsctl -c status_all
IDS Status -------------------------------------
IRQ Reg                 : False
Mask Reg                : False
Inter Card Enable       : True

IBUF0 Status -----------------------------------
Cnt. packets : 12
Err. packets : 0
Lost packets : 0
Buffer Enable: True

IBUF1 Status -----------------------------------
Cnt. packets : 0
Err. packets : 0
Lost packets : 0
Buffer Enable: True

IBUF2 Status -----------------------------------
Cnt. packets : 0
Err. packets : 0
Lost packets : 0
Buffer Enable: True

IBUF3 Status -----------------------------------
Cnt. packets : 0
Err. packets : 0
Lost packets : 0
Buffer Enable: True

CLASSIFIER Status ------------------------------
Prefetch Status : (1) - IDLE
Csf Status      : (1) - IDLE
Cnt. packets    : 12
Cnt. not IPv4   : 0
Cnt. UKN        : 0
Cnt. TCP        : 0
Cnt. UDP        : 0
Cnt. ICMP       : 12

PTRN_MATCH Status ------------------------------
FSM Status      : (1) - Idle
Cnt. packets    : 12
Match packets   : 0
Exp. All        : False
Exp. Not Match  : False

SWOBUF Status ----------------------------------
FSM Status      : (4) - Wait for Start of packet
Recv. Packets   : 12
Stored Packets  : 0
Buffer's flags  : 00000000

6.2.2   Exporting ICMP with specified content

  1. On PC1 run tcpdump -xi c6eth00
  2. On PC2 send two ICMP packets with defined content which match first two rules. ICMP packet with "George" content can be send using this command ping -c 1 -p 47656f726765 10.0.0.2. ICMP packet with "Thomas" content can be send using this command ping -c 1 -p 54686f6d6173 10.0.0.2.
  3. You can see that packets have been exported by Traffic Scanner.
# tcpdump -x -i c6eth00
tcpdump: WARNING: c6eth00: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on c6eth00, link-type EN10MB (Ethernet), capture size 96 bytes
18:43:22.485904 IP 10.0.0.1 > 10.0.0.2: icmp 64: echo request seq 0
      0x0000:  4500 0054 0000 4000 4001 26a7 0a00 0001  E..T..@.@.&.....
      0x0010:  0a00 0002 0800 3ea2 e05a 0000 aa86 0945  ......>..Z.....E
      0x0020:  2c4e 0700 6f72 6765 4765 6f72 6765 4765  ,N..orgeGeorgeGe
      0x0030:  6f72 6765 4765 6f72 6765 4765 6f72 6765  orgeGeorgeGeorge
      0x0040:  4765 6f72 6765 4765 6f72 6765 4765 6f72  GeorgeGeorgeGeor
      0x0050:  6765                                     ge
18:43:25.560589 IP 10.0.0.1 > 10.0.0.2: icmp 64: echo request seq 0
      0x0000:  4500 0054 0000 4000 4001 26a7 0a00 0001  E..T..@.@.&.....
      0x0010:  0a00 0002 0800 261e e15a 0000 ad86 0945  ......&..Z.....E
      0x0020:  0772 0800 6f6d 6173 5468 6f6d 6173 5468  .r..omasThomasTh
      0x0030:  6f6d 6173 5468 6f6d 6173 5468 6f6d 6173  omasThomasThomas
      0x0040:  5468 6f6d 6173 5468 6f6d 6173 5468 6f6d  ThomasThomasThom
      0x0050:  6173                                     as

6.2.3   Exporting all UDP packets

  1. Rule alert udp any any <> any any ( ) cause that all udp packets is being exported by Traffic Scanner. UDP packet can be generated on PC2 using this command
# dnet rand 8 | dnet udp | dnet ip proto udp | dnet eth | dnet send eth0
  1. If you run idsctl -c status_all you can see that UDP packet counter and stored packet counter have been incremented. So Traffic Scanner with quicktest configuration exports all UDP packet.

6.2.4   Exporting TCP packets with specified destination and content

  1. On PC1 run tcpdump -xi c6eth00
  2. On PC2 send two TCP packets with defined destination and content which match tcp rules. TCP packet with "Michael" and "William" content can be send using this commands:
# dnet hex "\x4d\x69\x63\x68\x61\x65\x6c" | dnet tcp dport 666 | dnet ip proto tcp dst 10.0.0.10 | dnet eth | dnet send eth0
# dnet hex "\x57\x69\x6c\x6c\x69\x61\x6d" | dnet tcp dport 1024 | dnet ip proto tcp | dnet eth | dnet send eth0
  1. You can see that packets have been exported by Traffic Scanner.
# tcpdump -xi c6eth00
tcpdump: WARNING: c6eth00: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on c6eth00, link-type EN10MB (Ethernet), capture size 96 bytes
22:55:51.206652 IP 0.77.241.58.44816 > 10.0.0.10.666: S 5108026:5108033(7) win 65535
      0x0000:  4500 002f af10 0000 ff06 1127 004d f13a  E../.......'.M.:
      0x0010:  0a00 000a af10 029a 004d f13a 0000 0000  .........M.:....
      0x0020:  5002 ffff 92e0 0000 4d69 6368 6165 6c    P.......Michael
22:56:05.822899 IP ip70-176-61-25.ph.ph.cox.net.44624 > 229.153.76.40.1024: S 1185955097:1185955104(7) win 65535
      0x0000:  4500 002f ae50 0000 ff06 57ed 46b0 3d19  E../.P....W.F.=.
      0x0010:  e599 4c28 ae50 0400 46b0 3d19 0000 0000  ..L(.P..F.=.....
      0x0020:  5002 ffff 29ff 0000 5769 6c6c 6961 6d    P...)...William

7   Building customized firmware

For accelerating user specified IDS ruleset new IDS firmware must be generated. For this task a Liberouter's Traffic Scanner Web interface is used. You can easily specify your Snort ruleset and after a while you will be notified by email when building your customized firmware is done. Then you can download firmware and load it using ids utility which is included in package.

Web interface can be found on this WWW page:

7.1   Files included in generated package

Each design and generated package with customized firmware consists of these files:

  • cam_init.txt - content of Content Addressable Memory (CAM). Used with tools camctl and idsctl.
  • combo6x-1-2vp50.mcs - firmware for main FPGA chip at combo6x card.
  • compile_log - compilation log.
  • design.xml - xml description of entire design. Describes address spaces of used components, design files and other necessary files.
  • pcippc-scampi-ph2-01-03 - program for PowerPC on PCI bridge.
  • ruleset.txt - ruleset corresponding to generated design.
  • sfpro-1-2vp20.mcs - firmware for FPGA chip at sfpro add-on card.

8   Examples of usage

8.1   Network placement

Traffic Scanner allow two types of connection depending on your network infrastructure. Both connection is for attacker invisible.

8.1.1   Span port connection

Huge amount of traffic goes through central router. Router have one or more span/mirror ports. Up to four 1 Gbps span ports can be connected to Traffic Scanner.

                      -----------                         -----------
  high speed network  |         |  1-4x 1Gbps mirror port |   PC +  |
<-------------------->|  CISCO  | ----------------------- |  COMBO  |
                      |         |                         |         |
                      -----------                         -----------

8.1.2   Network probe connection

There is 1 Gbps network line which you want to monitor. This Traffic Scanner connection allows monitoring two network lines with minimum added delay.

                              --------------
                       c6eth0 |            | c6eth1
Line0 (1 Gbps) <------------->|            |<-------------> Line0 (1 Gbps)
                              |    PC +    |
                       c6eth2 |   COMBO    | c6eth3
Line1 (1 Gbps) <------------->|            |<-------------> Line1 (1 Gbps)
                              |            |
                              --------------

8.2   Loading included designs

This package contain four Traffic Scanner configuration.

  • quicktest - configuration used for demonstration of Traffic Scanner. Please see section Quick test.
  • virus_search - configuration used for accelerating virus searching. Rules can be found in your mcs directory /usr/local/liberouter/mcs/ids/01_00/virus_search/rules
  • p2p_search - configuration used for accelerating peer2peer searching. Rules can be found in your mcs directory /usr/local/liberouter/mcs/ids/01_00/p2p_search/rules
  • bleeding_rules - configuration used for accelerating Bleeding Snort rules. Rules can be found in your mcs directory /usr/local/liberouter/mcs/ids/01_00/bleeding_rules/rules

Configuration can be loaded using ids utility.

Example:

# ids -p
Available designs in /usr/local/liberouter/mcs/index/IDS_1Gbps_Traffic_Scanner are:
1: bleeding_rules
2: p2p_search
3: quicktest
4: virus_search
# ids -n 4 # Load virus searching configuration

8.3   Using a Snort

Snort is a open source network intrusion detection and prevention system. Snort is the most widely deployed intrusion detection and prevention technology worldwide and has become the de facto standard for the industry. Snort low throughput (about 300 Mbps) can be improved by Traffic Scanner which preprocess incoming traffic. Using Snort with Traffic Scanner cause that about 99 % of traffic is filtered by Traffic Scanner and 1 % (malicious one) is processed by Snort.

Snort is available to download on this WWW page:

Snort can be easily run with preconfigured designs using below command. When is Snort used with Traffic Scanner it's recommended to switch off stream reassembling preprocessors (frag3, stream4).

# snort -i c6eth00 -c `/usr/local/liberouter/mcs/ids/01_00/<configuration>/snort.conf -l <logdir>

8.4   Analyzing data with Sguil

Sguil is tool built for network security analysts. Sguil's main component is an intuitive GUI that provides access to realtime events, session data, and raw packet captures. Sguil facilitates the practice of Network Security Monitoring and event driven analysis.

This tool is also capable of performing queries and data-mining with data produced by Snort.

Sguil is available to download on this WWW page:

8.5   Report generation with SnortALog

SnortALog is available to download on this WWW page:

Full reports can be easily generated by this command:

# cat ${LOGDIR}/alert | ./snortalog.pl -r -o ${OUTFILE} -report

Full report generation is very time consuming for a large log files. For a quicker report generation is recommended to use another way of log processing or you can use reduced report generation using this command:

# cat ${LOGDIR}/alert | ./snortalog.pl -r -o ${OUTFILE} -attack -class -proto -daily_event -hour -hour_attack

9   Known HW and SW limitations, problems

  1. Some unusual way of extracting the package may cause break symbolic links located in firmware/index/IDS_1Gbps_Traffic_Scanner/ . To repair links simply run firmware/make_index script.
  2. Support only for IEEE 802.3
  3. Scanning for malicious content only in IPV4 packets

9.1   Common errors occurring during work

  • csboot: unable to detach driver: Device or resource busy

Shut down c6eth interfaces first - run ids -s

  • Combosix device "/dev/combosix/0" doesn't exist. Please create all necessary devices.

Run ids startup script located in /etc/init.d - ids restart (if you followed this README and installed rc script) or run combo6devices script located in ids package in ids-1.1.2/base/sys_sw/drivers/linux

10   Contacting the Liberouter team

10.1   Mailing lists

For any questions or general technical support issues, please send mail to the liberouter-support@liberouter.org general mailing list. ids-group@liberouter.org is mailing list for registered users.

10.2   Submitting problem reports

Suggestions, bug reports and contributions of code are always valued. Please do not hesitate to report any problems you may find. Bug reports with attached fixes are of course even more welcome. You can use our mailing list to report bugs or (preferably) our bug tracking system at http://www.liberouter.org/bugtrack/ - project IDS.

10.3   Other COMBO6 based projects

Check our web pages for information about other projects, e.g. HW accelerated flow exporter or network interface card with hardware packet filtration.