| Date: | December 2007 |
|---|---|
| Version: | 1.3.0 |
| Author: | The Liberouter Project Team |
| Organization: | CESNET, z.s.p.o. |
This README document is used as short manual to build, install and simple use (functionality test). More detailed information about using FlowMon probe and its configuration is available in the FlowMon Handbook.
Please do not use this README only as a step-by-step manual and read all of README first to familiarize with all available features.
The FlowMon probe is a passive network monitoring device based on the COMBO6(X) technology. It is able to collect dynamic data about IP flows and export them to external collectors in the NetFlow version 5 and 9 format. The probe has great contribution to the safety and reliability of your network. You can obtain information about attacks and data transfers going in and out of your network.
You can check latest news about FlowMon Probe at our web sites:
FlowMon package is available via WWW download page:
To verify that the downloaded files are genuine and complete SHA1 digests are available. Download both the CHECKSUM.SHA1 file and the package file to the same directory. Run the sha1sum(1) command to verify the package file e.g. flowmon-1.2.0.tgz:
$ cat CHECKSUM.SHA1 | grep flowmon-1.2.0.tgz | sha1sum -c
If there are any errors, they will be reported.
/firmware - COMBO6(X) card firmware (*.mcs files)
/base - source code for software tools and drivers
/mk - build system (makefiles)
/sys_sw/drivers - kernel drivers
/sys_sw/hwtools - necessary hardware tools for COMBO6(X) card
/sys_sw/lib* - libraries necessary for other tools
/sys_sw/projects/flowmon - exporters, testing programs, configuration
and documentation for using FlowMon
/sys_sw/projects/netconf - NETCONF protocol programs
/sys_sw/swtools/csxtool - tool for handle COMBO6(X) XML files
/doc - FlowMon project documentation - FlowMon Handbook
ERRATA - list of known bugs and their solutions
README - this file
RELNOTES - differences against previous release
The FlowMon hardware has been designed to work on any PC-AT compatible computer running GNU/Linux or other "Unix-like" operating system. It has been tested only on computers that use the x86 family of processors. The hardware requirements necessary to run FlowMon are:
The "sandwich" consisting of COMBO6(X) and an interface card (COMBO-4MTX, COMBO-4SFP, COMBO-4SFPRO or COMBO-2XFP) fits into one PCI slot.
| Mother card | Add-on card | Description |
|---|---|---|
| COMBO6 | a PCI (32/33) card with XILINX FPGA | |
| COMBO-4MTX | add-on card with four copper GE interfaces | |
| COMBO-4SFP | add-on card with four SFP cages for GE interfaces | |
| COMBO6X | a PCI-X (64/66) card with XILINX FPGAs | |
| COMBO-4SFPRO | add-on card with four SFP cages for GE interfaces | |
| COMBO-2XFP | add-on card with two XFP cages for 10GE interfaces |
| NOTE: | more information about COMBO6(X) cards you can get on http://www.liberouter.org/hardware.php |
|---|
FlowMon software works on GNU/Linux OS with 2.4 and 2.6 kernels. The NetFlow v5 and v9 protocols are supported. The software has been tested on computers running Red Hat Enterprise Linux, Ubuntu and Debian.
More information about FlowMon probe you can find at http://www.flowmon.org
After plugging COMBO6(X) card into your PCI slot, you should test connection between the card and your PC. We use lspci(8) utility for this purpose. lspci(8) is a utility for displaying information about all PCI buses in the system and all devices connected to them. For correct recognition of the COMBO6(X) card you need update PCI ID Database used by lspci(8) or download pciutils-2.2.2 (program collection containing lspci(8)) or later. If the output of lspci(8) contains the following line your COMBO6(X) card is connected properly.
$ lspci -d 18ec: 03:01.0 Ethernet controller: Cesnet, z.s.p.o. COMBO6X (rev 01) 04:02.0 Ethernet controller: Cesnet, z.s.p.o. COMBO6 (rev 01)
| NOTE: | Numbers at the beginning of the line can be different. |
|---|
Compilation and installation process is covered by the build system located in the base/mk directory. For more details about required tools necessary to build package see base/mk/REQUIREMENTS file.
To build kernel drivers needed by FlowMon package you will need fully configured Linux 2.4 or 2.6 kernel source tree. More installation and compilation details about kernel drivers are available in the README file inside base/sys_sw/drivers/linux directory.
pkgtool(1) is used to build and install the FlowMon distribution package.
$ tar -xzvf flowmon-XX.YY.ZZ.tgz $ cd flowmon-XX.YY.ZZ/base $ ./pkgtool --build
| NOTE: | Replace XX.YY.ZZ with the chosen package version number such as 1.3.0. |
|---|
If this is the first Liberouter package you have ever installed, you can define installation directory with the --prefix=path option. But remember that this installation path MUST not exists (e.g. /usr/local is invalid installation path on most systems because this directory exists). This restriction is due to new installation framework which enables easy package uninstall or package switching. More information about these features can be found in the FlowMon Handbook in the liberouterpkg section.
The --prefix option takes effect only during building package. If no prefix is set then /usr/local/liberouter path is used.
$ ./pkgtool --build --prefix=/usr/local/flowmon
If you have previously installed some Liberouter package (nific, nic, ids, flowmon with liberouterpkg mechanism), the installation path is detected automatically as path used for the first installed package.
Remember that installation path given as --prefix parameter will contain next subdirectories for binaries, libraries, man pages, etc. These directories can be affected by future uninstalling or package switching so it could be used as installation directory only for Liberouter packages.
Installed tools are divided to three parts.
server - This is a default type of the installation that includes all necessary tools to control COMBO card, kernel drivers and NETCONF tools (server side as well as client side tools). All these tools will be installed to the installation directory (by default /usr/local/liberouter). This type of installation is suitable for PCs with installed COMBO6(X) card.
To use this type of installation, use --install option with server parameter:
# ./pkgtool --install=server
or simply use --install option without any parameter:
# ./pkgtool --install
client - Only NETCONF-client side tools are installed in this case. These tools are included in the server and web installation too. This type of installation is suitable for manager PCs without running web server.
To install only these tools, use --install option with client parameter:
# ./pkgtool --install=client
web - Last installation type enables to install FlowMon web configuration frontend into your Apache web server including NETCONF-client side tools needed by web configuration frontend. This type of installation is suitable for manager PCs with running web server.
To use this type of installation, please use --install option with web parameter:
# ./pkgtool --install=web
You can use more than one type of installation (typically server and web) by specifying keywords separated by comma(s):
# ./pkgtool --install=server,web
If you are going to use udev mechanism to creating device files, you can use pkgtool with --udev option. This option cause copying file with COMBO6(X) card rules (combo6.udev.rules) to the /etc/udev/rules directory.
# ./pkgtool --install --udev
| NOTE: | --udev option takes effect only with --install option. |
|---|
There are necessary following post-install steps which have to be done. Post-installation steps can be skipped (except last one) if you performed these steps during previous installation.
If you have used only client or web installation type you should skip steps 4 and 5.
Set PATH variable to point to installed FlowMon tools directory -> ${PREFIX}/bin
$ export PATH=/usr/local/liberouter/bin:$PATH
For permanent PATH change add following line into the /etc/profile configuration file
export PATH=/usr/local/liberouter/bin:$PATH
Configure dynamic linker run time bindings by setting /etc/ld.so.conf to point to the FlowMon library directory -> ${PREFIX}/lib
Add /usr/local/liberouter/lib (by default) entry to the /etc/ld.so.conf file.
# echo "/usr/local/liberouter/lib" >> /etc/ld.so.conf # ldconfig
Set /etc/man.config file to point to the FlowMon man pages -> ${PREFIX}/man
Add MANPATH /usr/local/liberouter/man entry to the /etc/man.config file (name of this file can little differ on different Linux distribution.
If you want to load FlowMon kernel modules when the PC starts up, add following lines to the /etc/modules file.
For COMBO6 card:
libermemalloc combo6core combo6 szedata netflow-ph1
For COMBO6X card:
libermemalloc combo6core combo6x szedata szedatax-c6pcr
After this, you don't need to run flowmonlkm script any more.
| NOTE: | In some distribution (e.g. Red Hat Enterprise Linux 4) you have to use file /etc/rc.modules in place of /etc/modules file. This file contains commands to load kernel modules at boot time (it is actually script so it must be an executable file), so such file will look like this: |
|---|
#!/bin/sh /sbin/modprobe libermemalloc /sbin/modprobe combo6core /sbin/modprobe combo6x /sbin/modprobe szedata /sbin/modprobe szedatax-c6pcr
The FlowMon probe driver needs device files in the /dev/combosix/ and /dev/szedata/ directory. To create them, run the flowdevices script in the packages directory base/sys_sw/drivers/linux
# ./flowdevices
To run FlowMon as a non-privileged user set r/w permissions on /dev/combosix/0 and /dev/szedata/0 devices.
You have to change the permissions of the device files so that the non-privileged user that will execute the FlowMon tools has read-write access. For example, you can create a new group, say combo-rw, add that user to this group and change the file group ownership and permissions as follows:
# chgrp combo-rw /dev/combosix/0 /dev/szedata/0 # chmod 664 /dev/combosix/0 /dev/szedata/0
Using recent 2.6 kernels it might be wanted to use the sysfs/udev mechanisms to create device files. For more details about udev see http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev.html
Example udev rules are placed in the package in sys_sw/drivers/linux/combo6.udev.rules file. If you didn't use option --udev during package installation, copy this file to /etc/udev/rules.d directory and modify it (change group or permissions). Following line will create COMBO6(X) device files required by the FlowMon probe:
``KERNEL=="combosix[0-9]*", NAME="combosix/%n", GROUP="combo-rw", MODE="0664"`` ``KERNEL=="szedata[0-9]*", NAME="szedata/%n", GROUP="combo-rw", MODE="0664"``
| NOTE: | Depending on your version of udevd you must adjust syntax of your udev rule file. Older versions of udevd require '=' character to match KERNEL key e.g. KERNEL="combosix[0-9]*". The newer one require '==' characters to match KERNEL key e.g. KERNEL=="combosix[0-9]*". |
|---|
Please copy (and rewrite any previous version) liberouterpkg script, located in the package base directory into your standard binary directory (e.g. /usr/local/bin/). This enables using liberouterpkg script as standard system tool:
$ liberouterpkg
| NOTE: | Path where the liberouterpkg script is placed must be different from used package installation path. |
|---|
This is the first Liberouter package using liberouterpkg tool enabling package switching and fully package uninstalling. If you have installed any previous Liberouter package without liberouterpkg, please remove it completely (including all libraries and drivers) to ensure proper behaviour of the installed package.
More information about these features can be found in the FlowMon Handbook in the liberouterpkg section.
If you are installing new version of previously installed package (e.g. you have installed flowmon-1.3.0 and now you are installing flowmon-1.3.1 package) you will be asked by pkgtool(1) to decide if you wish to keep your own (but may be obsolete) configuration files or to overwrite them with our default (but up-to-date) configuration files.
This procedure serves to start FlowMon probe - probe will not be automatically started again after rebooting PC. If you wish to start the FlowMon probe anytime you will reboot the PC, please see the FlowMon Handbook, section FlowMon init.d script.
After building and installing package including post-install steps, all FlowMon tools (included man pages) are available as any other system tools. There are two main scripts to start the probe - flowmon and flowmonlkm. Both scripts use /etc/liberouter/flowmon.conf configuration file.
For more details how to run flowmon* scripts use -h option or see flowmon and flowmonlkm man pages. Other FlowMon probe tools come with the manual pages. These manual pages can be viewed with the man(1) command e.g.
$ man flowmon_nf5
| NOTE: | Commands which begin with '#' must be run by root. To run commands as a non-privileged user '$' don't forget to set r/w permissions on /dev/combosix/0 and /dev/szedata/0 devices. |
|---|
flowmonlkm script is used for loading/removing FlowMon kernel modules. There are two main options for this script. Option -l is used for loading FlowMon kernel modules and option -r is used for removing these modules. Script uses lspci(8) utility to detect COMBO6(X) card.
When all kernel modules are loaded, csid(1) is able to detect type of plugged cards and provide more detailed information.
You should get following output on PC with installed
COMBO6 mother card and mtx add-on card:
$ csid combo6 mtx2 xcv2000
COMBO6X mother card and sfpro add-on card:
$ csid combo6x sfpro xc2vp20
| NOTE: | Chips can be different on some cards, so don't worry about differing last part of the csid output |
|---|
flowmon is main start up script for the FlowMon probe. Script boots firmware files (according to FIRMWARE_PATH variable from the /etc/liberouter/flowmon.conf), setting up probe behavior and is able to start NetFlow protocol exporter. By default, the probe settings are set according to variables from /etc/liberouter/flowmon.conf configuration file. Optionally you can define your own settings by options given to the script. More detailed information you can get by -h option or from man page:
$ flowmon -h $ man flowmon
| NOTE: | Both scripts (flowmon as well as flowmonlkm) using lsof(8) tool to detect any process using COMBO6(X) device(s). To run scripts properly you have to install lsof(8) tool. lsof(8) is utility to get information about files opened by processes. |
|---|
The FlowMon probe logs important events to syslog using the user facility. To have these log messages saved to a file, you must configure the syslog daemon appropriately. In the case of the classical Unix syslogd daemon, add the following line to /etc/syslog.conf
user.* /var/log/user.log
Regarding to your Linux distribution see syslogd(8) and logrotate(8) documentation how to adjust log file rotation.
# flowmonlkm -l $ flowmon -ec collector.liberouter.org:60000
| NOTE: | Server collector.liberouter.org is only example name of the collector host which doesn't exist. Use, please, your own (existing) collectors. |
|---|
To make sure that the firmware was loaded correctly you can run csid(1):
$ csid -s Board : combo6 Addon : mtx2 Chip : xcv2000 LAN ports: 4 Firmware : ok SW : 0xf1010002 HW : 0x00000006 Text : NETFLOW_1Gbps_Probe
HW number detected by csid -s reflects firmware version, e.g. HW: 0x00000006 reflects version NETFLOW_01_06.
To stop running exporter(s) and detach drivers, use flowmon script with the -s option:
$ flowmon -s
Then you can remove FlowMon loaded kernel modules by the flowmonlkm script:
# flowmonlkm -r
To read data from HW and send them to collector you can run several instances of FlowMon exporters. Before running exporter you must start FlowMon probe (load kernel modules and run flowmon script):
$ flowmon_nf5 -d collector.liberouter.org:60000 $ flowmon_nf9 -d collector.liberouter.org:60001
The exporter program will start sending flow records to the configured collector. To test that it is really the case, you can run a packet sniffer such as tcpdump(1) or ethereal(1), for example:
# tcpdump -i <interface> 'udp dst port <collector_port>' # tcpdump -i eth0 'udp dst port 60000'
You should see the NetFlow traffic sent by the flow exporter to the collector. To test basic collector functionality you can use flowmoncol(1) tool:
$ flowmoncol -l 60000
For any questions or general technical support issues, please send mail to the netflow [at] liberouter.org general mailing list.