NIFIC Handbook

The firmware is automatically loaded by NIFIC script d, which is described in Chapter 5, the design could be changed in nific.conf(5) file as is described in Chapter 4. Supported firmware version is described in the table:

NIFIC host PC software works on GNU/Linux OS with 2.6 kernel. The software has been tested on computers running CentOS.

After plugging COMBO-LXT card into your PCI slot, you should test connection between the card and your PC. We use lspci(8) utility for this purpose. lspci(8) is a utility for displaying information about all PCI buses in the system and all devices connected to them. For correct recognition of the COMBO-LXT card you need update PCI ID Database used by lspci(8) or download pciutils-2.2.2 (program collection containing lspci(8) or later. If the lspci(8) output contains the following line your COMBO-LXT card is connected properly:

	$ lspci -d 18ec:
	09:00.0 Ethernet controller: Cesnet, z.s.p.o.: Unknown device c032

NIFIC host PC is designed to be connected directly to line as active device, because it enables packet forwarding on full speed without any packet loss. However, when NIFIC is used as a passive device the mirror port or the Network tap could be used to connect NIFIC host PC to the desired network.

NOTE: LEDs are blinking and phyterctl(1) tool is working only when NIFIC design is booted in card by nific(1) script.

Ports numbers

The combination of COMBO-LXT and COMBO-10G2 provides two 10 Gbps Ethernet interfaces. The following picture shows the number of the interface in the front view. The interface 0 is always the nearest one to the PCIe bus.

Figure 3.2. Ethernet interface numbers

LEDs indicating state

The COMBO-10G2 card is equipped with LED diodes as the link states indicators. The LED diodes are situated on the back side of the combination of COMBO-LXT and COMBO-10G2 as is shown on the picture:

Figure 3.3. LEDs on the COMBO-10G2

There are two diodes per port: green diodes for RX (receiving) and orange or red for TX (transmitting). The diodes has following meanings:

• Green diode (RX)

  • shining - line is present

  • blinking - packets coming on the line

  • neither blinking nor shining - no line present

• Orange or red diode (TX)

  • shining - packets are not transmitted

  • blinking - packets are transmitted

  • neither blinking nor shining - should never happen

Liberouter project has a common centralized RPM repository for all projects. It is available on URL: https://www.liberouter.org/repo/STABLE/x86_64. Repository works over secured HTTPS channel. To obtain packages from it, you have to authorize yourself with login and password which you have received as our client. (Eventually, use login and password for SVN account.)

To make the standard tool yum(1) work properly with our repository, you have to setup some configuration. Add a new file liberouter.repo to directory /etc/yum.repos.d/ with content:

            [liberouter]
            name=Liberouter RPM repository
            baseurl=https://LOGIN:PASS@www.liberouter.org/repo/STABLE/x86_64/
            enabled=1
            gpgcheck=1
            gpgkey=https://www.liberouter.org/repo/RPM-GPG-KEY-Liberouter

# chmod 600 /etc/yum.repos.d/liberouter.repo

Now you can check your configuration using yum(1):

               # yum repolist
               ...
               liberouter           Liberouter RPM repository                enabled
               ...
            

NIFIC host PC software is distributed in form of several RPM packages. We currently support only CentOS 5 distribution

To install nific packages we recommend to use yum's groupinstall command

# yum groupinstall nific-combov2-10g2

or

# yum groupinstall nific-combov2-1g4

according to installed card type.

As described in Chapter 3 and Chapter 6, NIFIC includes remote control tools. They have to be installed individually on PC selected to be a NIFIC remote center.

The yum repositories have to be set up the same way as was described in previous section. If the repositories are set up, the NIFIC monitoring center package could be installed by:

# yum install nific-remote-monitor

The remote configuration tool (netopeer manager) could be installed by:

# yum install netopeer-manager

The Host PC packages could be removed by

# yum groupremove nific-combov2-10g2

or

# yum groupremove nific-combov2-1g4

according to installed card type.

Similary, the remote center packeges could be removed by

# yum remove nific-remote-monitor

# yum remove netopeer-manager

The nific.conf(5) file includes the actions, which NIFIC is executing when non IPv4 packet is detected on the network.

# ceth* interface  15        10 9 8 7 6 5 4 3 2 1 0     crop size
 DEF_NON_IPV4_IFC0="0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0	0"
 DEF_NON_IPV4_IFC1="0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0	0"
 DEF_NON_IPV4_IFC2="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1	0"
 DEF_NON_IPV4_IFC3="0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0	0"
 DEF_NON_IPV4_IFC4="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0	0"
 DEF_NON_IPV4_IFC5="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0	0"
 DEF_NON_IPV4_IFC6="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0	0"
 DEF_NON_IPV4_IFC7="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0     0"
 DEF_NON_IPV4_IFC8="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0	0"
 DEF_NON_IPV4_IFC9="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0	0"
DEF_NON_IPV4_IFC10="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0	0"
DEF_NON_IPV4_IFC11="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0	0"
DEF_NON_IPV4_IFC12="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0	0"
DEF_NON_IPV4_IFC13="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0	0"
DEF_NON_IPV4_IFC14="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0	0"
DEF_NON_IPV4_IFC15="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0	0"

The standard network interfaces are created on the basis of nific.conf(5) file. NIFIC filter packets up to 14 virtual sw interfaces, which could be mapped to the standard network interfaces. User could define the combinations of input and output virtual interface to be accessible by one standard network interface. According to the NIFIC hardware architecture, user has to define rules if he wants to transmit any IPv4 packet by virtual interfaces, the example could be found in examples of rules. When, standard network interface is set up, the OS sends also non-IPv4 packets to the network, so the actions for non-IPv4 packets also have to be set up.

#    VIF0="2	2	10.0.0.1	255.255.255.0"
VIF0_NAME="ceth2"
VIF1="3	3	172.16.0.1	255.255.255.0"
VIF1_NAME="ceth3"

For loading/removing NIFIC kernel modules the nificlkm(1) script is used. This script must be executed with root privileges. There are two main options for this script. Option -l is used for loading kernel modules. Option -r is used to remove any modules previously loaded. For detection of the card and appropriate kernel modules the lspci(8) utility is used.

Combosix device "/dev/combosix/0" doesn't exist.
Please create all necessary devices.
    

When all kernel modules are loaded, csid (1) is able to detect type of plugged cards and provide more detailed information. You should get following output on PC with installed COMBO-LXT mother card and COMBOI-10G2 add-on card:

$ csid
combo 10G1 n/a

nific(1) is main start up script for the NIFIC packet filter. Script could be run in two different modes: local or remote, the configuration is taken from nific.conf(5). Main functionality is described below, more detailed information you can get by -h option:

The first task is a kernel modules loading:

       $ su
       # nificlkm -l
       Loading NIFIC kernel modules
       combo6x                15252  0
       combo6                 21980  0
       combo6core             22836  4 szedata2_cv2,combov2,combo6x,combo6
       combov2_boot            4600  0
       combov2                16256  2 szedata2_cv2,combov2_boot
       i2c_core               22593  1 combov2
       combo6core             22836  4 szedata2_cv2,combov2,combo6x,combo6
       szedata2_cv2           12948  0
       szedata2               16268  1 szedata2_cv2

When the modules are loaded we can run the nific(1) script:

      # nific local
      ----------------------------------
        Starting Configuration Daemon
      ----------------------------------
      ----------------------------------
         Creating Virtual Interfaces
      ----------------------------------

User could check if the design is properly loaded by csid -s.

     $ csid -s
     Board    : combo
     Subtype  : LXT155
     S/N      : 8100323
     Addon0   : 10G1
     Chip0    : n/a
     S/N0     : 8100372
     Addon1   : 10G1
     Chip1    : n/a
     S/N1     : 8100372
     Channels : 2/2 (RX/TX)
     Firmware : ok
     SW       : 0x41f10100
     HW       : 0x00030002
     Text     : NIFIC_CV2_10G2
     PCI brver: 6d05.01.04 (2009/06/10 14:24)

     Device [combo6] szedata2
       (0x41c10400-0x41c105ff) {}: inactive
     Device [combo6] szedata2
       (0x41f10100-0x41f101ff) {}: active
     Device [combo6] szedata2
       (0xf1010100-0xf10101ff) {}: inactive

Now user can work with packet filter as is described in next Chapters. To stop the nific and unload modules use commands:

  # nific -s
  # nificlkm -r

NIFIC packet filter could be set up during boot of NIFIC host PC by script nific.rc which is located in /etc/init.d/. The startup script loads modules and runs the nific(1) script in local or remote mode as described in nific.conf(5). When user wants to use nific.rc script during the boot process, the chkconfig(8) must be configured:

  # chkconfig --add nific.rc

To verify that everything is right, user can find nific.rc in the list of services

  # chkconfig --list | grep nific.rc

To control service manually (start, stop, restart, etc.) use one of available options:

  # /etc/init.d/nific.rc help

However, the service will be started again after reboot of the NIFIC host PC. To stop the service permanently use:

  # chkconfig nific.rc off

To delete the service from list run:

  # chkconfig --del nific.rc

The "Filter language" is used to describe a rule set for the NIFIC filter. It is similar to the OpenBSD PF or FreeBSD ipfw language and is NOT case-sensitive.

Decimal and hexadecimal digits are defined:

dec_digit := '0' – '9'
hexadecimal_digit := dec_digit | ('a' – 'f')

Decimal constant is a sequence of decimal digits:

dec_const := dec_digit+

Hexadecimal constant is a sequence of hexadecimal digits:

hexadecimal_const := hexadecimal_digit+

IP address is a sequence of four decimal constants from range 0-255 separated by '.'. The address may be followed by a '/' and a number of bits from the left that are considered in evaluation. There must be no space around '/' character::

ipv4_const := dec_const ('.' dec_const){3} ['/' dec_const]

Examples of correct IPv4 addresses:

10.0.0.0/8
10.1.2.3

MAC address is a sequence of six doubles of hexadecimal numbers 00-FF separated by ':' :

mac_const := hexadecimal_const (':' hexadecimal_const){5}

List is a set of constant ranges surrounded by brackets '{' and '}'. The list may only contain one range type. Commas ',' are ignored inside lists:

list<constant_range> := '{' (constant_range | ',')* '}'

In case a list is used as an attribute value then the condition is satisfied when the attribute equals at least one of the list items. That means the rule:

100 pass 1 from { 10.0.0.0/8, !10.1.2.3 }

has the same meaning as the two rules, which passes any packet::

100 pass 1 from 10.0.0.0/8
100 pass 1 from !10.1.2.3

Therefore it is not recommended to use a negation in a list.

The rule specify an action which is performed on the packet when all requested conditions are met. The rule is defined by following syntax::

rule := id action ['on' interface] ['proto' protocol]
['from' src_addr ['mac' src_mac]['port' src_port]]
['to'   dst_addr ['mac' dst_mac]['port' dst_port]]
['flags' [set_flags] '/' checked_flags] ['flowid' flowid]

ID is the first thing defined in the rule. It also specifies rule priority. Lower numbers denotes higher priority. Id is from 0 to 65535:

id := dec_const

Packets may be blocked or passed (sent) to several interfaces. Passed packets may also be cropped to a defined sizes:

action := ('pass' (interface_number+) ['crop' size]) | 'block'
interface_number := dec_const
size := dec_const

Interface is defined by a decimal constant or list of decimal constants from the range 0-15 possibly with negation:

action := ('pass' (interface_number+) ['crop' size]) | 'block'
interface_number := dec_const
size := dec_const

Protocol is defined by a keyword 'TCP', 'UDP', 'ICMP' or a decimal number from the file /etc/protocols. It is also possible to use negation and lists of protocols:

protocol := protocol_range | list<protocol_range>
protocol_range := ['!'] protocol_const
protocol_const := 'tcp' | 'udp' | 'icmp' | dec_const

Source and destination address may be defined by keyword 'any' which means any address, by one IP address, by address with negation using character '!' or by address list::

src_addr := dst_addr := 'any' | ipv4_range | list<ipv4_range>
ipv4_range := ['!'] ipv4_const

Mac address is defined by single address or by list of addresses:

src_mac := dst_mac := mac_const | list<mac_const>

Source and destination port may be specified by port range or by list of port ranges:

src_port := dst_port := port_range | list<port_range>

Range is defined by no operator meaning the exact specified value, by unary operators non-equal '!=', lesser '<', greater '>', lesser or equal '<=' or greater or equal '>=' and binary operators in range without borders '><', out of range '<>' or in range including borders ':':

port_range := dec_const | (('!=' | '<' | '>' | '<=' | '>=') dec_const) | (dec_const ('><' | '<>' | ':') dec_const)

TCP flags are given in the form set/checked. Individual flags are represented by their first letter. For example 'S/SA' means to check SYN and ACK, where only SYN is set:

set_flags := checked_flags := ('F' | 'S' | 'R' | 'P' | 'A' | 'U' | 'E' | 'C')

Flow ID is a user defined identifier that is returned when the rule is matched.

flowid := dec_const

Rules are evaluated from the lowest priority number to the highest. When a packet is matched by some rule, evaluation is terminated and the rule action is used.

The rule set contains every rule or table definition on a separate row. If the row would be too long, it is possible to use backslash '\' which neutralizes the immediately following newline '\n'::

filter_ruleset := row*
row := [rule] '\n'

It is possible to use comments beginning with '#' and ended by the new line. Example of a simple rules:

#simple pass all traffic from interface 1 to interface 2 (software virtual interface)
10 pass 2 on 1

#pass all traffic from given IP to interface 2
20 pass 2 from 192.168.0.1

#pass all traffic from one IP to another with defined ports to interface 2
30 pass 2 from 192.168.0.1 port 4550 to 192.168.0.2 port 9660

#pass to interface 4 and crop to 400B all traffic from 147.15.2.3
40 pass 4 crop 400 from 147.15.2.3

#ports lesser than 1024
50 pass 4 crop 400 from 147.15.2.3 to any port <1024

#pass all traffic to known hosts
60 pass 1 2 3 to { 147.50.26.0/24 147.51.26.1 }

#all IPv4 packets sends to virtual interface 14 are redirected to the Ethernet interface 0 and all packets from interface 15 are redirected to Ethernet interface 1
70 pass 0 on 14
80 pass 1 on 15

#implicit block
100 block

To reduce memory requirements of the rule set try not to use negations (operator '!' for IPs and operator '!=' for ports). It is usually possible to transform the rule set into equivalent form without negations. Instead of writing:

200 pass 1 from !147.50.26.19
300 block

Write the equivalent but without negations:

200 block from 147.50.26.19
300 pass 1

Also number your rules wisely. If possible leave some rule IDs unassociated. For example when creating a new rule set leave at least hundred IDs free and use a very large number for the default rule ID:

100 pass 1 from 147.50.26.0/24
200 pass 1 from 147.50.27/24
65535 block

Using all IDs in a sequence is a bad idea, because then you will need to renumber the rules when adding a rule into the list. Example of a bad style:

1 pass 1 from 147.50.26.0/24
2 pass 1 from 147.50.27/24
3 block

When nific(1) script is used, it creates virtual network interfaces according to specified requirements in in nific.conf(5). If an example configuration is used, it creates two virtual network interfaces that can be controlled by ifconfig(8):

$ ifconfig ceth2
ceth2 Link encap:Ethernet  HWaddr 00:FF:06:2E:3A:A0
          inet addr:10.0.0.1  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::2ff:6ff:fe2e:3aa0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:0 (0.0 b)  TX bytes:468 (468.0 b)
$ ifconfig ceth3
ceth3 Link encap:Ethernet  HWaddr 00:FF:21:57:A7:D3
          inet addr:172.16.0.1  Bcast:172.16.0.255  Mask:255.255.255.0
          inet6 addr: fe80::2ff:21ff:fe57:a7d3/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:0 (0.0 b)  TX bytes:468 (468.0 b)
 

For packet capturing test user could use the tcpdump(1) tool:

$ tcpdump -i ceth2 

For packet transmitting test user could use the tcpreplay(8) tool, where file packets.pcap is transmitted:

$ tcpreplay -i ceth2 packets.pcap

More detailed description of using tcpdump(1) and tcpreplay(1) tools can be found in section TODO).

$ cat ./rules.user
# pass all traffic from physical interface 0 to software virtual interface 2
10 pass 2 on 0
# pass all traffic from software virtual interface 2 to physical interface 0
11 pass 0 on 2

# pass all traffic from physical interface 1 to software virtual interface 3
20 pass 3 on 1
# pass all traffic from software virtual interface 3 to physical interface 1
21 pass 1 on 3

Libpcap library description is available at http://www.tcpdump.org/.

The development of the user application for processing of packets incoming from NIFIC hardware is a simple task, because standard libpcap library could be used. User have to follow two steps:

Sze2loopback was designed for testing and development purposes. However, this tool could be used for simple visualization of incoming packets headers from NIFIC hardware.

For visualization purposes, the following command runs sze2loopback and prints "R" for every incoming packet on the screen:

  $ sze2loopback -R

When there is to many incoming packets user could use the -I option, then one R is printed per 10000 packets:

  $ sze2loopback -R -I 10000

Tool could be used also for simple visualization of incoming packets:

  $ sze2loopback -R -A

Netopeer tool is controlled by following commands:

close-session     terminate NETCONF session and quit netopeer 
copy-config       create/replace entire configuration with another 
delete-config     delete a configuration datastorage 
edit-config       modify part of configuration 
get               get configuration and device state information 
get-config        get specified configuration 
kill-session      force the termination of a NETCONF session 
lock              lock configuration datastorage of a device 
unlock            release a configuration lock 
commit            use candidate configuration as new running configuration 
discard-changes   revert candidate configuration to the current running configuration 

In the following use cases the batch file is used for storing of the commands. Netopeer tool could by called with batch file simply by:

$ netopeer -b batchfile

Netopeer tool has an obligatory parameter specifying address of the NIFIC host PC, lets say the machine demo.nific.org is our NIFIC host PC. This parameter can be extended by @ to contain ssh user login name:

$ netopeer ssh_user_name@demo.nific.org

If the password for the specified user is required, the password can be given 138 to netopeer by -w option to avoid asking a password by program:

$ netopeer ssh_user_name@demo.nific.org  -w  password

$ cat batchfile  
get-config running -f output.xml 
close-session 

$ netopeer ssh_user_name@demo.nific.org  -w  password-b batchfile

$ cat ./output.xml  
<?xml version="1.0" encoding="utf-8"?> 
<nific xmlns="http://www.liberouter.org/ns/netopeer/nific/1.0"> 
<monitorCenters> 
   <monitorCenter> 
     <description>My monitoring center</description> 
     <host content-type="name">monitor.center.org</host> 
     <port>6666</port> 
     <protocol>tcp</protocol>
     <onInterface>2</onInterface> 
   </monitorCenter> 
</monitorCenters> 
 <rules> 
   <rule> 
     <id>100</id> 
     <value>pass crop 400 from 10.0.0.1 to any</value> 
   </rule> 
   <rule> 
     <id>200</id> 
     <value>pass from 147.15.2.3 to any port &lt;1024</value> 
   </rule> 
   <rule> 
     <id>4000000000</id> 
     <value>block</value> 
   </rule> 
 </rules> 
</nific>  

$ cat ./local_config.xml  
<?xml version="1.0" encoding="utf-8"?> 
<nific xmlns="http://www.liberouter.org/ns/netopeer/nific/1.0"> 
 <monitorCenters> 
   <monitorCenter> 
     <description>My monitoring center</description> 
     <host content-type="name">monitor.center.org</host> 
     <port>6666</port> 
     <protocol>udp</protocol> 
     <onInterface>2</onInterface> 
   </monitorCenter> 
 </monitorCenters> 
 <rules> 
   <rule> 
     <id>100</id> 
     <value>pass crop 400 from 10.0.0.1 to any</value> 
   </rule> 
   <rule> 
     <id>200</id> 
     <value>pass from 147.15.2.3 to any port &lt;1024</value> 
   </rule> 
   <rule> 
     <id>4000000000</id> 
     <value>block</value> 
   </rule> 
 </rules> 
 </nific> 
  
$ cat batchfile
lock candidate 
copy-config candidate -f local_config.xml 
commit 
unlock candidate 
close-session 
  
$ netopeer ssh_user_name@demo.nific.org  -w  password-b batchfile

$ cat batchfile
get-config -i filter.xml -f output.xml running 
close-session 
  
$ cat filter.xml
<?xml version="1.0" encoding="utf-8"?> 
<nific xmlns="http://www.liberouter.org/ns/netopeer/nific/1.0"> 
<rules/> 
</nific> 
 
$ netopeer ssh_user_name@demo.nific.org  -w  password-b batchfile
  
$ cat ./output.xml  
<?xml version="1.0" encoding="utf-8"?> 
<nific xmlns="http://www.liberouter.org/ns/netopeer/nific/1.0"> 
 <rules> 
   <rule> 
     <id>100</id> 
     <value>pass crop 400 from 10.0.0.1 to any</value> 
   </rule> 
   <rule> 
     <id>200</id> 
     <value>pass from 147.15.2.3 to any port &lt;1024</value> 
   </rule> 
   <rule> 
     <id>4000000000</id> 
     <value>block</value> 
   </rule> 
 </rules> 
</nific> 

$ cat ./edit.xml  
<?xml version="1.0" encoding="utf-8"?> 
<nific xmlns="http://www.liberouter.org/ns/netopeer/nific/1.0" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"> 
 <rules> 
   <rule nc:operation="create"> 
     <id>10000</id> 
     <value>block to any port 80</value> 
   </rule> 
   <rule nc:operation="create"> 
     <id>11000</id> 
     <value>block to any port 8080</value> 
   </rule> 
 </rules> 
 </nific> 
  
$ cat batchfile
lock candidate 
edit-config -f edit.xml candidate 
commit 
unlock candidate 
close-session 
  
$ netopeer ssh_user_name@demo.nific.org  -w  password-b batchfile

$ cat ./edit.xml
 <?xml version="1.0" encoding="utf-8"?> 
 <nific xmlns="http://www.liberouter.org/ns/netopeer/nific/1.0" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"> 
 <rules> 
   <rule nc:operation="delete"> 
     <id>10000</id> 
   </rule> 
   <rule nc:operation="delete"> 
     <id>11000</id> 
   </rule> 
 </rules> 
 </nific> 
  
$ cat batchfile
lock candidate 
edit-config -f edit.xml candidate 
commit 
unlock candidate 
close-session 
  
$ netopeer ssh_user_name@demo.nific.org  -w  password-b batchfile

If an error occurs, netopeer in batch mode prints an error information:

$ netopeer ssh_user_name@demo.nific.org  -w  password-b batchfile
<error-tag> : resource-denied 
<error-type> : application 
<error-severity> : error 
<error-message> : Copy-config failed, unable to use repository - Permission denied