Cesnet Liberouter
  • Projects
  • Liberouter
  • Scampi
  • FlowMon
  • NIC
  • NIFIC
  • IDS
  • NetCOPE
  • VHDL design
  • System software
  • Testing
  • Formal verification
  • Netopeer
  • Documents
  • Our hardware
  • Card Availability
  • Our partners
 

PCI Bridge firmware upgrade package

Date: August 2007
Version: 1.0.0
Authors: The Liberouter Project Team
Organization: CESNET, z.s.p.o.

Contents

  • 1   Introduction
    • 1.1   Obtaining pci-bridge package
    • 1.2   Package structure
  • 2   Installation instructions
    • 2.1   Host computer
    • 2.2   Supported hardware
    • 2.3   Supported software
    • 2.4   Installing COMBO6X cards
    • 2.5   Building package
    • 2.6   Installing package
    • 2.7   Upgrading package
    • 2.8   Uninstalling package
  • 3   PCI bridge firmware upgrade guide
    • 3.1   Requirements on jumper settings
    • 3.2   Important connectors and jumpers on COMBO6X card
  • 4   Upgrading with Liberouter's "programming shortcut cable" and SVF tool
    • 4.1   SVF tool
    • 4.2   Post installation steps
    • 4.3   What to do when the card "dies" after corrupted programming
  • 5   Upgrade Xilinx JTAG cable and Xilinx ISE software
    • 5.1   Connecting Xilinx JTAG cable to COMBO6X board
    • 5.2   Upgrading firmware using Xilinx ISE
    • 5.3   Post installation steps
  • 6   Contacting the Liberouter team
    • 6.1   Mailing lists
    • 6.2   Submitting problem reports
    • 6.3   Other COMBO6 based projects

1   Introduction

This package is determined to install necessary tools for PCI bridge firmware upgrade on COMBO6X card with COMBO-4SFPRO add-on card.

1.1   Obtaining pci-bridge package

Test package is available via WWW download page:

http://www.liberouter.org/clients

You can also check latest news at our web pages:

http://www.liberouter.org/

1.2   Package structure

The pci-bridge package contains Linux kernel drivers, all necessary libraries (libcombo, libcommlbr, libtap) and tools svf(1) and csid(1). The package also contains installation system and README file.

/firmware   - firmware for PCI bridge in current stable version >= 04.05
/base       - source codes for software tools and drivers
   /mk                       - build system
   /sys_sw/drivers/linux     - Linux kernel drivers
   /sys_sw/hwtools/csid      - identification tool for COMBO card family
   /sys_sw/lib*              - libraries necessary for tools
   /sys_sw/svf               - tool for PCI bridge firmware upgrade
README      - this file

2   Installation instructions

2.1   Host computer

The Combo hardware has been designed to work on any PC-AT compatible computer running GNU/Linux or other "Unix-like" operating system. It has been tested only on computers that use the x86 family of processors. The hardware requirements necessary to run the SVF tool are:

  • Intel Pentium or equivalent x86 based PC
  • COMBO6X - 64-bit/66 MHz PCI bus

The "sandwich" consisting of COMBO6(X) and an interface card COMBO-4SFPRO fits into one PCI slot.

2.2   Supported hardware

Mother card Add-on card Description
COMBO6X   a PCI-X (64/66) card with XILINX FPGAs
COMBO-4SFPRO add-on card with four SFP cages for GE interfaces
NOTE:more information about Combo cards you can get on http://www.liberouter.org/hardware.php

2.3   Supported software

Software and drivers works on GNU/Linux OS with 2.4 and 2.6 kernels. The software has been tested on computers running Red Hat Enterprise Linux.

2.4   Installing COMBO6X cards

After plugging Combo card into your PCI slot, you should test connection between the card and your PC. We use lspci(8) utility for this purpose. lspci(8) is a utility for displaying information about all PCI buses in the system and all devices connected to them. For correct recognition of the Combo card you need update PCI ID Database used by lspci(8) or download pciutils-2.2.2 (program collection containing lspci(8)) or later. If the lspci(8) output contains the following line, your COMBO6X card is connected properly.

$ lspci -d 18ec:
03:01.0 Ethernet controller: Cesnet, z.s.p.o. COMBO6X (rev 01)
NOTE:Numbers at the beginning of the line can be different.

2.5   Building package

Compilation and installation process is covered by the build system located in the /base/mk directory. For more details about required tools necessary to build package see base/mk/REQUIREMENTS file.

To build kernel drivers needed by svf you will need fully configured Linux kernel source tree. More installation and compilation details about kernel drivers are available in the README file inside base/sys_sw/drivers/linux directory.

pkgtool(1) helps to build, install and uninstall a distribution package.

$ tar -xzvf pci-bridge-XX.YY.ZZ.tgz
$ cd pci-bridge-XX.YY.ZZ/base
$ ./pkgtool --build
NOTE:Replace XX.YY.ZZ with the chosen package version number such as 1.0.0.

Optionally you can define installation directory with the --prefix=path option. The --prefix option takes effect only during building package. If no prefix is set then /usr/local path is used.

$ ./pkgtool --build --prefix=/usr/local/pci-bridge

2.6   Installing package

All these steps require root privileges. The tools and kernel drivers will be installed to the installation directory (by default /usr/local).

# ./pkgtool --install

After successful installation some post-installation steps are required:

  1. Set PATH variable to point to the installed tools directory -> ${PREFIX}/bin and add it to ~/.bash_profile for loading at login

    $ echo "export PATH=$PATH:/usr/local/bin" >> ~/.bash_profile
    
  2. Set /etc/ld.so.conf to point to the library directory -> ${PREFIX}/lib

    create file liberouter.conf with content /usr/local/lib (by default or with {PREFIX}/lib if you have used option --prefix during building process) in /etc/ld.so.conf.d directory:

    # echo "/usr/local/lib" > /etc/ld.so.conf.d/liberouter.conf
    # ldconfig
    
  3. Set /etc/man.config file to point to the Liberouter's man pages -> ${PREFIX}/man

Add MANPATH /usr/local/man entry to the /etc/man.config file.
  1. If you want to load kernel modules when the PC starts up, add following lines to the /etc/modules file.

    For COMBO6X card:

    combo6core
    combo6x
    
    NOTE:In some distribution (Red Hat Enterprise Linux 4) you have to use file /etc/rc.modules in place of /etc/modules file. This file contains commands to load kernel modules at boot time (it is actually script so it must be executable file), so such file for COMBO6X card will look like this:
    #!/bin/sh
    /sbin/modprobe combo6core
    /sbin/modprobe combo6x
    

    Don't forget to set the /etc/rc.modules as executable:

    # chmod +x /etc/rc.modules
    
  2. The driver needs device files in the /dev/combosix/ directory. To create them, run the combo6devices script in the base/sys_sw/drivers/linux directory in package tree:

    # ./combo6devices
    

    To run as a non-privileged user set r/w permissions on /dev/combosix/0 device.

    You have to change the permissions of the device file so that the non-privileged user that will execute the SVF tool has read-write access. For example, you can create a new group, say combo-rw, add that user to this group and change the file group ownership and permissions as follows:

    # chgrp combo-rw /dev/combosix/0
    # chmod 664 /dev/combosix/0
    

    Using recent 2.6 kernels it might be wanted to use the sysfs/udev mechanisms to create device files. For more details about udev see http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev.html

    Example udev rules are placed in the package in sys_sw/drivers/linux/combo6.udev.rules file. If you didn't use option --udev during package installation, copy this file to /etc/udev/rules.d directory and modify it (change group or permissions). Following line will create combo6 device files required by the package:

    KERNEL=="combosix[0-9]*",       NAME="combosix/%n",     GROUP="combo-rw"

    NOTE:Depending on your version of udevd you must adjust syntax of your udev rule file. Older versions of udevd require '=' character to match KERNEL key e.g. KERNEL="combosix[0-9]*". The newer one require '==' characters to match KERNEL key e.g. KERNEL=="combosix[0-9]*".

2.7   Upgrading package

If you have used some previous version of this package, please follow all building and installing steps described above. Post-installation steps can be skipped if you performed these steps during previous installation (and an installation path was the same!).

2.8   Uninstalling package

It is not possible (for safety reasons) to automatically uninstall pci-bridge package. pkgtool(1) will only show directories where the tools are placed. By default the pci-bridge package uses following files (or directories):

/etc/liberouter/jtag_combo6x.conf
/etc/liberouter/jtag_parport.conf
/usr/local/bin/csid
/usr/local/bin/svf
/usr/local/etc/jtag_combo6x.conf
/usr/local/etc/jtag_parport.conf
/usr/local/include/combosix.h
/usr/local/include/nbqueue.h
/usr/local/include/debug.h
/usr/local/lib/libcombo*
/usr/local/lib/libtap*
/usr/local/lib/libcommlbr*
/usr/local/man/man1/csid.1
/usr/local/man/man1/svf.1
/usr/local/mcs/chips/combo6x/pci_contr/04_05/combo6x.svf
/usr/local/mcs/chips/combo6x/pci_contr/04_05/xc18v04.mcs
/usr/local/mcs/chips/combo6x/pci_contr/04_05/xc18v512.mcs
WARNING:Some of the directories may be shared with other applications. Removing these directories may break your system.

3   PCI bridge firmware upgrade guide

This section is a guide for upgrading PCI bridge firmware on COMBO6X cards. Old firmware can have compatibility issues with newer designs. It is recommended to keep the firmware up-to-date. The PCI bridge firmware version can be viewed using csid -s command on Liberouter's workstations and should look like this:

PCI brver: c610.04.05 (2007/03/29 19:01)

The version is specified by two numbers (e.g. 04.05). The new firmware can be found after default package installation at ${PREFIX}/mcs/chips/combo6x/pci_contr/. The firmware consists of these files (example for 04_05)::

combo6x.svf   xc18v04.mcs   xc18v512.mcs

There are two possibilities of upgrade depending on the type of the programming cable. The SVF files are used if upgrading using Liberouter's "programming shortcut cable" and the SVF tool. The MCS files are used if upgrading using Xilinx JTAG cable and Xilinx ISE software.

3.1   Requirements on jumper settings

Both upgrade options require these jumper settings on COMBO6X board for correct functionality:

jumper  J1 - short
jumpers J0, J2, J3 - open

3.2   Important connectors and jumpers on COMBO6X card

Connectors JP4 and JP7 are in parallel connection. They share the JTAG signals TDO, TDI, TMS, TCK and TMSP.

+---------------------------------------------------+
| COMBO6X board                  JP5                |
|                                +-+                |
|                                | |                |
|                       J0-3     | |                |
|                       |||||    | |                |
|                  JP4           | |    JP7         |
|                  +-------+     +-+    +----+      |
|                  +-------+            +----+      |
+---+                    +--------------------------+
    |     PCI (64/66)    |
    +--------------------+

4   Upgrading with Liberouter's "programming shortcut cable" and SVF tool

For this upgrade you will need Liberouter's shortcut cable and installed the Liberouter's PCI bridge package which contains the SVF tool.

Shortcut cable connecting connector JP5 and connector JP7 must be present on COMBO6X board as described by the pictures below.

 JP5
+---+
| : |
|IO29 -TDO----------------+
|IO33 ----TDI-------------|-+
|IO36 -------TCK--------+ | |
|IO40 ----------TMSP--+ | | |
| : |                 | | | |
+---+             +-- | | | |  ----+
              JP7 | . 4 6 8 10 . . |
                  | . . . . .  . . |
                  +----------------+

4.1   SVF tool

SVF tool is a part of all Liberouter installation packages. It is used for programming PROMs which store configuration of Xilinx FPGA's on COMBO6X cards. The programming data is read from Xilinx SVF file generated by Xilinx iMPACT Software.

Program is indeed a file parser and interpreter. It uses the libtap library to communicate with TAP controller in JTAG compliant devices. SVF file contains commands which specify the programming sequence and also the data, which should be shifted into the JTAG device.

SVF program has one necessary parameter - the input file. All other parameters are optional. The default device with TAP connector is COMBO6X device /dev/combosix/0. The program needs also access to devices in /dev directory. This may require root privileges.

Examples of usage

% cd /usr/local/mcs/chips/combo6x/pci_contr/04_05
  .. change to this directory before you use examples below

% svf -f combo6x.svf
  .. upgrade firmware with Liberouter's shortcut cable on card /dev/combosix/0

% svf -d /dev/combosix/1 -f combo6x.svf
  .. upgrade firmware with Liberouter's shortcut cable on card /dev/combosix/0

The programming may take several minutes to finish. The tool shows a progress bar for information about the programming progress. Don't kill the tool while programming is in progress. The PCI bridge firmware must be correct otherwise the card won't be detected by the computer at next start (no COMBO card in lspci listing).

4.2   Post installation steps

It is necessary to shutdown and power on the machine after the upgrade of the PCI bridge firmware (reboot is not enough). As root run the following command:

# shutdown -h now

4.3   What to do when the card "dies" after corrupted programming

If the programming using Liberouter's shortcut cable failed and you restarted your workstation your COMBO6X card is not detected by the system. You can't use SVF tool, because it uses the card for access to the programming chip. You have two possibilities to enable your card:

  1. Program the PCI bridge firmware using Xilinx JTAG cable and Xilinx ISE (see the next section).
  2. Build a custom LPT cable (Xilinx DCL5 Type3) and use SVF tool

For more information about construction of a Xilinx DCL5 Type3 cable see

http://wiki.openwrt.org/OpenWrtDocs/Customizing/Hardware/JTAG_Cable

The cable should be connected to the board according to this scheme:

Xilinx DCL5 Type3                             JP4
     +---+                                   +---+
     |    \                                  | : |
     |     ----- TMS ------------------TMSP--- 7 |
     |     ----- TCK ------------------------- 8 |
     |     ----- TDI ------------------------- 9 |
     |     ----- TDO ------------------------- 10|
     |     ----- GND ------------------------- 12|
     |    /                                  | : |
     +---+                                   +---+

Then use SVF tool with the parallel port connected to COMBO6X board (/dev/parport/0 - LPT1 0x378; /dev/parport/1 - LPT2 0x278):

# svf -d /dev/parport/0 -f combo6x.svf -v

After the programming shutdown and restart the computer and the card should be detected.

5   Upgrade Xilinx JTAG cable and Xilinx ISE software

For this upgrade you will need a Xilinx JTAG cable ("Platform Cable USB" or "Parallel Cable IV") and Xilinx ISE software installed on your workstation.

5.1   Connecting Xilinx JTAG cable to COMBO6X board

The JTAG signals are available at connector JP7. The signals consist of TMSP, TCK, TDI and TDO pins. To these pins attach the Xilinx programming cable. For connection a "High-Performance Ribbon Cable" (with 14pin connector at both sides) or a "Flying Wire Adapter" can be used.

5.2   Upgrading firmware using Xilinx ISE

  1. Start Xilinx
  2. Select Accessories -> Impact
  3. Discard project chooser
  4. Choose Boundary-scan mode
  5. Choose File -> Initialize Chain

The software should now auto-detect all available chips on the JTAG chain. For programming options use right click on desired chips (xc18v04 and xc18v512):

  1. Choose Assign New Configuration - select MCS file according to the selected chip
  2. Choose Program - program assigned configuration to the chip

5.3   Post installation steps

It is necessary to shutdown and power on the machine after the upgrade of the PCI bridge firmware (reboot is not enough). As root run the following command:

# shutdown -h now

6   Contacting the Liberouter team

6.1   Mailing lists

For any questions or general technical support issues, please send mail to the combo6@liberouter.org general mailing list.

6.2   Submitting problem reports

Suggestions, bug reports and contributions of code are always valued. Please do not hesitate to report any problems you may find. Bug reports with attached fixes are of course even more welcome. You can use our mailing list to report bugs or (preferably) our bug tracking system at http://www.liberouter.org/bugtrack/

6.3   Other COMBO6 based projects

Check our web pages for information about other projects, e.g. HW accelerated flow exporter or intrusion detection system.

http://www.liberouter.org/
Main Page About Liberouter Team Mailing list SVN Contacts