RSVP-TE daemon for DiffServ over MPLS under Linux

I have added the patch for the 2.4.20-2.4.22 kernel contributed by Roberto Albanese to the website. It can be found here.

I have fixed some small glitches in the installation guide. The web site got updated too, mainly the the FAQ section. Concerning the download section, eh well, it is gone. Always use the files from the latest release together with the subsequent updates (if any). Finally as reported on the mailing list, the mailing list search engine is up-to-date and running again.

Finally, rsvpd.0.70-rc2 is ready. Pay attention to the updated installation instructions. At lot of reports are due to wrong compilations of the daemon. Changes in this version: upgrade to MPLS Linux 1.172, added source address matching and fixed destination port matching in tunnel. Improved Intserv support and mem leak fixes from Takao (thank!). I have re-added the label.conf file, improved error messages when the label.conf file is invalid. Improved Makefiles, edit the top-level Makefile if you want to disable MPLS or Linux (the latter is untested). Better support for long LSP specifications in rtest2 (cheers Panos). Note: installation procedure changes are always dangerous, however due to high number of user errors I will try to simplify things, this is gradual process, pay attention and report any issues. Don't forget to apply the dsmark patch. This is the second version of iptables-1.2.4 (just a small Makefile change + minor fix). There are no changes to the configuration files. tools/rstat is broken at the moment.

I added my presentation from Linux Kongress 2002 and a paper describing the features, components and architecture to the FAQ section (last question).

RSVP daemon v0.70-rc1: L-LSP support (thanks to Daniela and Giovanna!), an updated iptables library, a new kernel patch (MPLS 1.160) (thanks James), DiffServ over IP and MPLS (apply the dsmark patch). Fixes: kernel crashes, allocating labels on not MPLS enabled interfaces. rtest2 is now a lot more verbose when you give it an illigal configuration file. Oh yes, the rtest2 configuration has yet another field i.e. PHBID to support L-LSPs. See the updated rtest2 example. I also updated the the installation guide (with the help of Bin Zhou). Note the dsmark patch is highly recommended. I still recommend iptables-1.2.4 though succes stories with iptables-1.2.7a are welcome. There is also code to support MPLS_SPECS though the support is not enabled by default (see the Makefiles), support for CoS_TSPEC has been removed. The new ds_config is included and DiffServ over IP and MPLS is enabled by default.

A new ds_config: fixes the problem where either IP or MPLS was classified but not both. See the 17/07/02 news item for the configuration of the script.

(link removed) To run ds_configNEW in mpls+ip mode, just edit it and set ENABLE_MPLS_IP = true. Make sure you patched your kernel with the dsmpls+ip patch.
Fixes:

Mpls + ip support (if you install the dsmpls+ip patch) The OR queue now works (at least in mpls+ip mode) Traffic marked with an unsupported DSCP is now classified in the BE queue, instead of the invisible default queue. Some other bugfixes (e.g. rtnetlink error) Additional functionalities (try "./ds_configNEW -s eth1 BE") You don't need to patch iproute anymore. If you already patched iproute, you can leave it this way, no harm done.

RSVP daemon v0.61-rc3: segfault fix for rapirecv_auto (rapi_lib.c), rapirecv_auto should be a bit more robust and can be started before the daemon, send a Path Error when the next hop for a Label Request is an interface that is not label enabled, more relaxed handling of ER-hops (pop on hop address == outgoing interface address), protection switch support (triggered by RTM_NEWLINK & IFF_DOWN), from Takao (thanks man!): upcall fixes in rsvp_api.c, various memory leak fixes and dont send an explicit route on Path Tear messages.

RSVP daemon v0.61-rc2: fix for virtual tunnels name clash, buffer overrun fix for tools/rstat (was already done for labeltest/rstat). Use iptables-1.2.4 with our own DSCP matching code because the iptables-1.2.6a matching does not seem to work. Remember that the traffic should be marked before it enters the ingress of the LSP. Marking can be done with iptables -A OUTPUT -t mangle ... -j FTOS --set-ftos (dscpvalue*4) (iptables 1.2.4) or iptables -A OUTPUT -t mangle ... -j DSCP --set-dscp dscpvalue (iptables-1.2.6a).

RSVP daemon v0.61-rc1: fixed a regression in rapirecv_auto and optimised cleanstate a bit. Upgrade if you are using rapirecv_auto (or are unable to use the version of rsvpd-0.60). rsvpd-0.61 will be a bug fixing only release. Expect major functionality improvements for rsvpd-0.70. I also fixed the link to the Installation Guide and I updated the rtest2 example.

I tagged rsvpd.0.60-rc9 as the RSVP daemon v0.60. The web site is updated. Support for the old daemon will be limited and it is recommended to upgrade. Download the daemon and the kernel, iptables and iproute2 patches at the Download section

RSVP daemon v0.60-rc9: A couple of compile fixes for compiling without MPLS and EX_ROUTE suport. I included a label.conf example. Minor update. No need to upgrade if you are already running the 0.60 daemon.

RSVP daemon v0.60-rc8: rapi_lib and rsvp_api fixes. A fix for a SE reservation tear down segfault. Also a number of fixes from Takao: rapi_lib upcall fix, label table overflow fix, proper label allocation error signalling, close the MPLS ioctl fd. I also increased the maximum number of label interfaces to 8 and included a hack around the RedHat IPv6 compilation problems. And finally there is a is_config script which enables IntServ reservations on the interfaces (again thanks to Takao).

Daniela from Ericsson and I made a installation guide for rsvpd-0.60. Make sure to check it out if you have installation problems. Please report all issues to the mailing list.

RSVP daemon v0.60-rc7: This one fixes a rapirecv_auto crash, an unmapping bug, some DSCP matching fixes and possibly a RedHat compile fix. SE style reservations should be stable now. ds_config is back better than ever with EF, AF11-AF32 and BE support. mplsadm stand-alone is now part of the distribution (labeltest/). New is also clearstate a small script that clears all /proc and iptables state. There's also a new iptables patch. This patch adds DSCP matching (from CVS) and MPLS marking. Make sure to add both dscp and DSCP when doing the make patch-o-matic.

A new iptables patch against version 1.2.6a that includes DSCP matching. This patch adds MPLS mark support and fixes the problem with tfos matching in version 1.2.6a (the ftos.patch.config.in in the original 1.2.6a had a wrong indentation).

RSVP daemon v0.60-rc6: This one fixes: ingress crash, mapping traffic on LSP without an EXP, clear all state when daemon exits, rapi_release() and unmapped traffic.

A tc patch for kernel 2.4.17 used in RSVPd-0.60. See -rc1 anouncement why patching tc is is not a priority at the moment.
A small RSVP-0.60-rcx FAQ has been posted to the mailing list.

Today's release fixes some remaining netlink problems, there is an EXP fix in tunnel, the daemon now removes the label state properly when exiting and most of the compile time warnings are removed. The daemon now reports the proper version number. mplsadm standalone utility is now part of the distribution (you can find it in /labeltest). Pm_Parse.h is back in and the rapi_release call is properly removed this time. RSVP daemon v0.60-rc5(full tree). As before, use the kernel patch and the iptables patch from -rc1.

Release early, release often as ESR would say. Anyway RSVP daemon v0.60-rc4 (-rc3 was a limited release) fixes compiling without MPLS support, introduces a new netlink library, fixes two SEGFAULTS and should work better when embedding tunnel.c. Thanks to the testers for the feedback.

Already a new pre-release version: RSVP daemon v0.60-rc2. This package does contain the top-level directory.

Pre-release version of the RSVP daemon v0.60-rc1. Use the new kernel patch against kernel 2.4.17. There's also a new iptables patch against version 1.2.4 which contains the MPLS patches and the DSCP patches. New in this release are SE style reservations, faster rerouting (if enabled), tunnel now supports port matching and does a much better job at reporting and of course lots of bug fixes. This pre-release version lacks EXP2TCINDEX mappings. This will be fixed in the final 0.60 version. Note: the rtest2 configuration files have an additional field at the back, indicating whether or not SE style reservation should be used.

Make sure to have a look at line 6 of ds_config if you are using the version which is part of the distribution (and not the version of the web site)

We shipped a wrong version of ipt.c replace the file in rsvpd/linux with this one. We will release a new daemon as soon as we resolve some IPv6 compilation bugs on selected platforms.

This daemon in combination with the necessary kernel and user space patches supports to set-up of EXP inferred Label Switched Paths (E-LSP). The daemon is based upon the Nistswich version 2.0 daemon for Free BSD by USC and a port of an Intserv RSVP daemon to Linux by Alexey Kuznetsov. Note that both daemons are based on the same code base (ISI RSVP implementation) but they forked a while ago. This efforts combines the daemons again so that the MPLS support found in the Nistswitch version is now available on Linux. Moreover support for DiffServ over MPLS (DS/MPLS) is added.

The MPLS Linux kernel code is based upon mpls-linux by James R. Leu. Our release adds DiffServ over MPLS support, the use of multiple routing tables (and thus multiple LIBs) and LSP byte and packet counters.

In order to support DS/MPLS various other components were patched including: iptables (DSCP based matching), ip (mpls protocol), mplsadm (multiple tables, exp and exp mask/shift support), the Linux kernel (kernel space support for DSCP filtering, MPLS patch).

Note the prime candidates for using this software are people looking to experiment with RSVP-TE as a signalling deamon for Diffserv over MPLS under Linux.

• LSP set-up
  • Constraint routed LSP set up
  • DiffServ over MPLS LSP set up (E-LSP and L-LSP)
  • LSP with Intserv style reservations (use is_config)
• Mapping traffic to LSPs based on:
  • Protocol, port
  • Destination Prefix
  • DSCP
• Miscellaneous
  • LSP statistics
  • LSP "traceroute"

We will set-up an LSP. Map icmp packets going to the destination of the LSP to the LSP. We will verify whether the packets are send over the LSP by checking the LSP counters.
Consider the following network:

          1.1     1.2     2.1    2.2   
   --------        -------         -------- 
  | Ingres |------| CORE  |-------| Egress |
   --------        -------         -------- 
         eth1    eth1   eth2     eth1        

Start the daemons on all machines (ingress-core-egress):

	./rsvpd -D

Ingress send PATH messages to destination (type on rsvpd console):

	T1> dest lsp tcp 10.0.2.2/12

       	T1> sender 10.0.1.1/12

Egress reply with RESV messages (type on rsvpd console):

	T1> dest lsp tcp 10.0.2.2/12

	T1> reserve 10.0.2.2 ff 10.0.1.1/12

Now continue on the ingress (not on the daemon console!)

./tunnel -L -c

LSPID          Destination (type label/exp/iface)     viface  Packets     Bytes
12                10.0.2.2 ( gen 21650/  0/ eth1)     T21650  0           0


./tunnel -m -p icmp -d 10.0.2.2/32 -l 12

Map icmp packets that go to 10.0.2.2 on LSP with LSPID 12.

ping 10.0.2.2
PING 10.0.2.2 (10.0.2.2): 56 data bytes
64 bytes from 10.0.2.2: icmp_seq=0 ttl=254 time=0.3 ms
64 bytes from 10.0.2.2: icmp_seq=1 ttl=254 time=0.2 ms
64 bytes from 10.0.2.2: icmp_seq=2 ttl=254 time=0.2 ms
64 bytes from 10.0.2.2: icmp_seq=3 ttl=254 time=0.2 ms

./tunnel -L -c

LSPID          Destination (type label/exp/iface)     viface  Packets     Bytes
12                10.0.2.2 ( gen 21650/  0/ eth1)     T21650        4       336
  |            Destination DSCP  Proto    Packets     Bytes   Packets     Bytes
  \->          10.0.0.0/12   BE   icmp          4       336         4       336
  

Great four packets sent over the LSP!

./tunnel -u -p icmp -l 12

Unmap the icmp packets.

./tunnel -m -a -d 10.0.2.2/32 -l12/3

This will map all packets destined to 10.0.2.2 on the LSP with LSPID 12, EXP field in the shim header of these packets will be 3. Note when using -a RSVP signalling messages will not be sent over the LSP which is a good thing :)

./tunnel -u -a -d 10.0.2.2/32 -l12/3

When unmaping the traffic you have to remember what exactly you have mapped (tunnel -L does not show this). See the debugging section.

./tunnel -m -x 0x2 -d 10.0.2.2/32 -l12/1

Map traffic which is destined for 10.0.2.2 and is marked with DSCP 0x2 on the LSP with LSPID 12. These packets will have EXP=1.

The following commands might come in handy when you want to find out how the system works or if you want to debug it:

• #less /var/log/rsvpd.log
• #tunnel -L -c
• #tunnel -t -l lspid
• #cat /proc/net/mpls*
• #iptables -L -t mangle
• #ip rule show
• #ip route show table tableid(tableid value obtained from ip rule show)
• #tc -s qdisc ls devx
• The Ethereal software packet analyser supports IP MPLS packets with shim headers and RSVP signalling messages.

Compiling the daemon fails

I get weird errors like: ifindex is not mapped to a vif

Make sure the correct mpls.h is in /usr/include/linux (by copying it or by linking /usr/src/linux-2.4.19/include/linux to /usr/include/linux). If you are running RedHat you might need to look at rsvp_socks.h. If you are using the old daemon make sure that you apply the new ipt.c (see news section).

When I set-up an LSP nothing happens!

You have to map traffic to your LSP. Use the tunnel tool in $(RSVPDIR)/labeltest.

How can I test whether or not packets are send over the LSP?

Check the LSP counters (tunnel -L -c) or use Ethereal.

How do I set-up an CR-LSP?

How do I set-up a lot of LSPs automatically?

You have to use the tools rtest2 -f filename (ingress, example) and rapirecv_auto (egress). Make sure that the RSVP daemon is running on all the nodes.

I want to rate limit my LSP. How do I do that?

There are several ways to do this:

• you can use the above method to set-up a mapping from the label to a tc_index and use the diffserv-like configuration scripts.
• use a "u32-filter" or "fw-filter" instead of the tc_index filter
• add policing to a filter
• use an ingress queue

See also: the advanced routing howto

[NOTE: this section needs some more input, if you have some nice scripts, please post them to the mailinglist]

How do I mark packets with a DSCP?

Use the iptables DSCP target. Make sure you compiled the kernel with 'DSCP target' enabled. E.g. to set the DSCP to EF (=0x2e) .

Example:

  iptables -A OUTPUT -t mangle ... -j DSCP --set-dscp 0x2e

Don't use the dscp match (-m), this is used to match the dscp and not for setting it.

I want more information about the architecture/source. Where can I get it?

Here is my presentation (fully animated, not animated) from Linux Kongress 2002 and a paper describing the features, components and architecture of the daemon. More information is also available in the docs/ directory of the source tree. Also remark that the amount of code that we wrote is limited. Most of the work went in adapting, merging, patching, hacking and integrating existing components. This means that you have to rely on the documentation of these components (just like we had to do). And remember: "Use the source Luke".

I have looked at ds_config and have some questions

I don't understand the label.conf file

You know the rap: do not use this on production machines or machines with important data. Use at own risk ...
Known bugs:

• Setting up a CR-LSP with hops with masks != 32 fails. Apparantly dst_route() and friends in libroute.c do not support netmasks.
• No signalling support for (patches are very welcome :)
  • Resource affinities
  • The RSVP hello protocol
  • ...
• Capabilities of the original daemons like IPv6, BSD, multicast and Intserv reservation support were not tested after the merging and might very well be broken.
• Patches adding new functionality (more RSVP-TE signalling support (HELLO, RA), additional PHB configuration scripts ...) are very welcome and will be applied in subsequent versions (if they pass our "QA" :)
• Compiling on a RedHat7 machine does not seem to work. Upgrading to RH7.1 does seem to do the trick.
• Lots of other bugs I guess

Please use the public mailinglist for discussions, to give us feedback and maybe even some scripts or patches.
Subscription info and archives can be found here. The archives can be searched here
Please don't send any dsmpls related questions to me personally use the mailing list. Before you send a question consult the mailing list archives. This can save me some valuable time and you get your answer immediatly. Hope to hear from you all soon!

MPLS IETF working group

MPLS page at Source Forge

NistSwich Page

Software packages at freshmeat

The Linux advanced routing HOWTO

Tequila Home page (theoretical background on enabling QoS in Internets of large scale)

The three authors are working (doing a PHD) at IBCN the Broadband communication networks group which is headed by Piet Demeester and is a part of the Department of Information Technology (INTEC). The IBCN testlab is called Atlantis. INTEC is part of the Faculty of Applied Science at the Gent University. The IBCN research activity is also part of the INTEC department of the Interuniversity Microelectronics Centre (IMEC).

Pim Van Heuven RSVP daemon (porting, patching, enhancements), tunnel, ip rule and routing table interface, documentation, web site. (Please don't send me private messsages which relate only to this project, use the mailing list.)

Steven Van den Berghe MPLS kernel space, multiple LIB design

Jan Coppens TC and PHB

Tom Aernoudt iptables interface, MPLS kernel space bug hunter