tc class ceil inheritance

Question

Using traffic shaping (tc) on Linux, trying to limit throughput in 2 levels.

Class 1:20 inherits from 1:10 but traffic restriction does not apply. The traffic is being limiting using the limit of the last class.

class htb 1:1 root rate 1Gbit ceil 1Gbit burst 15125b cburst 1375b                                      
class htb 1:10 parent 1:1 rate 4Mbit ceil 5Mbit burst 15Kb cburst 1600b                               
class htb 1:20 parent 1:10 leaf 200: prio 0 rate 9Mbit ceil 10Mbit burst 15Kb cburst 1600b               
class htb 1:30 parent 1:10 leaf 300: prio 0 rate 9Mbit ceil 10Mbit burst 15Kb cburst 1600b    

I'm filtering per station using the following filter:

filter parent 1: protocol all pref 5 u32            
filter parent 1: protocol all pref 5 u32 fh 800: ht divisor 1                                           
filter parent 1: protocol all pref 5 u32 fh 800::20 order 32 key ht 800 bkt 0 flowid 1:20               
  match 00000800/0000ffff at -4                     
  match 5be42dc1/ffffffff at -12                    
  match 00009cd3/0000ffff at -16  

Using iperf TCP to test:

[ ID] Interval       Transfer     Bandwidth                                                            
[  3]  0.0- 1.0 sec  1.25 MBytes  10.5 Mbits/sec                                                       
[  3]  1.0- 2.0 sec  1.12 MBytes  9.44 Mbits/sec                                                       
[  3]  2.0- 3.0 sec  1.00 MBytes  8.39 Mbits/sec

All traffic is passing though the 2 classes:

class htb 1:10 parent 1:1 rate 4Mbit ceil 5Mbit linklayer ethernet burst 15Kb/1 mpu 0b overhead 0b cburst 1600b/1 mpu 0b overhead 0b level 6                                                                    
 Sent 31091090 bytes 20545 pkt (dropped 0, overlimits 0 requeues 0)                                                                                                                                             
 rate 0bit 0pps backlog 0b 0p requeues 0                                                                                                                                                                        
 lended: 0 borrowed: 0 giants: 0                                                                                                                                                                                
 tokens: -135041557 ctokens: -86778070                                                                                                                                                                          

class htb 1:20 parent 1:10 leaf 200: prio 0 quantum 112500 rate 9Mbit ceil 10Mbit linklayer ethernet burst 15Kb/1 mpu 0b overhead 0b cburst 1600b/1 mpu 0b overhead 0b level 0                                  
 Sent 31091090 bytes 20545 pkt (dropped 17, overlimits 0 requeues 0)                                                                                                                                            
 rate 0bit 0pps backlog 0b 101p requeues 0          
 lended: 20545 borrowed: 0 giants: 0                
 tokens: -20701 ctokens: 1075

Answer 1

If you want to limit traffic per ip then you can use this method (i`m using it to limit virtual servers bandwidth virtualized with openvz, but you can customize it by your requirements):

tc qdisc del dev venet0 root
tc qdisc add dev venet0 root handle 1: htb default 10
tc class add dev venet0 parent 1: classid 1:1 htb rate 1000mbit burst 15k
tc class add dev venet0 parent 1:1 classid 1:10 htb rate 50mbit ceil 50mbit burst 15k

tc qdisc add dev venet0 parent 1:10 handle 10: sfq perturb 10

interface=eth0
tc qdisc del dev $interface root
tc qdisc add dev $interface root handle 1: htb default 10
tc class add dev $interface parent 1: classid 1:1 htb rate 1000mbit burst 15k
tc class add dev $interface parent 1:1 classid 1:110 htb rate 50mbit ceil 50mbit burst 15k

tc qdisc add dev $interface parent 1:110 handle 110: sfq perturb 10

and then for each ipv4 address:

tc filter add dev venet0 protocol ip parent 1:0 prio 1 u32 match ip dst $IPADDRESS flowid 1:10
tc filter add dev $interface protocol ip parent 1:0 prio 1 u32 match ip src $IPADDRESS flowid 1:110

For each ipv6:

tc filter add dev venet0 protocol ipv6 parent 1:0 prio 2 u32 match ip6 dst $ipv6/128 flowid 1:10
tc filter add dev $interface protocol ipv6 parent 1:0 prio 2 u32 match ip6 src $ipv6/128 flowid 1:110

DS48236.vpsnet.com:~$ tc qdisc show

qdisc htb 1: dev eth0 root refcnt 2 r2q 10 default 10 direct_packets_stat 2630716
qdisc sfq 110: dev eth0 parent 1:110 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 120: dev eth0 parent 1:120 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 130: dev eth0 parent 1:130 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 140: dev eth0 parent 1:140 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 150: dev eth0 parent 1:150 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 160: dev eth0 parent 1:160 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 170: dev eth0 parent 1:170 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 180: dev eth0 parent 1:180 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 190: dev eth0 parent 1:190 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 1110: dev eth0 parent 1:1110 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 1111: dev eth0 parent 1:1111 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 1100: dev eth0 parent 1:1100 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc htb 1: dev venet0 root refcnt 2 r2q 10 default 10 direct_packets_stat 0
qdisc sfq 10: dev venet0 parent 1:10 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 20: dev venet0 parent 1:20 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 30: dev venet0 parent 1:30 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 40: dev venet0 parent 1:40 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 50: dev venet0 parent 1:50 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 60: dev venet0 parent 1:60 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 70: dev venet0 parent 1:70 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 80: dev venet0 parent 1:80 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 90: dev venet0 parent 1:90 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 110: dev venet0 parent 1:110 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 111: dev venet0 parent 1:111 limit 127p quantum 1514b divisor 1024 perturb 10sec
qdisc sfq 100: dev venet0 parent 1:100 limit 127p quantum 1514b divisor 1024 perturb 10sec
DS48236.vpsnet.com:~$

If you want to limit traffic per one interface then you need to modify:

For each ipv4 address:

tc filter add dev venet0 protocol ip parent 1:0 prio 1 u32 match ip dst $IPADDRESS flowid 1:10
tc filter add dev $interface protocol ip parent 1:0 prio 1 u32 match ip src $IPADDRESS flowid 1:110

and do not create qdisc and class for venet0

So with one interface it will look something like:

interface=eth0
tc qdisc del dev $interface root
tc qdisc add dev $interface root handle 1: htb default 10
tc class add dev $interface parent 1: classid 1:1 htb rate 1000mbit burst 15k
tc class add dev $interface parent 1:1 classid 1:110 htb rate 50mbit ceil 50mbit burst 15k

tc qdisc add dev $interface parent 1:110 handle 110: sfq perturb 10

For each ip:

tc filter add dev $interface protocol ip parent 1:0 prio 1 u32 match ip dst $IPADDRESS flowid 1:110
tc filter add dev $interface protocol ip parent 1:0 prio 1 u32 match ip src $IPADDRESS flowid 1:110