Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Openstack, neutron, Can't ping external network

I have followed the Icehouse doc to install a 3 node environment my openstack nodes are builded as virtual machines using libvirt in CentOS

http://docs.openstack.org/icehouse/install-guide/install/yum/content/basics-networking-neutron.html

-controller-node : 10.0.0.11 (management network) 
-network-node : 10.0.0.21 (management network), 10.0.1.21 (data network) 
-compute-node : 10.0.0.31 (management network), 10.0.1.31 (data network) 
-external network : 192.168.125.0/24 
-demo network : 172.30.1.0/24

In my network node, external NIC is configured witout IP as per the documentation.(PROMISC="yes")

DEVICE="eth1"
TYPE="Ethernet"
ONBOOT="yes"
PROMISC="yes"
BOOTPROTO="none"
HWADDR="52:54:00:D3:92:E2"
UUID="7f8a9e99-fbd0-4c59-900f-2369c9e8f780"

However, after adding networking service. I cannot ping the external network in a below stage.

http://docs.openstack.org/icehouse/install-guide/install/yum/content/neutron_initial-networks-verify.html

1) inside(tenant router gateway on network-node) -> outside(external geteway)

[root@network-node ~]# ping 192.168.125.254
PING 192.168.125.254 (192.168.125.254) 56(84) bytes of data.
From 192.168.125.54 icmp_seq=1 Destination Host Unreachable
From 192.168.125.54 icmp_seq=2 Destination Host Unreachable
From 192.168.125.54 icmp_seq=3 Destination Host Unreachable

2) outside -> inside

[root@desktop ~]# ping 192.168.125.150
PING 192.168.125.150 (192.168.125.150) 56(84) bytes of data.
--> receive no response

This is the info of my environment:

1) controller node

###The status of the port for external is "DOWN"###

[root@controller-node]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:91:4e:06 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.11/24 brd 10.0.0.255 scope global eth0
    inet6 fe80::5054:ff:fe91:4e06/64 scope link 
       valid_lft forever preferred_lft forever
[root@controller-node]# 
[root@controller-node]# neutron net-list
+--------------------------------------+----------+-------------------------------------------------------+
| id                                   | name     | subnets                                               |
+--------------------------------------+----------+-------------------------------------------------------+
| 9cebb2a6-fd73-4ef7-81d2-188652f57ecd | demo-net | c66648c9-c34b-4806-af39-3c982378a411 172.30.1.0/24    |
| e5f7b93c-475c-4c9d-95e4-8d1cf7728013 | ext-net  | a1e1fcc6-d596-4959-8923-9b46d64445af 192.168.125.0/24 |
+--------------------------------------+----------+-------------------------------------------------------+
[root@controller-node]# neutron subnet-list
+--------------------------------------+-------------+------------------+--------------------------------------------------------+
| id                                   | name        | cidr             | allocation_pools                                       |
+--------------------------------------+-------------+------------------+--------------------------------------------------------+
| a1e1fcc6-d596-4959-8923-9b46d64445af | ext-subnet  | 192.168.125.0/24 | {"start": "192.168.125.150", "end": "192.168.125.159"} |
| c66648c9-c34b-4806-af39-3c982378a411 | demo-subnet | 172.30.1.0/24    | {"start": "172.30.1.2", "end": "172.30.1.254"}         |
+--------------------------------------+-------------+------------------+--------------------------------------------------------+
[root@controller-node]# 
[root@controller-node]# neutron port-list
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                              |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+
| 9810105a-edf5-41bc-a140-81ccf71f6bc4 |      | fa:16:3e:34:fd:cb | {"subnet_id": "a1e1fcc6-d596-4959-8923-9b46d64445af", "ip_address": "192.168.125.150"} |
| 98c762ea-d7f7-4c1d-9b74-73efc9990236 |      | fa:16:3e:cb:0c:11 | {"subnet_id": "c66648c9-c34b-4806-af39-3c982378a411", "ip_address": "172.30.1.1"}      |
| f5eec840-e629-448b-ba9a-fbcd60501247 |      | fa:16:3e:ae:a6:fa | {"subnet_id": "c66648c9-c34b-4806-af39-3c982378a411", "ip_address": "172.30.1.2"}      |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+
[root@controller-node]# 
[root@controller-node]# neutron port-show 9810105a-edf5-41bc-a140-81ccf71f6bc4
+-----------------------+----------------------------------------------------------------------------------------+
| Field                 | Value                                                                                  |
+-----------------------+----------------------------------------------------------------------------------------+
| admin_state_up        | True                                                                                   |
| allowed_address_pairs |                                                                                        |
| binding:host_id       | os-network                                                                             |
| binding:profile       | {}                                                                                     |
| binding:vif_details   | {"port_filter": true, "ovs_hybrid_plug": true}                                         |
| binding:vif_type      | ovs                                                                                    |
| binding:vnic_type     | normal                                                                                 |
| device_id             | 8ae4b1fa-fb60-4690-bbe2-febbfbcf7555                                                   |
| device_owner          | network:router_gateway                                                                 |
| extra_dhcp_opts       |                                                                                        |
| fixed_ips             | {"subnet_id": "a1e1fcc6-d596-4959-8923-9b46d64445af", "ip_address": "192.168.125.150"} |
| id                    | 9810105a-edf5-41bc-a140-81ccf71f6bc4                                                   |
| mac_address           | fa:16:3e:34:fd:cb                                                                      |
| name                  |                                                                                        |
| network_id            | e5f7b93c-475c-4c9d-95e4-8d1cf7728013                                                   |
| security_groups       |                                                                                        |
| status                | DOWN                                                                                   |
| tenant_id             |                                                                                        |
+-----------------------+----------------------------------------------------------------------------------------+
[root@controller-node]# 
[root@controller-node]# neutron router-show demo-router
+-----------------------+-----------------------------------------------------------------------------+
| Field                 | Value                                                                       |
+-----------------------+-----------------------------------------------------------------------------+
| admin_state_up        | True                                                                        |
| external_gateway_info | {"network_id": "e5f7b93c-475c-4c9d-95e4-8d1cf7728013", "enable_snat": true} |
| id                    | 8ae4b1fa-fb60-4690-bbe2-febbfbcf7555                                        |
| name                  | demo-router                                                                 |
| routes                |                                                                             |
| status                | ACTIVE                                                                      |
| tenant_id             | c94f1dc5870a4d06a8b6ba947e1ac554                                            |
+-----------------------+-----------------------------------------------------------------------------+
[root@controller-node]# 
[root@controller-node]# neutron router-list
+--------------------------------------+-------------+-----------------------------------------------------------------------------+
| id                                   | name        | external_gateway_info                                                       |
+--------------------------------------+-------------+-----------------------------------------------------------------------------+
| 8ae4b1fa-fb60-4690-bbe2-febbfbcf7555 | demo-router | {"network_id": "e5f7b93c-475c-4c9d-95e4-8d1cf7728013", "enable_snat": true} |
+--------------------------------------+-------------+-----------------------------------------------------------------------------+
[root@controller-node]# 
[root@controller-node]# 

2) network node

[root@network-node ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:f6:31:07 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.21/24 brd 10.0.0.255 scope global eth0
    inet6 fe80::5054:ff:fef6:3107/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:d3:92:e2 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::5054:ff:fed3:92e2/64 scope link 
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:48:c8:65 brd ff:ff:ff:ff:ff:ff
    inet 10.0.1.21/24 brd 10.0.1.255 scope global eth2
    inet6 fe80::5054:ff:fe48:c865/64 scope link 
       valid_lft forever preferred_lft forever
5: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN 
    link/ether ea:8e:aa:ad:57:60 brd ff:ff:ff:ff:ff:ff
6: br-ex: <BROADCAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether a2:f1:0b:6b:34:4f brd ff:ff:ff:ff:ff:ff
    inet6 fe80::a0f1:bff:fe6b:344f/64 scope link 
       valid_lft forever preferred_lft forever
9: br-int: <BROADCAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 32:a4:53:15:fc:4f brd ff:ff:ff:ff:ff:ff
    inet6 fe80::30a4:53ff:fe15:fc4f/64 scope link 
       valid_lft forever preferred_lft forever
12: gre0: <NOARP> mtu 1476 qdisc noop state DOWN 
    link/gre 0.0.0.0 brd 10.0.0.31
13: gretap0: <BROADCAST,MULTICAST> mtu 1476 qdisc noop state DOWN qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
22: tun0@NONE: <POINTOPOINT,NOARP> mtu 1476 qdisc noqueue state DOWN 
    link/gre 0.0.0.0 peer 10.0.0.31
    inet 10.0.1.21 peer 10.0.1.31/32 scope global tun0
24: br-tun: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether de:a8:a4:b1:b1:46 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::dc0e:8cff:fe67:d352/64 scope link 
       valid_lft forever preferred_lft forever
[root@network-node ~]# 
[root@network-node ~]# ovs-vsctl show
23804a8f-7c89-4422-9b9f-67bf26a34c51
    Bridge br-int
        fail_mode: secure
        Port br-int
            Interface br-int
                type: internal
        Port "qr-98c762ea-d7"
            tag: 1
            Interface "qr-98c762ea-d7"
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tapf5eec840-e6"
            tag: 1
            Interface "tapf5eec840-e6"
                type: internal
    Bridge br-ex
        Port "eth1"
            Interface "eth1"
        Port br-ex
            Interface br-ex
                type: internal
        Port "qg-9810105a-ed"
            Interface "qg-9810105a-ed"
                type: internal
    Bridge br-tun
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port "gre-0a00011f"
            Interface "gre-0a00011f"
                type: gre
                options: {in_key=flow, local_ip="10.0.1.21", out_key=flow, remote_ip="10.0.1.31"}
        Port br-tun
            Interface br-tun
                type: internal
    ovs_version: "1.11.0"
[root@network-node ~]#
[root@network-node ~]# ip netns list
qdhcp-9cebb2a6-fd73-4ef7-81d2-188652f57ecd
qrouter-8ae4b1fa-fb60-4690-bbe2-febbfbcf7555
[root@network-node ~]# 
[root@network-node ~]# ip netns exec qrouter-8ae4b1fa-fb60-4690-bbe2-febbfbcf7555 route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
172.30.1.0      0.0.0.0         255.255.255.0   U     0      0        0 qr-98c762ea-d7
192.168.125.0   0.0.0.0         255.255.255.0   U     0      0        0 qg-9810105a-ed
0.0.0.0         192.168.125.254 0.0.0.0         UG    0      0        0 qg-9810105a-ed
[root@network-node ~]# 

Any hints would be much appreciated!

Cheers,

hbseo

like image 851
jazzsir Avatar asked Aug 28 '14 08:08

jazzsir


1 Answers

I know it is late... but let me try to answer. Not being able to ping the external network might be related to the virtualization and promiscuous mode but I think that step should not work. I will try to explain why. For a very good explanation on how the networking is setup you can check out this presentation: https://www.hastexo.com/system/files/neutron_packet_flows-notes-handout.pdf

You have multiple parallel network spaces involved here.

First you have the global or main routing table of the host. In the case of the network node you have 3 ethernet interface, two with and IP address and one without.

Second you have your external network. This is not configured on your network host. If you would run

netstat -rn

you would not see any route towards your external network. This means that when you send you ICMP packets to the external gateway they would go the default gateway set on the network node and not be answered. This network lives in the neutron router's network namespace.

Third you have the tenant network which also does not live in your main network space but in the same neutron router namespace.

To exemplify I will show you how it looks in my setup. I have an all in one node that is running inside a Virtualbox machine and it is running Ubuntu Trusty. My interfaces are eth0 for admin (also my default gateway connected to a NAT network in Virtualbox), eth1 for the tunnels, eth2 is my external net (also attached to a NAT network with the range 172.16.100.0/24).

Here is how the routing table of the Ubuntu VM looks like:

root@columbo:~# netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         10.0.1.1        0.0.0.0         UG        0 0          0 eth0
10.0.1.0        0.0.0.0         255.255.255.0   U         0 0          0 eth0
10.0.2.0        0.0.0.0         255.255.255.0   U         0 0          0 eth1
192.168.56.0    0.0.0.0         255.255.255.0   U         0 0          0 eth3
192.168.122.0   0.0.0.0         255.255.255.0   U         0 0          0 virbr0

So no sign of my external net (172.16.100.1). If I try to ping it I get nothing:

ping 172.16.100.1
PING 172.16.100.1 (172.16.100.1) 56(84) bytes of data.
^C
--- 172.16.100.1 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2004ms

Now you can see what network namespaces you have using the command ip netns list

root@columbo:~# ip netns list
qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b
qdhcp-120a6fde-7e2d-4856-90ee-5609a5f3035f
qdhcp-b7ab2080-a71a-44f6-9f66-fde526bb73d3

Inside a namespace you can run most of your normal commands - see bellow

root@columbo:~# ip netns exec qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         172.16.100.1    0.0.0.0         UG        0 0          0 qg-c69702a9-ae
10.255.1.0      0.0.0.0         255.255.255.0   U         0 0          0 qr-e706aba1-46
172.16.100.0    0.0.0.0         255.255.255.0   U         0 0          0 qg-c69702a9-ae

and

root@columbo:~# ip netns exec qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b arp -an
? (10.255.1.12) at fa:16:3e:40:50:36 [ether] on qr-e706aba1-46
? (172.16.100.1) at 52:54:00:12:35:00 [ether] on qg-c69702a9-ae
? (10.255.1.14) at fa:16:3e:75:5f:b4 [ether] on qr-e706aba1-46

and now ping will work (inside the namespace)

root@columbo:~# ip netns exec qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b ping 172.16.100.1
PING 172.16.100.1 (172.16.100.1) 56(84) bytes of data.
64 bytes from 172.16.100.1: icmp_seq=1 ttl=255 time=2.55 ms
64 bytes from 172.16.100.1: icmp_seq=2 ttl=255 time=0.555 ms
^C
--- 172.16.100.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 0.555/1.554/2.553/0.999 ms

I can also ping the router which has the ip 172.16.100.50:

root@columbo:~# ip netns exec qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b ping 172.16.100.50
PING 172.16.100.50 (172.16.100.50) 56(84) bytes of data.
64 bytes from 172.16.100.50: icmp_seq=1 ttl=64 time=0.065 ms
64 bytes from 172.16.100.50: icmp_seq=2 ttl=64 time=0.065 ms
^C
--- 172.16.100.50 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.065/0.065/0.065/0.000 ms

Same for my tenant network

root@columbo:~# neutron net-list
+--------------------------------------+---------------+------------------------------------------------------+
| id                                   | name          | subnets                                              |
+--------------------------------------+---------------+------------------------------------------------------+
| 120a6fde-7e2d-4856-90ee-5609a5f3035f | SecondVlan    | 5432f1c9-0bb6-4619-b897-65d301071f72 5.5.5.0/25      |
| f2597437-a005-44ad-9ce2-168fbc331e56 | outside_world | 3fe35e71-53d7-4432-8c82-a06856b79316 172.16.100.0/24 |
| b7ab2080-a71a-44f6-9f66-fde526bb73d3 | SERVER_VLAN_1 | 87d769f1-5cf3-48cf-8741-44a01479ff3e 10.255.1.0/24   |
+--------------------------------------+---------------+------------------------------------------------------+

the ping of the tenant router in the SERVER vlan

root@columbo:~# ip netns exec qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b ping 10.255.1.1
PING 10.255.1.1 (10.255.1.1) 56(84) bytes of data.
64 bytes from 10.255.1.1: icmp_seq=1 ttl=64 time=0.050 ms
64 bytes from 10.255.1.1: icmp_seq=2 ttl=64 time=0.064 ms
^C
--- 10.255.1.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.050/0.057/0.064/0.007 ms

I hope it is useful for somebody and not totally flawed.

like image 135
Carol Domokos Avatar answered Nov 16 '22 02:11

Carol Domokos