AWS Inter Region VPN with VYOS

Question

I'm trying to setup a VPN between two AWS Region(Oregon and Ireland). On the Oregon region, I've used the AWS VPN service and in Ireland, I've used the vyos EC2 instance from the marketplace. In Oregon I've launched one instance and opened its security group for ICMP request(0.0.0.0/0 for the tests). The security group of my Vyos instance is also open.

The VPN is UP on both sides but I've a strange issue.

Test1:

Ping from EC2-Oregon to EC2-Vyos: working

Test2:

Ping from EC2-Vyos to EC2-Oregon: Not working

But if I launch a wireshark on the EC2-Oregon, I saw the ICMP request, the problem seems to be located on the ICMP reply:

11.609958 169.254.12.138 -> 10.10.2.45   ICMP 98 Echo (ping) request    id=0x2f3d, seq=1/256, ttl=63
11.929702   10.0.1.177 -> 10.10.2.45   ICMP 71 Time-to-live exceeded  (Time to live exceeded in transit)
12.610213 169.254.12.138 -> 10.10.2.45   ICMP 98 Echo (ping) request   id=0x2f3d, seq=2/512, ttl=63
12.929659   10.0.1.177 -> 10.10.2.45   ICMP 71 Time-to-live exceeded (Time to live exceeded in transit)
13.610111 169.254.12.138 -> 10.10.2.45   ICMP 98 Echo (ping) request  id=0x2f3d, seq=3/768, ttl=63
13.929952   10.0.1.177 -> 10.10.2.45   ICMP 71 Time-to-live exceeded (Time to live exceeded in transit)

IPs Informations:
10.0.1.177: EC2-Vyos
10.10.2.45: EC2-Oregon
169.254.12.138: The IP of my vti1 in the Vyos conf

Test3:

Ping another EC2 instance in Irlande to EC2-Oregon: Not working

But on the EC2-Oregon instance, one more time I saw the ICMP request(10.2.10 is my EC2 instance in Ireland):

  0.361551    10.0.2.10 -> 10.10.2.45   ICMP 98 Echo (ping) request  id=0x130d, seq=4/1024, ttl=62
  0.361569   10.10.2.45 -> 10.0.2.10    ICMP 98 Echo (ping) reply    id=0x130d, seq=4/1024, ttl=64
  0.627332   10.0.1.177 -> 10.10.2.45   ICMP 71 Time-to-live exceeded (Time to live exceeded in transit)
  1.369717    10.0.2.10 -> 10.10.2.45   ICMP 98 Echo (ping) request  id=0x130d, seq=5/1280, ttl=62

I don't understand where the problem is located.

My vyos configuration:

interfaces {
ethernet eth0 {
    address dhcp
    duplex auto
    hw-id 0a:14:25:f4:8f:e9
    smp_affinity auto
    speed auto
}
loopback lo {
}
vti vti0 {
    address 169.254.12.62/30
    description "VPC tunnel 1"
    mtu 1436
}
vti vti1 {
    address 169.254.12.138/30
    description "VPC tunnel 2"
    mtu 1436
}
}
protocols {
bgp 65000 {
    neighbor 169.254.12.61 {
        remote-as 7224
        soft-reconfiguration {
            inbound
        }
        timers {
            holdtime 30
            keepalive 30
        }
    }
    neighbor 169.254.12.137 {
        remote-as 7224
        soft-reconfiguration {
            inbound
        }
        timers {
            holdtime 30
            keepalive 30
        }
    }
    network 10.0.0.0/16 {
    }
}
static {
    route 10.0.0.0/16 {
        next-hop 10.0.1.1 {
        }
    }
}
}
service {
ssh {
    disable-password-authentication
    port 22
}
}
system {
config-management {
    commit-revisions 20
}
console {
    device ttyS0 {
        speed 9600
    }
}
host-name VyOS-AMI
login {
    user vyos {
        authentication {
            encrypted-password "*"
            public-keys aws_key-XXXXXX {
                key AAAAB3....
                type ssh-rsa
            }
        }
        level admin
    }
}
ntp {
    server 0.pool.ntp.org {
    }
    server 1.pool.ntp.org {
    }
    server 2.pool.ntp.org {
    }
}
package {
    auto-sync 1
    repository community {
        components main
        distribution helium
        password ""
        url http://packages.vyos.net/vyos
        username ""
    }
}
syslog {
    global {
        facility all {
            level notice
        }
        facility protocols {
            level debug
        }
    }
}
time-zone UTC
}
vpn {
ipsec {
    esp-group AWS {
        compression disable
        lifetime 3600
        mode tunnel
        pfs enable
        proposal 1 {
            encryption aes128
            hash sha1
        }
    }
    ike-group AWS {
        dead-peer-detection {
            action restart
            interval 15
            timeout 30
        }
        key-exchange ikev1
        lifetime 28800
        proposal 1 {
            dh-group 2
            encryption aes128
            hash sha1
        }
    }
    ipsec-interfaces {
        interface eth0
    }
    nat-traversal enable
    site-to-site {
        peer 52.XX.XXX.113 {
            authentication {
                id 52.XX.XXX.132
                mode pre-shared-secret
                pre-shared-secret 7bRiFaXXXXXX
                remote-id 52.XX.XXX.113
            }
            connection-type initiate
            description "VPC tunnel 1"
            ike-group AWS
            local-address 10.0.1.177
            vti {
                bind vti0
                esp-group AWS
            }
        }
        peer 54.186.XXX.33 {
            authentication {
                id 52.XX.XXX.132
                mode pre-shared-secret
                pre-shared-secret AT2Q4XXXXXXXXXX
                remote-id 54.XXX.XXX.33
            }
            connection-type initiate
            description "VPC tunnel 2"
            ike-group AWS
            local-address 10.0.1.177
            vti {
                bind vti1
                esp-group AWS
            }
        }
    }
}
}

UPDATE:

Problem solved.

You can find the entire(working) of the configuration process on this repo https://github.com/mboret/aws-vyos

Answer 1

Finally, with an upgrade of Vyos my problem was solved. (This problem turns out to stem from VyOS bugs nos. 358 and 405, fixed in version 1.1.2.)

Solution(on the Vyos instance and not in the "configure" mode):

add system image http://packages.vyos.net/iso/release/1.1.5/vyos-1.1.5-amd64.iso
(Press Enter and answer Yes at each question).
reboot

And now I can ping and access through the VPN connection. The only one thing which still not working, is the test2, the Vyos instance can't ping through the VPN but it's not important for me, all other instances are reachable through it.

I've create a repository with the process to setup an AWS inter region VPN: https://github.com/mboret/aws-vyos