← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1971050] [NEW] Nested KVM Networking Issue

 

Public bug reported:

## Host environment
 - Operating system: (ubuntu 20.04 server)
 - OS/kernel version: (5.13.0.40 Generic)
 - Architecture: (64 bit cpu architecture)
 - QEMU version: (latest using sudo apt install virt-manager)

## Emulated/Virtualized environment
 - Operating system: (ubuntu 20.04 server)
 - OS/kernel version: ( 5.13.0.40 Generic)
 - Architecture: (64 bit cpu architecture)


## Description of problem
<!-- Describe the problem, including any error/crash messages seen. -->
Hi, 

Inside openstack i have an instance of Ubuntu 20.04 and i have installed
KVM ( using virt-manager ) to setup a Virtual Machine ... i have done
that and i created a VM of ubuntu 20.04 inside the Openstack Instance
but there are networking issue while i set the default parameter as
setting up the VM ( i mean the networking is as default to NAT ) , So
when the VM is up and running the PING to 8.8.8.8 is available and also
ping to google.com is also valid which shows that the DNS is correctly
working ... but there is not connectivity with packages while i do sudo
apt update, it will not get any package update and also the wget to
google.com is shows that its connected to it but it wont able to
download!!! the same happen with curl to any other websites...


I'm confirming that the openstack instance has full access to the internet including ping and wget , .... but the VM is not working correctly!

P.S. I have set the ip forwarding, Iptables , ... also disabled firewals
but notting changed!!


Would you please fix this ?


## Steps to reproduce
1. creating an openstack instance from ubuntu 20.04 server image
2. updating and upgrading packages setting ip forwarding to 1 ( Enabled), firewall
3. and kernel to 5.13.0.40 and installing virt-manager then reboot 
3. creating a VM with default KVM networking ( NAT ) using ubuntu 20.04 server image
4. trying ping, wget, curl , ...


These are my commands after creating an instance with 8VCPU, 16VRAM, 100VDisk, ubuntu cloud 20.04 image:
sudo apt update && sudo apt full-upgrade -y && sudo apt install linux-image-5.13.0-40-generic linux-headers-5.13.0-40-generic -y && sudo reboot
sudo apt update && sudo uname -a
Linux test 5.13.0-40-generic #45~20.04.1-Ubuntu SMP Mon Apr 4 09:38:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
sudo apt install virt-manager -y && sudo reboot
sudo systemctl status libvirtd
Its running    IP range 192.168.122.2
sudo usermod -a -G libvirt ubuntu
then download ubuntu server 20.04 image from https://releases.ubuntu.com/20.04/ubuntu-20.04.4-live-server-amd64.iso
and create a new VM using KVM by virt-manager as shown bellow:
https://gitlab.com/qemu-project/qemu/uploads/8bd4c7381a60832b3a5fcd9dbd3665de/image.png


qemu-system-x86_64 --version
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.21)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers


Here is my networking :
```
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc fq_codel state UP group default qlen 1000
    link/ether fa:16:3e:10:60:0e brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    inet 10.20.30.52/24 brd 10.20.30.255 scope global dynamic ens3
       valid_lft 34758sec preferred_lft 34758sec
    inet6 fe80::f816:3eff:fe10:600e/64 scope link
       valid_lft forever preferred_lft forever
3: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 52:54:00:98:07:1a brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 state DOWN group default qlen 1000
    link/ether 52:54:00:98:07:1a brd ff:ff:ff:ff:ff:ff
5: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master virbr0 state UNKNOWN group default qlen 1000
    link/ether fe:54:00:f9:5d:4d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fef9:5d4d/64 scope link
       valid_lft forever preferred_lft forever
```


And this is my Iptable

```
iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
LIBVIRT_INP  all  --  anywhere             anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
LIBVIRT_FWX  all  --  anywhere             anywhere
LIBVIRT_FWI  all  --  anywhere             anywhere
LIBVIRT_FWO  all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
LIBVIRT_OUT  all  --  anywhere             anywhere

Chain LIBVIRT_FWI (1 references)
target     prot opt source               destination
ACCEPT     all  --  anywhere             192.168.122.0/24     ctstate RELATED,ESTABLISHED
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

Chain LIBVIRT_FWO (1 references)
target     prot opt source               destination
ACCEPT     all  --  192.168.122.0/24     anywhere
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

Chain LIBVIRT_FWX (1 references)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere

Chain LIBVIRT_INP (1 references)
target     prot opt source               destination
ACCEPT     udp  --  anywhere             anywhere             udp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:domain
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:67

Chain LIBVIRT_OUT (1 references)
target     prot opt source               destination
ACCEPT     udp  --  anywhere             anywhere             udp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:domain
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootpc
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:68
```


I think this is a bug because i have configured the same ssttings on
baremetal system  and it was completely OK ... but here when i use the
OPENSTACK Instance  the problem occures! ( Actually i think this problem
happen in Nested KVM  situation!)


I would be glad to hear about hint on how to solve this issue!

Thanks
Best regards

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1971050

Title:
  Nested KVM Networking Issue

Status in neutron:
  New

Bug description:
  ## Host environment
   - Operating system: (ubuntu 20.04 server)
   - OS/kernel version: (5.13.0.40 Generic)
   - Architecture: (64 bit cpu architecture)
   - QEMU version: (latest using sudo apt install virt-manager)

  ## Emulated/Virtualized environment
   - Operating system: (ubuntu 20.04 server)
   - OS/kernel version: ( 5.13.0.40 Generic)
   - Architecture: (64 bit cpu architecture)

  
  ## Description of problem
  <!-- Describe the problem, including any error/crash messages seen. -->
  Hi, 

  Inside openstack i have an instance of Ubuntu 20.04 and i have
  installed KVM ( using virt-manager ) to setup a Virtual Machine ... i
  have done that and i created a VM of ubuntu 20.04 inside the Openstack
  Instance but there are networking issue while i set the default
  parameter as setting up the VM ( i mean the networking is as default
  to NAT ) , So when the VM is up and running the PING to 8.8.8.8 is
  available and also ping to google.com is also valid which shows that
  the DNS is correctly working ... but there is not connectivity with
  packages while i do sudo apt update, it will not get any package
  update and also the wget to google.com is shows that its connected to
  it but it wont able to download!!! the same happen with curl to any
  other websites...

  
  I'm confirming that the openstack instance has full access to the internet including ping and wget , .... but the VM is not working correctly!

  P.S. I have set the ip forwarding, Iptables , ... also disabled
  firewals but notting changed!!

  
  Would you please fix this ?


  ## Steps to reproduce
  1. creating an openstack instance from ubuntu 20.04 server image
  2. updating and upgrading packages setting ip forwarding to 1 ( Enabled), firewall
  3. and kernel to 5.13.0.40 and installing virt-manager then reboot 
  3. creating a VM with default KVM networking ( NAT ) using ubuntu 20.04 server image
  4. trying ping, wget, curl , ...

  
  These are my commands after creating an instance with 8VCPU, 16VRAM, 100VDisk, ubuntu cloud 20.04 image:
  sudo apt update && sudo apt full-upgrade -y && sudo apt install linux-image-5.13.0-40-generic linux-headers-5.13.0-40-generic -y && sudo reboot
  sudo apt update && sudo uname -a
  Linux test 5.13.0-40-generic #45~20.04.1-Ubuntu SMP Mon Apr 4 09:38:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
  sudo apt install virt-manager -y && sudo reboot
  sudo systemctl status libvirtd
  Its running    IP range 192.168.122.2
  sudo usermod -a -G libvirt ubuntu
  then download ubuntu server 20.04 image from https://releases.ubuntu.com/20.04/ubuntu-20.04.4-live-server-amd64.iso
  and create a new VM using KVM by virt-manager as shown bellow:
  https://gitlab.com/qemu-project/qemu/uploads/8bd4c7381a60832b3a5fcd9dbd3665de/image.png

  
  qemu-system-x86_64 --version
  QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.21)
  Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers


  Here is my networking :
  ```
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
      inet 127.0.0.1/8 scope host lo
         valid_lft forever preferred_lft forever
      inet6 ::1/128 scope host
         valid_lft forever preferred_lft forever
  2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc fq_codel state UP group default qlen 1000
      link/ether fa:16:3e:10:60:0e brd ff:ff:ff:ff:ff:ff
      altname enp0s3
      inet 10.20.30.52/24 brd 10.20.30.255 scope global dynamic ens3
         valid_lft 34758sec preferred_lft 34758sec
      inet6 fe80::f816:3eff:fe10:600e/64 scope link
         valid_lft forever preferred_lft forever
  3: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
      link/ether 52:54:00:98:07:1a brd ff:ff:ff:ff:ff:ff
      inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
         valid_lft forever preferred_lft forever
  4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 state DOWN group default qlen 1000
      link/ether 52:54:00:98:07:1a brd ff:ff:ff:ff:ff:ff
  5: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master virbr0 state UNKNOWN group default qlen 1000
      link/ether fe:54:00:f9:5d:4d brd ff:ff:ff:ff:ff:ff
      inet6 fe80::fc54:ff:fef9:5d4d/64 scope link
         valid_lft forever preferred_lft forever
  ```

  
  And this is my Iptable

  ```
  iptables -L
  Chain INPUT (policy ACCEPT)
  target     prot opt source               destination
  LIBVIRT_INP  all  --  anywhere             anywhere

  Chain FORWARD (policy ACCEPT)
  target     prot opt source               destination
  LIBVIRT_FWX  all  --  anywhere             anywhere
  LIBVIRT_FWI  all  --  anywhere             anywhere
  LIBVIRT_FWO  all  --  anywhere             anywhere

  Chain OUTPUT (policy ACCEPT)
  target     prot opt source               destination
  LIBVIRT_OUT  all  --  anywhere             anywhere

  Chain LIBVIRT_FWI (1 references)
  target     prot opt source               destination
  ACCEPT     all  --  anywhere             192.168.122.0/24     ctstate RELATED,ESTABLISHED
  REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

  Chain LIBVIRT_FWO (1 references)
  target     prot opt source               destination
  ACCEPT     all  --  192.168.122.0/24     anywhere
  REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

  Chain LIBVIRT_FWX (1 references)
  target     prot opt source               destination
  ACCEPT     all  --  anywhere             anywhere

  Chain LIBVIRT_INP (1 references)
  target     prot opt source               destination
  ACCEPT     udp  --  anywhere             anywhere             udp dpt:domain
  ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:domain
  ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
  ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:67

  Chain LIBVIRT_OUT (1 references)
  target     prot opt source               destination
  ACCEPT     udp  --  anywhere             anywhere             udp dpt:domain
  ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:domain
  ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootpc
  ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:68
  ```


  I think this is a bug because i have configured the same ssttings on
  baremetal system  and it was completely OK ... but here when i use the
  OPENSTACK Instance  the problem occures! ( Actually i think this
  problem happen in Nested KVM  situation!)

  
  I would be glad to hear about hint on how to solve this issue!

  Thanks
  Best regards

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1971050/+subscriptions



Follow ups