← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1829506] [NEW] Error mounting device on create instance (LXC)

 

Public bug reported:

Description
===========

After installing Openstack in Ubuntu 18.04 and using LXC as
virtualization engine, the creation of an instance fails with a log in
`nova-compute.log` reporting that:

 ```Instance failed to spawn: TypeError: mount() takes exactly 4
arguments (3 given)```

Extra analysis of the logs an code points towards this file of `python-
nova` package:

 /usr/lib/python2.7/dist-packages/nova/virt/disk/mount/api.py

The problem seems that the call at line 251

```
   246	    def mnt_dev(self):
   247	        """Mount the device into the file system."""
   248	        LOG.debug("Mount %(dev)s on %(dir)s",
   249	                  {'dev': self.mapped_device, 'dir': self.mount_dir})
   250	        out, err = nova.privsep.fs.mount(None, self.mapped_device,
   251	                                         self.mount_dir)
   252	        if err:
   253	            self.error = _('Failed to mount filesystem: %s') % err
   254	            LOG.debug(self.error)
   255	            return False
   256	
   257	        self.mounted = True
   258	        return True
```

is not matching the function signature defined in /usr/lib/python2.7
/dist-packages/nova/privsep/fs.py as:

```
    30	def mount(fstype, device, mountpoint, options):
    31	    mount_cmd = ['mount']
    32	    if fstype:
    33	        mount_cmd.extend(['-t', fstype])
    34	    if options is not None:
    35	        mount_cmd.extend(options)
    36	    mount_cmd.extend([device, mountpoint])
    37	    return processutils.execute(*mount_cmd)
```

So, it should be safe just to add `None` as `options` parameter.

Steps to reproduce
==================

Environment info:

  OS: Ubuntu 18.04.2 LTS
  Kernel: Linux 4.15.0-46-generic #49-Ubuntu SMP x86_64
  Openstack: queens

  Related packages versions:

  nova-api:          Installed: 2:17.0.9-0ubuntu1
  python-nova:       Installed: 2:17.0.9-0ubuntu1
  nova-compute:      Installed: 2:17.0.9-0ubuntu1
  nova-compute-lxc:  Installed: 2:17.0.9-0ubuntu1

Command executed to create the instance:

```
# openstack server create --flavor lxc.small --image lxc_ubuntu_18.04 --nic net-id=${NEUTRON_NETWORK_NAT_ID} --security-group default --key-name vagrant test_lxc_instance
+-------------------------------------+---------------------------------------------------------+
| Field                               | Value                                                   |
+-------------------------------------+---------------------------------------------------------+
| OS-DCF:diskConfig                   | MANUAL                                                  |
| OS-EXT-AZ:availability_zone         |                                                         |
| OS-EXT-SRV-ATTR:host                | None                                                    |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None                                                    |
| OS-EXT-SRV-ATTR:instance_name       |                                                         |
| OS-EXT-STS:power_state              | NOSTATE                                                 |
| OS-EXT-STS:task_state               | scheduling                                              |
| OS-EXT-STS:vm_state                 | building                                                |
| OS-SRV-USG:launched_at              | None                                                    |
| OS-SRV-USG:terminated_at            | None                                                    |
| accessIPv4                          |                                                         |
| accessIPv6                          |                                                         |
| addresses                           |                                                         |
| adminPass                           | GhYewcoXch3w                                            |
| config_drive                        |                                                         |
| created                             | 2019-05-17T10:00:57Z                                    |
| flavor                              | lxc.small (0)                                           |
| hostId                              |                                                         |
| id                                  | fe69997d-10c2-4850-bdba-2468da7148d6                    |
| image                               | lxc_ubuntu_18.04 (de3ca120-42f6-47a2-be1b-65c7276f1566) |
| key_name                            | vagrant                                                 |
| name                                | test_lxc_instance                                       |
| progress                            | 0                                                       |
| project_id                          | 4c2cf8fa37c8451d87ddd2cf28cfd6bc                        |
| properties                          |                                                         |
| security_groups                     | name='35a8788c-364b-4da3-9c26-4aebf0ac04c4'             |
| status                              | BUILD                                                   |
| updated                             | 2019-05-17T10:00:57Z                                    |
| user_id                             | 0d48615bb8c747f4b5ec5bfc693b6832                        |
| volumes_attached                    |                                                         |
+-------------------------------------+---------------------------------------------------------+
```

Expected result
===============
```
# openstack server list
+--------------------------------------+-------------------+--------+-----------------+------------------+-----------+
| ID                                   | Name              | Status | Networks        | Image            | Flavor    |
+--------------------------------------+-------------------+--------+-----------------+------------------+-----------+
| dac7b2b8-841b-4a16-a54f-2d5177f05934 | test_lxc_instance | ACTIVE | nat=10.123.1.11 | lxc_ubuntu_18.04 | lxc.small |
+--------------------------------------+-------------------+--------+-----------------+------------------+-----------+
```

Actual result
=============
```
# openstack server list
+--------------------------------------+-------------------+--------+----------+------------------+-----------+
| ID                                   | Name              | Status | Networks | Image            | Flavor    |
+--------------------------------------+-------------------+--------+----------+------------------+-----------+
| fe69997d-10c2-4850-bdba-2468da7148d6 | test_lxc_instance | ERROR  |          | lxc_ubuntu_18.04 | lxc.small |
+--------------------------------------+-------------------+--------+----------+------------------+-----------+
```

Logs
====

Included some logs generated during the command, and retrieved using:

 
```
grep -r "2019-05-17 10:0[01]" /var/log >> openstack_queens_lxc_instance_creation_mount_error.logs
```

** Affects: nova
     Importance: Undecided
     Assignee: Miguel Ángel Herranz Trillo (maherranzt)
         Status: In Progress

** Attachment added: "openstack_queens_lxc_instance_creation_mount_error.logs"
   https://bugs.launchpad.net/bugs/1829506/+attachment/5264452/+files/openstack_queens_lxc_instance_creation_mount_error.logs

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1829506

Title:
  Error mounting device on create instance (LXC)

Status in OpenStack Compute (nova):
  In Progress

Bug description:
  Description
  ===========

  After installing Openstack in Ubuntu 18.04 and using LXC as
  virtualization engine, the creation of an instance fails with a log in
  `nova-compute.log` reporting that:

   ```Instance failed to spawn: TypeError: mount() takes exactly 4
  arguments (3 given)```

  Extra analysis of the logs an code points towards this file of
  `python-nova` package:

   /usr/lib/python2.7/dist-packages/nova/virt/disk/mount/api.py

  The problem seems that the call at line 251

  ```
     246	    def mnt_dev(self):
     247	        """Mount the device into the file system."""
     248	        LOG.debug("Mount %(dev)s on %(dir)s",
     249	                  {'dev': self.mapped_device, 'dir': self.mount_dir})
     250	        out, err = nova.privsep.fs.mount(None, self.mapped_device,
     251	                                         self.mount_dir)
     252	        if err:
     253	            self.error = _('Failed to mount filesystem: %s') % err
     254	            LOG.debug(self.error)
     255	            return False
     256	
     257	        self.mounted = True
     258	        return True
  ```

  is not matching the function signature defined in /usr/lib/python2.7
  /dist-packages/nova/privsep/fs.py as:

  ```
      30	def mount(fstype, device, mountpoint, options):
      31	    mount_cmd = ['mount']
      32	    if fstype:
      33	        mount_cmd.extend(['-t', fstype])
      34	    if options is not None:
      35	        mount_cmd.extend(options)
      36	    mount_cmd.extend([device, mountpoint])
      37	    return processutils.execute(*mount_cmd)
  ```

  So, it should be safe just to add `None` as `options` parameter.

  Steps to reproduce
  ==================

  Environment info:

    OS: Ubuntu 18.04.2 LTS
    Kernel: Linux 4.15.0-46-generic #49-Ubuntu SMP x86_64
    Openstack: queens

    Related packages versions:

    nova-api:          Installed: 2:17.0.9-0ubuntu1
    python-nova:       Installed: 2:17.0.9-0ubuntu1
    nova-compute:      Installed: 2:17.0.9-0ubuntu1
    nova-compute-lxc:  Installed: 2:17.0.9-0ubuntu1

  Command executed to create the instance:

  ```
  # openstack server create --flavor lxc.small --image lxc_ubuntu_18.04 --nic net-id=${NEUTRON_NETWORK_NAT_ID} --security-group default --key-name vagrant test_lxc_instance
  +-------------------------------------+---------------------------------------------------------+
  | Field                               | Value                                                   |
  +-------------------------------------+---------------------------------------------------------+
  | OS-DCF:diskConfig                   | MANUAL                                                  |
  | OS-EXT-AZ:availability_zone         |                                                         |
  | OS-EXT-SRV-ATTR:host                | None                                                    |
  | OS-EXT-SRV-ATTR:hypervisor_hostname | None                                                    |
  | OS-EXT-SRV-ATTR:instance_name       |                                                         |
  | OS-EXT-STS:power_state              | NOSTATE                                                 |
  | OS-EXT-STS:task_state               | scheduling                                              |
  | OS-EXT-STS:vm_state                 | building                                                |
  | OS-SRV-USG:launched_at              | None                                                    |
  | OS-SRV-USG:terminated_at            | None                                                    |
  | accessIPv4                          |                                                         |
  | accessIPv6                          |                                                         |
  | addresses                           |                                                         |
  | adminPass                           | GhYewcoXch3w                                            |
  | config_drive                        |                                                         |
  | created                             | 2019-05-17T10:00:57Z                                    |
  | flavor                              | lxc.small (0)                                           |
  | hostId                              |                                                         |
  | id                                  | fe69997d-10c2-4850-bdba-2468da7148d6                    |
  | image                               | lxc_ubuntu_18.04 (de3ca120-42f6-47a2-be1b-65c7276f1566) |
  | key_name                            | vagrant                                                 |
  | name                                | test_lxc_instance                                       |
  | progress                            | 0                                                       |
  | project_id                          | 4c2cf8fa37c8451d87ddd2cf28cfd6bc                        |
  | properties                          |                                                         |
  | security_groups                     | name='35a8788c-364b-4da3-9c26-4aebf0ac04c4'             |
  | status                              | BUILD                                                   |
  | updated                             | 2019-05-17T10:00:57Z                                    |
  | user_id                             | 0d48615bb8c747f4b5ec5bfc693b6832                        |
  | volumes_attached                    |                                                         |
  +-------------------------------------+---------------------------------------------------------+
  ```

  Expected result
  ===============
  ```
  # openstack server list
  +--------------------------------------+-------------------+--------+-----------------+------------------+-----------+
  | ID                                   | Name              | Status | Networks        | Image            | Flavor    |
  +--------------------------------------+-------------------+--------+-----------------+------------------+-----------+
  | dac7b2b8-841b-4a16-a54f-2d5177f05934 | test_lxc_instance | ACTIVE | nat=10.123.1.11 | lxc_ubuntu_18.04 | lxc.small |
  +--------------------------------------+-------------------+--------+-----------------+------------------+-----------+
  ```

  Actual result
  =============
  ```
  # openstack server list
  +--------------------------------------+-------------------+--------+----------+------------------+-----------+
  | ID                                   | Name              | Status | Networks | Image            | Flavor    |
  +--------------------------------------+-------------------+--------+----------+------------------+-----------+
  | fe69997d-10c2-4850-bdba-2468da7148d6 | test_lxc_instance | ERROR  |          | lxc_ubuntu_18.04 | lxc.small |
  +--------------------------------------+-------------------+--------+----------+------------------+-----------+
  ```

  Logs
  ====

  Included some logs generated during the command, and retrieved using:

   
  ```
  grep -r "2019-05-17 10:0[01]" /var/log >> openstack_queens_lxc_instance_creation_mount_error.logs
  ```

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1829506/+subscriptions


Follow ups