← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1360720] [NEW] nova network always report error when boot VM

 

Public bug reported:

When boot a VM with nova network, it always report " NovaException:
Failed to add interface: device br100 is a bridge device itself; can't
enslave a bridge device to a bridge device." and this caused my VM can
not be started.

Reproduce steps:
1) Install OpenStack with Devstack
jay@jay001:~/src/devstack$ cat localrc 
HOST_IP=192.168.0.103
ADMIN_PASSWORD=nova
MYSQL_PASSWORD=nova
RABBIT_PASSWORD=nova
SERVICE_PASSWORD=nova
SERVICE_TOKEN=tokentoken
FLAT_INTERFACE=br100
#VIRT_DRIVER=docker
#RECLONE=yes
 
VERBOSE=True
LOG_COLOR=True
SCREEN_LOGDIR=/opt/stack/logs
 
#disable_service horizon
 
#OFFLINE=False
#OFFLINE=True
#ENABLED_SERVICES+=,heat,h-api-cfn,h-api-cw,h-eng
ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng
#IMAGE_URLS+=",http://fedorapeople.org/groups/heat/prebuilt-jeos-images/F16-x86_64-cfntools.qcow2,http://fedorapeople.org/groups/heat/prebuilt-jeos-images/F16-i386-cfntools.qcow2";
#ENABLED_SERVICES+=ceilometer-acompute,ceilometer-acentral,ceilometer-collector,ceilometer-api,ceilometer-alarm-notify,ceilometer-alarm-eval
#CEILOMETER_BACKEND=mysql
2) After install finished, boot a VM
jay@jay001:~/src/devstack$ nova boot --image  cirros-0.3.2-x86_64-uec --flavor 1 vm1
+--------------------------------------+----------------------------------------------------------------+
| Property                             | Value                                                          |
+--------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                                         |
| OS-EXT-AZ:availability_zone          | nova                                                           |
| OS-EXT-SRV-ATTR:host                 | -                                                              |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                                              |
| OS-EXT-SRV-ATTR:instance_name        | instance-00000002                                              |
| OS-EXT-STS:power_state               | 0                                                              |
| OS-EXT-STS:task_state                | scheduling                                                     |
| OS-EXT-STS:vm_state                  | building                                                       |
| OS-SRV-USG:launched_at               | -                                                              |
| OS-SRV-USG:terminated_at             | -                                                              |
| accessIPv4                           |                                                                |
| accessIPv6                           |                                                                |
| adminPass                            | F5NXNAVJMXNi                                                   |
| config_drive                         |                                                                |
| created                              | 2014-08-23T23:54:50Z                                           |
| flavor                               | m1.tiny (1)                                                    |
| hostId                               |                                                                |
| id                                   | 48eec530-4279-423c-a134-0bbb19287d72                           |
| image                                | cirros-0.3.2-x86_64-uec (b8e84ec2-a63c-4f24-b9bb-6532f507668e) |
| key_name                             | -                                                              |
| metadata                             | {}                                                             |
| name                                 | vm1                                                            |
| os-extended-volumes:volumes_attached | []                                                             |
| progress                             | 0                                                              |
| security_groups                      | default                                                        |
| status                               | BUILD                                                          |
| tenant_id                            | 0694df50d3c34d128160d9a4a90db5ff                               |
| updated                              | 2014-08-23T23:54:50Z                                           |
| user_id                              | 60cfc7aa7cc04b54a6bcb2d778146b86                               |
+--------------------------------------+----------------------------------------------------------------+
jay@jay001:~/src/devstack$ nova list
+--------------------------------------+------+--------+------------+-------------+----------+
| ID                                   | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+--------+------------+-------------+----------+
| 48eec530-4279-423c-a134-0bbb19287d72 | vm1  | BUILD  | spawning   | NOSTATE     |          |
+--------------------------------------+------+--------+------------+-------------+----------+
jay@jay001:~/src/devstack$ nova list
+--------------------------------------+------+--------+------------+-------------+----------+
| ID                                   | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+--------+------------+-------------+----------+
| 48eec530-4279-423c-a134-0bbb19287d72 | vm1  | ERROR  | -          | NOSTATE     |          |
+--------------------------------------+------+--------+------------+-------------+----------+
3) Stop nova-network and restart it,  nova-network failed to start
nsure_bridge" from (pid=15502) inner /opt/stack/nova/nova/openstack/common/lockutils.py:328
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 449, in fire_timers
    timer()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 58, in __call__
    cb(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 207, in main
    result = function(*args, **kwargs)
  File "/opt/stack/nova/nova/openstack/common/service.py", line 490, in run_service
    service.start()
  File "/opt/stack/nova/nova/service.py", line 164, in start
    self.manager.init_host()
  File "/opt/stack/nova/nova/network/manager.py", line 1776, in init_host
    super(FlatDHCPManager, self).init_host()
  File "/opt/stack/nova/nova/network/manager.py", line 334, in init_host
    self._setup_network_on_host(ctxt, network)
  File "/opt/stack/nova/nova/network/manager.py", line 1785, in _setup_network_on_host
    self._initialize_network(network)
  File "/opt/stack/nova/nova/network/manager.py", line 1451, in _initialize_network
    self.l3driver.initialize_gateway(network)
  File "/opt/stack/nova/nova/network/l3.py", line 105, in initialize_gateway
    gateway=(network_ref['gateway'] is not None))
  File "/opt/stack/nova/nova/network/linux_net.py", line 1411, in plug
    return _get_interface_driver().plug(network, mac_address, gateway)
  File "/opt/stack/nova/nova/network/linux_net.py", line 1460, in plug
    network, gateway)
  File "/opt/stack/nova/nova/openstack/common/lockutils.py", line 325, in inner
    return f(*args, **kwargs)
  File "/opt/stack/nova/nova/network/linux_net.py", line 1575, in ensure_bridge
    raise exception.NovaException(msg)
NovaException: Failed to add interface: device br100 is a bridge device itself; can't enslave a bridge device to a bridge device.
 
2014-08-24 07:58:08.956 ERROR nova.openstack.common.threadgroup [-] Failed to add interface: device br100 is a bridge device itself; can't enslave a bridge device to a bridge device.
 
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/openstack/common/threadgroup.py", line 125, in wait
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     x.wait()
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/openstack/common/threadgroup.py", line 47, in wait
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     return self.thread.wait()
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168, in wait
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     return self._exit_event.wait()
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 120, in wait
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     return hubs.get_hub().switch()
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 287, in switch
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     return self.greenlet.switch()
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 207, in main
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     result = function(*args, **kwargs)
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/openstack/common/service.py", line 490, in run_service
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     service.start()
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/service.py", line 164, in start
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     self.manager.init_host()
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/manager.py", line 1776, in init_host
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     super(FlatDHCPManager, self).init_host()
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/manager.py", line 334, in init_host
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     self._setup_network_on_host(ctxt, network)
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/manager.py", line 1785, in _setup_network_on_host
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     self._initialize_network(network)
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/manager.py", line 1451, in _initialize_network
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     self.l3driver.initialize_gateway(network)
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/l3.py", line 105, in initialize_gateway
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     gateway=(network_ref['gateway'] is not None))
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/linux_net.py", line 1411, in plug
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     return _get_interface_driver().plug(network, mac_address, gateway)
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/linux_net.py", line 1460, in plug
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     network, gateway)
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/openstack/common/lockutils.py", line 325, in inner
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     return f(*args, **kwargs)
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/linux_net.py", line 1575, in ensure_bridge
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     raise exception.NovaException(msg)
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup NovaException: Failed to add interface: device br100 is a bridge device itself; can't enslave a bridge device to a bridge device.
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup 
2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup

** Affects: nova
     Importance: Undecided
         Status: New

** Affects: ubuntu
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1360720

Title:
  nova network always report error when boot VM

Status in OpenStack Compute (Nova):
  New
Status in Ubuntu:
  New

Bug description:
  When boot a VM with nova network, it always report " NovaException:
  Failed to add interface: device br100 is a bridge device itself; can't
  enslave a bridge device to a bridge device." and this caused my VM can
  not be started.

  Reproduce steps:
  1) Install OpenStack with Devstack
  jay@jay001:~/src/devstack$ cat localrc 
  HOST_IP=192.168.0.103
  ADMIN_PASSWORD=nova
  MYSQL_PASSWORD=nova
  RABBIT_PASSWORD=nova
  SERVICE_PASSWORD=nova
  SERVICE_TOKEN=tokentoken
  FLAT_INTERFACE=br100
  #VIRT_DRIVER=docker
  #RECLONE=yes
   
  VERBOSE=True
  LOG_COLOR=True
  SCREEN_LOGDIR=/opt/stack/logs
   
  #disable_service horizon
   
  #OFFLINE=False
  #OFFLINE=True
  #ENABLED_SERVICES+=,heat,h-api-cfn,h-api-cw,h-eng
  ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng
  #IMAGE_URLS+=",http://fedorapeople.org/groups/heat/prebuilt-jeos-images/F16-x86_64-cfntools.qcow2,http://fedorapeople.org/groups/heat/prebuilt-jeos-images/F16-i386-cfntools.qcow2";
  #ENABLED_SERVICES+=ceilometer-acompute,ceilometer-acentral,ceilometer-collector,ceilometer-api,ceilometer-alarm-notify,ceilometer-alarm-eval
  #CEILOMETER_BACKEND=mysql
  2) After install finished, boot a VM
  jay@jay001:~/src/devstack$ nova boot --image  cirros-0.3.2-x86_64-uec --flavor 1 vm1
  +--------------------------------------+----------------------------------------------------------------+
  | Property                             | Value                                                          |
  +--------------------------------------+----------------------------------------------------------------+
  | OS-DCF:diskConfig                    | MANUAL                                                         |
  | OS-EXT-AZ:availability_zone          | nova                                                           |
  | OS-EXT-SRV-ATTR:host                 | -                                                              |
  | OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                                              |
  | OS-EXT-SRV-ATTR:instance_name        | instance-00000002                                              |
  | OS-EXT-STS:power_state               | 0                                                              |
  | OS-EXT-STS:task_state                | scheduling                                                     |
  | OS-EXT-STS:vm_state                  | building                                                       |
  | OS-SRV-USG:launched_at               | -                                                              |
  | OS-SRV-USG:terminated_at             | -                                                              |
  | accessIPv4                           |                                                                |
  | accessIPv6                           |                                                                |
  | adminPass                            | F5NXNAVJMXNi                                                   |
  | config_drive                         |                                                                |
  | created                              | 2014-08-23T23:54:50Z                                           |
  | flavor                               | m1.tiny (1)                                                    |
  | hostId                               |                                                                |
  | id                                   | 48eec530-4279-423c-a134-0bbb19287d72                           |
  | image                                | cirros-0.3.2-x86_64-uec (b8e84ec2-a63c-4f24-b9bb-6532f507668e) |
  | key_name                             | -                                                              |
  | metadata                             | {}                                                             |
  | name                                 | vm1                                                            |
  | os-extended-volumes:volumes_attached | []                                                             |
  | progress                             | 0                                                              |
  | security_groups                      | default                                                        |
  | status                               | BUILD                                                          |
  | tenant_id                            | 0694df50d3c34d128160d9a4a90db5ff                               |
  | updated                              | 2014-08-23T23:54:50Z                                           |
  | user_id                              | 60cfc7aa7cc04b54a6bcb2d778146b86                               |
  +--------------------------------------+----------------------------------------------------------------+
  jay@jay001:~/src/devstack$ nova list
  +--------------------------------------+------+--------+------------+-------------+----------+
  | ID                                   | Name | Status | Task State | Power State | Networks |
  +--------------------------------------+------+--------+------------+-------------+----------+
  | 48eec530-4279-423c-a134-0bbb19287d72 | vm1  | BUILD  | spawning   | NOSTATE     |          |
  +--------------------------------------+------+--------+------------+-------------+----------+
  jay@jay001:~/src/devstack$ nova list
  +--------------------------------------+------+--------+------------+-------------+----------+
  | ID                                   | Name | Status | Task State | Power State | Networks |
  +--------------------------------------+------+--------+------------+-------------+----------+
  | 48eec530-4279-423c-a134-0bbb19287d72 | vm1  | ERROR  | -          | NOSTATE     |          |
  +--------------------------------------+------+--------+------------+-------------+----------+
  3) Stop nova-network and restart it,  nova-network failed to start
  nsure_bridge" from (pid=15502) inner /opt/stack/nova/nova/openstack/common/lockutils.py:328
  Traceback (most recent call last):
    File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 449, in fire_timers
      timer()
    File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 58, in __call__
      cb(*args, **kw)
    File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 207, in main
      result = function(*args, **kwargs)
    File "/opt/stack/nova/nova/openstack/common/service.py", line 490, in run_service
      service.start()
    File "/opt/stack/nova/nova/service.py", line 164, in start
      self.manager.init_host()
    File "/opt/stack/nova/nova/network/manager.py", line 1776, in init_host
      super(FlatDHCPManager, self).init_host()
    File "/opt/stack/nova/nova/network/manager.py", line 334, in init_host
      self._setup_network_on_host(ctxt, network)
    File "/opt/stack/nova/nova/network/manager.py", line 1785, in _setup_network_on_host
      self._initialize_network(network)
    File "/opt/stack/nova/nova/network/manager.py", line 1451, in _initialize_network
      self.l3driver.initialize_gateway(network)
    File "/opt/stack/nova/nova/network/l3.py", line 105, in initialize_gateway
      gateway=(network_ref['gateway'] is not None))
    File "/opt/stack/nova/nova/network/linux_net.py", line 1411, in plug
      return _get_interface_driver().plug(network, mac_address, gateway)
    File "/opt/stack/nova/nova/network/linux_net.py", line 1460, in plug
      network, gateway)
    File "/opt/stack/nova/nova/openstack/common/lockutils.py", line 325, in inner
      return f(*args, **kwargs)
    File "/opt/stack/nova/nova/network/linux_net.py", line 1575, in ensure_bridge
      raise exception.NovaException(msg)
  NovaException: Failed to add interface: device br100 is a bridge device itself; can't enslave a bridge device to a bridge device.
   
  2014-08-24 07:58:08.956 ERROR nova.openstack.common.threadgroup [-] Failed to add interface: device br100 is a bridge device itself; can't enslave a bridge device to a bridge device.
   
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/openstack/common/threadgroup.py", line 125, in wait
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     x.wait()
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/openstack/common/threadgroup.py", line 47, in wait
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     return self.thread.wait()
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168, in wait
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     return self._exit_event.wait()
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 120, in wait
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     return hubs.get_hub().switch()
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 287, in switch
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     return self.greenlet.switch()
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 207, in main
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     result = function(*args, **kwargs)
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/openstack/common/service.py", line 490, in run_service
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     service.start()
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/service.py", line 164, in start
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     self.manager.init_host()
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/manager.py", line 1776, in init_host
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     super(FlatDHCPManager, self).init_host()
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/manager.py", line 334, in init_host
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     self._setup_network_on_host(ctxt, network)
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/manager.py", line 1785, in _setup_network_on_host
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     self._initialize_network(network)
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/manager.py", line 1451, in _initialize_network
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     self.l3driver.initialize_gateway(network)
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/l3.py", line 105, in initialize_gateway
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     gateway=(network_ref['gateway'] is not None))
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/linux_net.py", line 1411, in plug
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     return _get_interface_driver().plug(network, mac_address, gateway)
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/linux_net.py", line 1460, in plug
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     network, gateway)
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/openstack/common/lockutils.py", line 325, in inner
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     return f(*args, **kwargs)
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup   File "/opt/stack/nova/nova/network/linux_net.py", line 1575, in ensure_bridge
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup     raise exception.NovaException(msg)
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup NovaException: Failed to add interface: device br100 is a bridge device itself; can't enslave a bridge device to a bridge device.
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup 
  2014-08-24 07:58:08.956 TRACE nova.openstack.common.threadgroup

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1360720/+subscriptions


Follow ups

References