maas-builds team mailing list archive

Thread
Date
Build failed in Jenkins: precise-adt-maas-daily » amd64,maas-lenovo-lab #311

To: maas-builds@xxxxxxxxxxxxxxxxxxx
From: Jenkins Notification <devnull@xxxxxxxxxxxxx>
Date: Wed, 11 Dec 2013 00:15:37 +0000 (UTC)
In-reply-to: <1057574239.206.1386706967472.JavaMail.jenkins@tachash>
See <http://d-jenkins.ubuntu-ci:8080/job/precise-adt-maas-daily/./ARCH=amd64,label=maas-lenovo-lab/311/>

------------------------------------------
[...truncated 4443 lines...]
 > maasserver:0036_populate_nodegroup_cluster_name
 > maasserver:0037_nodegroup_cluster_name_unique
 > maasserver:0038_nodegroupinterface_ip_range_fix
 > maasserver:0039_add_filestorage_content
 > maasserver:0039_add_nodegroup_to_bootimage
 > maasserver:0040_make_filestorage_data_not_null
 > maasserver:0041_remove_filestorage_data
 > maasserver:0042_fix_039_conflict
 > maasserver:0043_unique_hostname_preparation
 > maasserver:0044_node_hostname_unique
 > maasserver:0045_add_tag_kernel_opts
 > maasserver:0046_add_nodegroup_maas_url
 > maasserver:0047_add_owner_to_filestorage
 > maasserver:0048_add_key_to_filestorage
 > maasserver:0049_filestorage_key_unique
 > maasserver:0050_shared_to_per_tenant_storage
 > maasserver:0051_bigger_distro_series_name
 > maasserver:0052_add_node_storage
 > maasserver:0053_node_routers
 > maasserver:0054_download_progress
 > maasserver:0055_nullable_bytes_downloaded
 > maasserver:0056_netboot_off_for_allocated_nodes
 > maasserver:0057_remove_hardware_details
 > maasserver:0058_add_agent_name_to_node
 - Loading initial data for maasserver.
Installed 0 object(s) from 0 fixture(s)
Running migrations for metadataserver:
 - Migrating forwards to 0014_commission_result_rename_data_bin_col.
 > metadataserver:0001_initial
 > metadataserver:0002_add_nodecommissionresult
 > metadataserver:0003_populate_hardware_details
 > metadataserver:0004_add_commissioningscript
 > metadataserver:0005_nodecommissionresult_add_timestamp
 > metadataserver:0006_nodecommissionresult_add_status
 > metadataserver:0007_nodecommissionresult_change_name_size
 > metadataserver:0008_rename_lshw_commissioning_output
 > metadataserver:0009_delete_status
 > metadataserver:0010_add_script_result
 > metadataserver:0011_commission_result_binary_data_col
 > metadataserver:0012_commission_result_binary_data_recode
 > metadataserver:0013_commission_result_drop_old_data_col
 > metadataserver:0014_commission_result_rename_data_bin_col
 - Loading initial data for metadataserver.
Installed 1 object(s) from 1 fixture(s)
 * Restarting web server apache2       [80G  ... waiting .
[74G[ OK ]
squid-deb-proxy stop/waiting
squid-deb-proxy start/running, process 30073
maas-txlongpoll start/running, process 30139
maas-region-celery start/running, process 30200
Setting up maas (1.4+bzr1701+dfsg-0+1718+212~ppa0~ubuntu12.04.1) ...
Setting up maas-dns (1.4+bzr1701+dfsg-0+1718+212~ppa0~ubuntu12.04.1) ...
 * Stopping domain name service... bind9       [80G waiting for pid 26272 to die

[74G[ OK ]
 * Starting domain name service... bind9       [80G 
[74G[ OK ]
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
 -> Finished parsing the build-deps
adt-run: trace: & ubtree1t-maas-package-test: [----------------------------------------
adt-run: trace: & ubtree1t-maas-package-test:  - - - - - - - - - - results - - - - - - - - - -
ubtree1t-maas-package-test FAIL non-zero exit status 1
adt-run: trace: & ubtree1t-maas-package-test:  - - - - - - - - - - stdout - - - - - - - - - -
Ignoring indexes: http://pypi.python.org/simple/
Downloading/unpacking nose-timer
  Running setup.py egg_info for package nose-timer
    
Installing collected packages: nose-timer
  Running setup.py install for nose-timer
    
Successfully installed nose-timer
Cleaning up...
maas-integration.TestMAASIntegration.test_create_admin ... ok
maas-integration.TestMAASIntegration.test_restart_dbus_avahi ... ok
maas-integration.TestMAASIntegration.test_update_maas_url ... ok
maas-integration.TestMAASIntegration.test_check_initial_services ... ok
maas-integration.TestMAASIntegration.test_update_pxe_config ... ok
maas-integration.TestMAASIntegration.test_import_pxe_files ... ok
maas-integration.TestMAASIntegration.test_update_preseed_arm ... ok
maas-integration.TestMAASIntegration.test_login_api ... ok
maas-integration.TestMAASIntegration.test_set_http_proxy ... SKIP: Not testing proxy.
maas-integration.TestMAASIntegration.test_cluster_connected ... ok
maas-integration.TestMAASIntegration.test_set_up_dhcp_region ... ok
maas-integration.TestMAASIntegration.test_check_dhcp_service ... ok
maas-integration.TestMAASIntegration.test_set_up_dhcp_cluster ... SKIP: Not testing cluster controller
maas-integration.TestMAASIntegration.test_update_dns_config ... ok
maas-integration.TestMAASIntegration.test_boot_nodes_enlist ... ok
maas-integration.TestMAASIntegration.test_check_nodes_declared ... ok
maas-integration.TestMAASIntegration.test_set_nodes_ipmi_config ... ok
maas-integration.TestMAASIntegration.test_commission_nodes ... ok
maas-integration.TestMAASIntegration.test_check_nodes_ready ... ok
maas-integration.TestMAASIntegration.test_apply_tag_to_all_nodes ... ok
maas-integration.TestMAASIntegration.test_check_tag_applied_to_all_nodes ... ok
maas-integration.TestMAASIntegration.test_configure_juju ... ok
maas-integration.TestMAASIntegration.test_juju_bootstrap ... ok
maas-integration.TestMAASIntegration.test_juju_setup ... ok
maas-integration.TestMAASIntegration.test_juju_deploy_mediawiki ... ok
maas-integration.TestMAASIntegration.test_mediawiki_homepage ... ERROR
SKIP: Not testing Cluster controller

======================================================================
ERROR: maas-integration.TestMAASIntegration.test_mediawiki_homepage
----------------------------------------------------------------------
_StringException: juju --debug status: {{{
2013-12-11 00:15:16 DEBUG juju.environs open.go:75 ConfigForName found bootstrap config map[string]interface {}{"ca-cert":"-----BEGIN CERTIFICATE-----\nMIICVzCCAcKgAwIBAgIBADALBgkqhkiG9w0BAQUwQjENMAsGA1UEChMEanVqdTEx\nMC8GA1UEAwwoanVqdS1nZW5lcmF0ZWQgQ0EgZm9yIGVudmlyb25tZW50ICJtYWFz\nIjAeFw0xMzEyMTAyMzQxMjJaFw0yMzEyMTAyMzQ2MjJaMEIxDTALBgNVBAoTBGp1\nanUxMTAvBgNVBAMMKGp1anUtZ2VuZXJhdGVkIENBIGZvciBlbnZpcm9ubWVudCAi\nbWFhcyIwgZ0wCwYJKoZIhvcNAQEBA4GNADCBiQKBgQCyOxvAEm1spra4dAVnHIJt\nGzMBTrRppaqFzsuRlVG6s18ExSvxdMwSiWMWsUWIfOKzMHWRHj8W3U0VtvxAc3A4\ntdt47ijLl2kExCBdfQ9HyxBE06q2RcKsFx1U2AQOa0/GaIAyyDrE1rMezr/XfKG2\n3+yQcaFQQXuZhB7NztGNewIDAQABo2MwYTAOBgNVHQ8BAf8EBAMCAKQwDwYDVR0T\nAQH/BAUwAwEB/zAdBgNVHQ4EFgQUAVa/B/DWQZstPkiUYrxBSVrzbc4wHwYDVR0j\nBBgwFoAUAVa/B/DWQZstPkiUYrxBSVrzbc4wCwYJKoZIhvcNAQEFA4GBAFW0uGPo\nZk4U+GVSrtP6YCsovbQUVXVcZe4G+QbTqnWnZLB7XMfPNEnkxxUwSMilFJ3QADDU\nNZGTLSS0YBD7dJmCMSLPnXSKaQDoWXgNq7/nNp0HOm0UOX8MnejvK5vCwlzVrSqc\nVa1QAy9Qkql+n/fQLnErJvXEMcXCGXjiSYl7\n-----END CERTIFICATE-----\n", "default-series":"precise", "development":false, "image-metadata-url":"", "admin-secret":"nothing", "logging-config":"<root>=DEBUG", "maas-agent-name":"b486a2aa-92f8-486a-8df2-d77376e6290a", "maas-server":"http://192.168.21.5:80/MAAS";, "name":"maas", "state-port":37017, "tools-url":"", "api-port":17070, "authorized-keys":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCqvWKfCoQ9xZD0hwbg+72uLPA/79W2S++UyTqlAdQv3tCYrY4ZZksYEYdR1d5HTxh0Fvt3qF9/e/e+KZaQsRTcGx6berv5/HEYxfTwW71mqu6lfCp4UOvcv0nqR4MrXE8o8X0JBPaGxSo3VnEGisRLR+ulMAmlp5Rnorh07XHIRorHz0HMCXJeaVccqhwrDIfed8YawOzH151Hr0K7B3sOAzSabDOB/DD5cmOwdT/bq5lDnoX3Crck2TgVSODL7+eRQ0SkMBllWdExHWgIwYy/vymE5UILlfFgei+T23K6q62R8fiW7NcjhAtIy+vRk8Q2RfHnMzAyHjeq9TIOP+8n root@autopkgtest\n", "ca-private-key":"-----BEGIN RSA PRIVATE KEY-----\nMIICXAIBAAKBgQCyOxvAEm1spra4dAVnHIJtGzMBTrRppaqFzsuRlVG6s18ExSvx\ndMwSiWMWsUWIfOKzMHWRHj8W3U0VtvxAc3A4tdt47ijLl2kExCBdfQ9HyxBE06q2\nRcKsFx1U2AQOa0/GaIAyyDrE1rMezr/XfKG23+yQcaFQQXuZhB7NztGNewIDAQAB\nAoGAQVyP5rZbTrcs1yQlPxUjd6O75sTYQM4yZPPbj740xMU3GWa9oMT2ee3MWuwT\n1eQGtzQUSW3QMTBjLo12oKQ1CNpsliDYUyu5ftJyYroX3GmYwZQRIAYSP/CkWwLs\nf7dEwJ/AmJD4gTnVX2vwvu2DTOw7UU0UIiBAscQe5RE1f3ECQQDVAzm7IPYg1rTR\niCfRmdlqbwkepXhDfSsfhmtwFHfOUkUAxYi8Gc0OUolmNE7jKpaQkWI/mg5fE56f\nNra7DzyfAkEA1jL4m1N1uezNYZeqIQZJh14oi/utF5JdjUMmxEpPyGE6CMScA2IR\neIFgipB87C7cfFiLONn0OvRjkbXzf/OlpQJAIgEX0DWc1PUxfFl98cnbmVAMZqYH\nos5yed/detDppzge59ag01ZTE7w9TuF+zmjYDFAJHRocvTTGl6gGV6pBBQJAObqt\nFDfPL9fplpborFPkd05iQTAIe6Tf/9WMWjTpAclqebh7j9HbgY3k9hNxB7qtg/nd\nM+pSD52URE40nVSOXQJBALDm+hXIQwRe4h3oWk66UCJ6ht6kducIKyumKoXsL9Xk\n+/psiqU+SV7pzX3FEEFe37bnxQUSmKBUKVvBop/LJ1A=\n-----END RSA PRIVATE KEY-----\n", "firewall-mode":"instance", "maas-oauth":"w3uYdhPkkjYSxCz7f7:bSb92mNEzWsT6RQV3L:YqUet7wN62RyH2H2JeMQrafJyy9EWMUe", "ssl-hostname-verification":true, "type":"maas"}
2013-12-11 00:15:16 DEBUG juju.provider.maas environprovider.go:33 opening environment "maas".
2013-12-11 00:15:16 DEBUG juju state.go:160 waiting for DNS name(s) of state server instances [/MAAS/api/1.0/nodes/node-b3c88c84-61f4-11e3-bd33-525400123456/]
2013-12-11 00:15:16 INFO juju.state open.go:68 opening state; mongo addresses: ["8p8hk.master:37017"]; entity ""
2013-12-11 00:15:16 INFO juju.state open.go:106 connection established
2013-12-11 00:15:16 INFO juju supercommand.go:286 command finished
}}}

juju status: {{{{'environment': 'maas', 'services': {'mediawiki': {'units': {'mediawiki/0': {'agent-state': 'started', 'machine': '2', 'public-address': '94pf6.master', 'agent-version': '1.16.3.1', 'open-ports': ['80/tcp']}, 'mediawiki/1': {'agent-state': 'started', 'machine': '3', 'public-address': 'm4b67.master', 'agent-version': '1.16.3.1', 'open-ports': ['80/tcp']}}, 'charm': 'cs:precise/mediawiki-10', 'relations': {'db': ['mysql']}, 'exposed': True}, 'mysql': {'units': {'mysql/0': {'agent-state': 'started', 'machine': '1', 'public-address': '6c73y.master', 'agent-version': '1.16.3.1'}}, 'charm': 'cs:precise/mysql-29', 'relations': {'cluster': ['mysql'], 'db': ['mediawiki']}, 'exposed': False}}, 'machines': {'1': {'agent-state': 'started', 'series': 'precise', 'agent-version': '1.16.3.1', 'instance-id': '/MAAS/api/1.0/nodes/node-b3e76258-61f4-11e3-bd33-525400123456/', 'dns-name': '6c73y.master'}, '0': {'agent-state': 'started', 'series': 'precise', 'agent-version': '1.16.3.1', 'instance-id': '/MAAS/api/1.0/nodes/node-b3c88c84-61f4-11e3-bd33-525400123456/', 'dns-name': '8p8hk.master'}, '3': {'agent-state': 'started', 'series': 'precise', 'agent-version': '1.16.3.1', 'instance-id': '/MAAS/api/1.0/nodes/node-b3bd8014-61f4-11e3-a6e4-525400123456/', 'dns-name': 'm4b67.master'}, '2': {'agent-state': 'started', 'series': 'precise', 'agent-version': '1.16.3.1', 'instance-id': '/MAAS/api/1.0/nodes/node-b3de3e08-61f4-11e3-a6e4-525400123456/', 'dns-name': '94pf6.master'}}}}}}

Traceback (most recent call last):
  File "/tmp/tmp.omCzBqg9MN/ubtree1-ubtree/debian/tests/utils.py", line 55, in wrapper
    result = func(*args, **kwargs)
  File "/tmp/tmp.omCzBqg9MN/ubtree1-ubtree/debian/tests/maas-integration.py", line 686, in test_mediawiki_homepage
    sleep(15)
  File "/tmp/tmp.omCzBqg9MN/ubtree1-ubtree/debian/tests/utils.py", line 49, in _handle_timeout
    raise TimeoutError(error_message)
TimeoutError: Timer expired


maas-integration.TestMAASIntegration.test_update_pxe_config: 0.0003s
maas-integration.TestMAASIntegration.test_check_dhcp_service: 0.0163s
maas-integration.TestMAASIntegration.test_check_initial_services: 0.0237s
maas-integration.TestMAASIntegration.test_update_preseed_arm: 0.0434s
maas-integration.TestMAASIntegration.test_create_admin: 0.1443s
maas-integration.TestMAASIntegration.test_restart_dbus_avahi: 0.1563s
maas-integration.TestMAASIntegration.test_cluster_connected: 0.2162s
maas-integration.TestMAASIntegration.test_apply_tag_to_all_nodes: 0.2211s
maas-integration.TestMAASIntegration.test_boot_nodes_enlist: 0.4594s
maas-integration.TestMAASIntegration.test_commission_nodes: 0.4879s
maas-integration.TestMAASIntegration.test_update_maas_url: 1.1367s
maas-integration.TestMAASIntegration.test_update_dns_config: 1.2841s
maas-integration.TestMAASIntegration.test_set_nodes_ipmi_config: 2.5146s
maas-integration.TestMAASIntegration.test_login_api: 3.0186s
maas-integration.TestMAASIntegration.test_set_up_dhcp_region: 4.7288s
maas-integration.TestMAASIntegration.test_check_tag_applied_to_all_nodes: 5.4600s
maas-integration.TestMAASIntegration.test_configure_juju: 5.9493s
maas-integration.TestMAASIntegration.test_juju_setup: 6.2841s
maas-integration.TestMAASIntegration.test_check_nodes_declared: 156.8663s
maas-integration.TestMAASIntegration.test_check_nodes_ready: 198.8640s
maas-integration.TestMAASIntegration.test_mediawiki_homepage: 360.0360s
maas-integration.TestMAASIntegration.test_import_pxe_files: 489.6766s
maas-integration.TestMAASIntegration.test_juju_deploy_mediawiki: 681.5572s
maas-integration.TestMAASIntegration.test_juju_bootstrap: 716.7447s
----------------------------------------------------------------------
Ran 26 tests in 2635.915s

FAILED (SKIP=3, errors=1)
adt-run: trace: & ubtree1t-maas-package-test: ----------------------------------------]
adt-run: trace1: ** needs_reset, previously=False
adt-run: trace: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ tests done.
adt-run: trace: ** stop
adt-run: trace: ** close, scratch=tb-scratch~/tmp/tmp.omCzBqg9MN:-/|/tmp/tmp.omCzBqg9MN/!
+ RC=4
+ [ 4 -eq 20 ]
+ [ 0 -eq 1 ]
+ [ -x /home/ubuntu/adt-export-result ]
+ RES=PASS
+ [ 4 -gt 0 ]
+ RES=FAIL
+ /home/ubuntu/adt-export-result -D /root/adt-log maas FAIL
+ chown -R ubuntu /root/adt-log /var/tmp/testresults
+ chmod og+r /var/log/syslog
+ ls /var/crash/
+ [ -n  ]
+ exit 4
Connection to localhost closed.
+ RET=4
+ [ 0 -eq 1 ]
+ [ 4 -gt 0 ]
+ log_failure_msg adt-run exited with status 4.
+ log_msg Failure: adt-run exited with status 4.\n
+ date +%F %X
+ printf 2013-12-10 19:15:33: Failure: adt-run exited with status 4.\n
2013-12-10 19:15:33: Failure: adt-run exited with status 4.
+ [ 0 -eq 0 ]
+ mkdir -p <http://d-jenkins.ubuntu-ci:8080/job/precise-adt-maas-daily/./ARCH=amd64,label=maas-lenovo-lab/ws/results>
+ ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o CheckHostIP=no -i /var/tmp/adt/disks/adtkey -p 54323 -tt -o BatchMode=yes -l ubuntu localhost sudo chown -R ubuntu /root/adt-log; find /root/adt-log -type f -empty | xargs rm 2>/dev/null
Warning: Permanently added '[localhost]:54323' (ECDSA) to the list of known hosts.
find: `/root/adt-log': Permission denied
Connection to localhost closed.
+ true
+ scp -r -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o CheckHostIP=no -i /var/tmp/adt/disks/adtkey -P 54323 ubuntu@localhost:/root/adt-log/* /var/crash/*crash /var/log/syslog /var/tmp/testresults <http://d-jenkins.ubuntu-ci:8080/job/precise-adt-maas-daily/./ARCH=amd64,label=maas-lenovo-lab/ws/results>
+ true
+ log_info_msg Test artifacts copied to <http://d-jenkins.ubuntu-ci:8080/job/precise-adt-maas-daily/./ARCH=amd64,label=maas-lenovo-lab/ws/results>
+ log_msg Info: Test artifacts copied to <http://d-jenkins.ubuntu-ci:8080/job/precise-adt-maas-daily/./ARCH=amd64,label=maas-lenovo-lab/ws/results\n>
+ date +%F %X
+ printf 2013-12-10 19:15:36: Info: Test artifacts copied to <http://d-jenkins.ubuntu-ci:8080/job/precise-adt-maas-daily/./ARCH=amd64,label=maas-lenovo-lab/ws/results\n>
2013-12-10 19:15:36: Info: Test artifacts copied to <http://d-jenkins.ubuntu-ci:8080/job/precise-adt-maas-daily/./ARCH=amd64,label=maas-lenovo-lab/ws/results>
+ [ -f <http://d-jenkins.ubuntu-ci:8080/job/precise-adt-maas-daily/./ARCH=amd64,label=maas-lenovo-lab/311/artifact/results/summary.log> ]
+ [ -n 2013-12-10_23-26-15 ]
+ ls -tr <http://d-jenkins.ubuntu-ci:8080/job/precise-adt-maas-daily/./ARCH=amd64,label=maas-lenovo-lab/ws/results/*.result>
+ tail -1
+ resfile=
+ [ -n  ]
+ log_failure_msg Test didn't end normally. Generating error file
+ log_msg Failure: Test didn't end normally. Generating error file\n
+ date +%F %X
+ printf 2013-12-10 19:15:36: Failure: Test didn't end normally. Generating error file\n
2013-12-10 19:15:36: Failure: Test didn't end normally. Generating error file
+ date +%Y%m%d-%H%M%S
+ errfile=<http://d-jenkins.ubuntu-ci:8080/job/precise-adt-maas-daily/./ARCH=amd64,label=maas-lenovo-lab/ws/results/precise_amd64_maas_20131210-191536.error>
+ echo precise amd64 maas
+ rsync -a <http://d-jenkins.ubuntu-ci:8080/job/precise-adt-maas-daily/./ARCH=amd64,label=maas-lenovo-lab/ws/results/precise_amd64_maas_20131210-191536.error> /precise/tmp/
rsync: mkdir "/precise/tmp" failed: No such file or directory (2)
rsync error: error in file IO (code 11) at main.c(674) [Receiver=3.1.0]
+ true
+ [ 0 -eq 0 ]
+ ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o CheckHostIP=no -i /var/tmp/adt/disks/adtkey -p 54323 -tt -o BatchMode=yes -l ubuntu localhost sudo poweroff
Warning: Permanently added '[localhost]:54323' (ECDSA) to the list of known hosts.


Broadcast message from ubuntu@autopkgtest

	(/dev/pts/1) at 19:15 ...




The system is going down for power off NOW!

Connection to localhost closed.
+ exit 4
+ on_exit
+ log_info_msg Cleaning up
+ log_msg Info: Cleaning up\n
+ date +%F %X
+ printf 2013-12-10 19:15:36: Info: Cleaning up\n
2013-12-10 19:15:36: Info: Cleaning up
+ [ -f /var/tmp/adt/disks/run/precise-amd64-maas-20131210_182615.L18VTo.img.pid ]
+ cat /var/tmp/adt/disks/run/precise-amd64-maas-20131210_182615.L18VTo.img.pid
+ kill -9 5819
+ rm -f /var/tmp/adt/disks/run/precise-amd64-maas-20131210_182615.L18VTo.img.pid
+ rm -f /var/tmp/adt/disks/run/precise-amd64-maas-20131210_182615.L18VTo.img
+ rm -f /var/tmp/adt/disks/run/precise-amd64-maas-20131210_182615.L18VTo.img.monitor /var/tmp/adt/disks/run/precise-amd64-maas-20131210_182615.L18VTo.img.serial
+ rm -f /var/lock/adt/ssh.54323.lock
+ rm -f /var/lock/adt/vnc.5911.lock
+ [ -d /tmp/adt-amd64.I0vIej ]
+ rm -Rf /tmp/adt-amd64.I0vIej
+ rm -f /var/tmp/adt/disks/run/precise-amd64-maas-*.img*
+ find /var/lock/adt -name *.lock -mtime +1
+ exit 4
Build step 'Execute shell' marked build as failure
Archiving artifacts
Follow ups

Jenkins build is back to normal : precise-adt-maas-daily » amd64,maas-lenovo-lab #312
From: Jenkins Notification, 2013-12-12
References

Build failed in Jenkins: precise-adt-maas-daily » amd64,maas-lenovo-lab #310
From: Jenkins Notification, 2013-12-10