maas-builds team mailing list archive

Thread
Date
Jenkins Failure - precise-adt-maas-daily 300 - Built with (revno: ) and lp:~maas-maintainers/maas/packaging.precise.sru (revno: )

To: maas-builds@xxxxxxxxxxxxxxxxxxx
From: Jenkins Notification <devnull@xxxxxxxxxxxxx>
Date: Thu, 11 Sep 2014 01:04:11 +0000 (UTC)
Built with  (revno: ) and lp:~maas-maintainers/maas/packaging.precise.sru (revno: )

See http://d-jenkins.ubuntu-ci:8080/job/precise-adt-maas-daily/300/

[...truncated 3873 lines...]
 > maasserver:0037_nodegroup_cluster_name_unique
 > maasserver:0038_nodegroupinterface_ip_range_fix
 > maasserver:0039_add_filestorage_content
 > maasserver:0039_add_nodegroup_to_bootimage
 > maasserver:0040_make_filestorage_data_not_null
 > maasserver:0041_remove_filestorage_data
 > maasserver:0042_fix_039_conflict
 > maasserver:0043_unique_hostname_preparation
 > maasserver:0044_node_hostname_unique
 > maasserver:0045_add_tag_kernel_opts
 > maasserver:0046_add_nodegroup_maas_url
 > maasserver:0047_add_owner_to_filestorage
 > maasserver:0048_add_key_to_filestorage
 > maasserver:0049_filestorage_key_unique
 > maasserver:0050_shared_to_per_tenant_storage
 > maasserver:0051_bigger_distro_series_name
 > maasserver:0052_add_node_storage
 > maasserver:0053_node_routers
 > maasserver:0054_download_progress
 > maasserver:0055_nullable_bytes_downloaded
 > maasserver:0056_netboot_off_for_allocated_nodes
 > maasserver:0057_remove_hardware_details
 > maasserver:0058_add_agent_name_to_node
 - Loading initial data for maasserver.
Installed 0 object(s) from 0 fixture(s)
Running migrations for metadataserver:
 - Migrating forwards to 0014_commission_result_rename_data_bin_col.
 > metadataserver:0001_initial
 > metadataserver:0002_add_nodecommissionresult
 > metadataserver:0003_populate_hardware_details
 > metadataserver:0004_add_commissioningscript
 > metadataserver:0005_nodecommissionresult_add_timestamp
 > metadataserver:0006_nodecommissionresult_add_status
 > metadataserver:0007_nodecommissionresult_change_name_size
 > metadataserver:0008_rename_lshw_commissioning_output
 > metadataserver:0009_delete_status
 > metadataserver:0010_add_script_result
 > metadataserver:0011_commission_result_binary_data_col
 > metadataserver:0012_commission_result_binary_data_recode
 > metadataserver:0013_commission_result_drop_old_data_col
 > metadataserver:0014_commission_result_rename_data_bin_col
 - Loading initial data for metadataserver.
Installed 1 object(s) from 1 fixture(s)
 * Restarting web server apache2       [80G  ... waiting .
[74G[ OK ]
squid-deb-proxy stop/waiting
squid-deb-proxy start/running, process 6801
maas-txlongpoll start/running, process 6867
maas-region-celery start/running, process 6928
Setting up maas (1.4+bzr1693+dfsg-0ubuntu2.3~ctools0) ...
Setting up maas-dns (1.4+bzr1693+dfsg-0ubuntu2.3~ctools0) ...
 * Stopping domain name service... bind9       [80G waiting for pid 2997 to die

[74G[ OK ]
 * Starting domain name service... bind9       [80G 
[74G[ OK ]
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
 -> Finished parsing the build-deps
adt-run1: shared downtmp with FIFO supported, showing test output in realtime
adt-run: & ubtree0t-maas-package-test: [----------------------------------------
Ignoring indexes: http://pypi.python.org/simple/
Downloading/unpacking nose-timer
  Running setup.py egg_info for package nose-timer
    
Installing collected packages: nose-timer
  Running setup.py install for nose-timer
    
Successfully installed nose-timer
Cleaning up...
maas-integration.TestMAASIntegration.test_create_admin ... ok
maas-integration.TestMAASIntegration.test_restart_dbus_avahi ... ok
maas-integration.TestMAASIntegration.test_update_maas_url ... ok
maas-integration.TestMAASIntegration.test_restart_provisioning_server ... ok
maas-integration.TestMAASIntegration.test_check_initial_services ... ok
maas-integration.TestMAASIntegration.test_update_preseed_arm ... ok
maas-integration.TestMAASIntegration.test_login_api ... ok
maas-integration.TestMAASIntegration.test_set_http_proxy ... ok
maas-integration.TestMAASIntegration.test_cluster_connected ... ok
maas-integration.TestMAASIntegration.test_set_up_dhcp_region ... ok
maas-integration.TestMAASIntegration.test_check_dhcp_service ... ok
maas-integration.TestMAASIntegration.test_set_up_dhcp_cluster ... SKIP: Not testing cluster controller
maas-integration.TestMAASIntegration.test_update_dns_config ... ok
maas-integration.TestMAASIntegration.test_add_new_zones ... SKIP: Zone feature only available after 1.5
maas-integration.TestMAASIntegration.test_list_zones ... SKIP: Zone feature only available after 1.5
maas-integration.TestMAASIntegration.test_delete_zone ... SKIP: Zone feature only available after 1.5
maas-integration.TestMAASIntegration.test_import_boot_images ... ok
maas-integration.TestMAASIntegration.test_imported_boot_images ... SKIP: Don't check imported images on older releases.
maas-integration.TestMAASIntegration.test_boot_nodes_enlist ... ok
maas-integration.TestMAASIntegration.test_check_nodes_declared ... ok
maas-integration.TestMAASIntegration.test_assign_nodes_to_test_zone ... SKIP: Zone feature only available after 1.5
maas-integration.TestMAASIntegration.test_set_nodes_ipmi_config ... ok
maas-integration.TestMAASIntegration.test_commission_nodes ... ok
maas-integration.TestMAASIntegration.test_check_nodes_ready ... ok
maas-integration.TestMAASIntegration.test_apply_tag_to_all_nodes ... ok
maas-integration.TestMAASIntegration.test_check_tag_applied_to_all_nodes ... ok
maas-integration.TestMAASIntegration.test_configure_juju ... ok
maas-integration.TestMAASIntegration.test_juju_bootstrap ... ok
maas-integration.TestMAASIntegration.test_juju_setup ... ok
maas-integration.TestMAASIntegration.test_juju_deploy_mediawiki ... ok
maas-integration.TestMAASIntegration.test_mediawiki_homepage ... ERROR
SKIP: Not testing Cluster controller

======================================================================
ERROR: maas-integration.TestMAASIntegration.test_mediawiki_homepage
----------------------------------------------------------------------
_StringException: juju --debug status: {{{
2014-09-11 01:03:58 DEBUG juju.environs open.go:75 ConfigForName found bootstrap config map[string]interface {}{"name":"maas", "state-port":37017, "tools-url":"", "api-port":17070, "authorized-keys":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvfKX/DiMpXTwn8wPs3ETo4CXoFn0LBOd5DM8CO7asfXe+aYjxR6Qkc0dSh82+gR3Tsxtp69smi3ERMbfRpiCsMNxxI6UsGGvmBpcv8fK873MLmSF7iDEPKLxQoW3NPbccqb5DMLU3uBBANNInqbvOnP546e0zsghTobpUi7LRsxsvZfwZMLllGa/VUSAaY2v8Ljb4/Ect9ZSn71+eypTADfz/QMM2ywCsOJAOHSu+iMI9nNRJst5FHoBYLQhgd+FKlG6kr0QhZXet7KTlwdHDLluiyld+AEl6Rgr0vTYq6arTqi+mEKiV1VLnShTJjOT6RU7GXQyPV9PlSi89Wonr root@autopkgtest\n", "bootstrap-timeout":18000, "ca-private-key":"-----BEGIN RSA PRIVATE KEY-----\nMIICWwIBAAKBgQChRRAT1nLdfW0CByqPNrgOmO3Wc7VYLWkKkYebW3lypbzLq5wA\nzWtR1erErBZkLTh5lkqOnqWHeZPyHKQQXditbd3izP14Wt5pGDdDvPuuxDgUUM3q\noje8DF25cbg6J52FsakPpRIOoHxmPVg2t/BhCwubo9Jo4vhEgWzWpilv6wIDAQAB\nAoGAM9pQ4RACvtSlSBGlzYZ6h7cEQf/SPxfavlssjvdKhWJk4de7wUZLS3TMl4/9\nYxhnIDECp/i6oGSm/SHYXqrOD/KEcgm96QBbGVLvM9/89+1SLgxgRbHU3+R2z8Xx\n87sR8ESNtWyOatJm4+igwCzg4bb1a3deC2xtWFKZBsd6BQECQQDB0HZwQe2TGf3P\nPpPfbA+agDe5NigePQik8m3vgdm3Doz6FCkV3T2LEr/YUIoqUfoj0PznI8cCBSm7\nZagAQRETAkEA1QN0WphTZxwpaemjC9BBJt/EHh/+8UmZPlLAXwIqPlbaHtKXtdds\nlCDf7/S2TRJFLhdoWW6lo5zvnyKroQnYyQJAN8BjkX8le/8qB4eUudw30NW5Mcnm\nWGxuJCSogxDzM7k3zGgoOWXe4jaOgQcJ94qFTfA8D7E2+dnKW2GupgQRiQJAXUhx\n/+qNO65dhIBIYrgEkB2WKQsh/lCYx249yF2OWlZ2luDqhzub1RLAfrjPzNifaxvd\nGq0V5JypYysvtgo14QJANeg+X8srAuMMWLDqv1HFgzZGYWY01O6AgW+94R7MDvkD\nwTpwY3Fq5+ZQtAfWUmUTTMGLJzGFrJEOVXitago1hA==\n-----END RSA PRIVATE KEY-----\n", "firewall-mode":"instance", "maas-oauth":"qwEUc4C2HdQwEJyyDm:ecGWJ9MUkj5MWFtNhp:7Cn5Jgy4W3gXxXpF9jx7kBbKbVLnddHM", "ssl-hostname-verification":true, "type":"maas", "ca-cert":"-----BEGIN CERTIFICATE-----\nMIICVzCCAcKgAwIBAgIBADALBgkqhkiG9w0BAQUwQjENMAsGA1UEChMEanVqdTEx\nMC8GA1UEAwwoanVqdS1nZW5lcmF0ZWQgQ0EgZm9yIGVudmlyb25tZW50ICJtYWFz\nIjAeFw0xNDA5MTEwMDE3NTdaFw0yNDA5MTEwMDIyNTdaMEIxDTALBgNVBAoTBGp1\nanUxMTAvBgNVBAMMKGp1anUtZ2VuZXJhdGVkIENBIGZvciBlbnZpcm9ubWVudCAi\nbWFhcyIwgZ0wCwYJKoZIhvcNAQEBA4GNADCBiQKBgQChRRAT1nLdfW0CByqPNrgO\nmO3Wc7VYLWkKkYebW3lypbzLq5wAzWtR1erErBZkLTh5lkqOnqWHeZPyHKQQXdit\nbd3izP14Wt5pGDdDvPuuxDgUUM3qoje8DF25cbg6J52FsakPpRIOoHxmPVg2t/Bh\nCwubo9Jo4vhEgWzWpilv6wIDAQABo2MwYTAOBgNVHQ8BAf8EBAMCAKQwDwYDVR0T\nAQH/BAUwAwEB/zAdBgNVHQ4EFgQUIKUKUMs1UcBN1egw+zOZzG+8QbwwHwYDVR0j\nBBgwFoAUIKUKUMs1UcBN1egw+zOZzG+8QbwwCwYJKoZIhvcNAQEFA4GBAEus/BYW\niyautvHzcAu4EJ1yobb8sMhxW1fmML/yuFqsXf5PlHQyP7C4w4BHy0YVPV+jJenX\ng1B76M8N2D9KAshifc9GgPUafLSHL4swqEARj0vXIBM+7OVfHqeKBNrtX6gu9YBv\nlxm3tQG1guT54ohkWz5xH5l7Ys64UIdumPsW\n-----END CERTIFICATE-----\n", "default-series":"precise", "development":false, "image-metadata-url":"", "admin-secret":"nothing", "logging-config":"<root>=DEBUG", "maas-agent-name":"38f49d4f-02dc-4d76-8c98-76956bb4077a", "maas-server":"http://192.168.21.5:80/MAAS"}
2014-09-11 01:03:58 DEBUG juju.provider.maas environprovider.go:33 opening environment "maas".
2014-09-11 01:03:58 WARNING juju.environs.config config.go:604 unknown config field "bootstrap-timeout"
2014-09-11 01:03:58 WARNING juju.environs.config config.go:604 unknown config field "bootstrap-timeout"
2014-09-11 01:03:59 DEBUG juju state.go:160 waiting for DNS name(s) of state server instances [/MAAS/api/1.0/nodes/node-25398c70-3949-11e4-8351-525400123456/]
2014-09-11 01:03:59 INFO juju.state open.go:68 opening state; mongo addresses: ["7nwak.master:37017"]; entity ""
2014-09-11 01:03:59 INFO juju.state open.go:106 connection established
2014-09-11 01:03:59 WARNING juju.environs.config config.go:604 unknown config field "bootstrap-timeout"
2014-09-11 01:03:59 INFO juju supercommand.go:286 command finished
}}}

juju status: {{{{'environment': 'maas', 'services': {'mediawiki': {'units': {'mediawiki/0': {'agent-state': 'started', 'machine': '2', 'public-address': 'gjb6q.master', 'agent-version': '1.16.3.1', 'open-ports': ['80/tcp']}, 'mediawiki/1': {'agent-state': 'started', 'machine': '3', 'public-address': 'kjqaq.master', 'agent-version': '1.16.3.1', 'open-ports': ['80/tcp']}}, 'charm': 'cs:precise/mediawiki-18', 'relations': {'db': ['mysql']}, 'exposed': True}, 'mysql': {'units': {'mysql/0': {'agent-state': 'started', 'machine': '1', 'public-address': 'kmc9k.master', 'agent-version': '1.16.3.1'}}, 'charm': 'cs:precise/mysql-48', 'relations': {'cluster': ['mysql'], 'db': ['mediawiki']}, 'exposed': False}}, 'machines': {'1': {'agent-state': 'started', 'series': 'precise', 'agent-version': '1.16.3.1', 'instance-id': '/MAAS/api/1.0/nodes/node-25399c74-3949-11e4-be4f-525400123456/', 'dns-name': 'kmc9k.master'}, '0': {'agent-state': 'started', 'series': 'precise', 'agent-version': '1.16.3.1', 'instance-id': '/MAAS/api/1.0/nodes/node-25398c70-3949-11e4-8351-525400123456/', 'dns-name': '7nwak.master'}, '3': {'agent-state': 'started', 'series': 'precise', 'agent-version': '1.16.3.1', 'instance-id': '/MAAS/api/1.0/nodes/node-255ac37c-3949-11e4-be4f-525400123456/', 'dns-name': 'kjqaq.master'}, '2': {'agent-state': 'started', 'series': 'precise', 'agent-version': '1.16.3.1', 'instance-id': '/MAAS/api/1.0/nodes/node-37fa23c4-3949-11e4-be4f-525400123456/', 'dns-name': 'gjb6q.master'}}}}}}

Traceback (most recent call last):
  File "/tmp/adt-run.FUBfUs/ubtree0-build/real-tree/debian/tests/utils.py", line 70, in wrapper
    result = func(*args, **kwargs)
  File "/tmp/adt-run.FUBfUs/ubtree0-build/real-tree/debian/tests/maas-integration.py", line 807, in test_mediawiki_homepage
    sleep(15)
  File "/tmp/adt-run.FUBfUs/ubtree0-build/real-tree/debian/tests/utils.py", line 64, in _handle_timeout
    raise TimeoutError(error_message)
TimeoutError: Timer expired


maas-integration.TestMAASIntegration.test_update_preseed_arm: 0.0002s
maas-integration.TestMAASIntegration.test_check_dhcp_service: 0.0131s
maas-integration.TestMAASIntegration.test_restart_provisioning_server: 0.0454s
maas-integration.TestMAASIntegration.test_check_initial_services: 0.0589s
maas-integration.TestMAASIntegration.test_create_admin: 0.1512s
maas-integration.TestMAASIntegration.test_restart_dbus_avahi: 0.1820s
maas-integration.TestMAASIntegration.test_apply_tag_to_all_nodes: 0.2298s
maas-integration.TestMAASIntegration.test_check_tag_applied_to_all_nodes: 0.2476s
maas-integration.TestMAASIntegration.test_commission_nodes: 0.4478s
maas-integration.TestMAASIntegration.test_set_http_proxy: 0.5220s
maas-integration.TestMAASIntegration.test_login_api: 0.8329s
maas-integration.TestMAASIntegration.test_update_dns_config: 1.0654s
maas-integration.TestMAASIntegration.test_update_maas_url: 1.1378s
maas-integration.TestMAASIntegration.test_set_nodes_ipmi_config: 2.5876s
maas-integration.TestMAASIntegration.test_boot_nodes_enlist: 3.5448s
maas-integration.TestMAASIntegration.test_configure_juju: 6.4114s
maas-integration.TestMAASIntegration.test_juju_setup: 6.5033s
maas-integration.TestMAASIntegration.test_set_up_dhcp_region: 12.6413s
maas-integration.TestMAASIntegration.test_cluster_connected: 30.8108s
maas-integration.TestMAASIntegration.test_check_nodes_declared: 155.6923s
maas-integration.TestMAASIntegration.test_check_nodes_ready: 199.3936s
maas-integration.TestMAASIntegration.test_mediawiki_homepage: 360.0271s
maas-integration.TestMAASIntegration.test_juju_deploy_mediawiki: 1052.8078s
maas-integration.TestMAASIntegration.test_juju_bootstrap: 1072.7827s
maas-integration.TestMAASIntegration.test_import_boot_images: 1712.7016s
----------------------------------------------------------------------
Ran 31 tests in 4621.644s

FAILED (SKIP=7, errors=1)
adt-run1: testbed executing test finished with exit status 1
adt-run: & ubtree0t-maas-package-test: ----------------------------------------]
ubtree0t-maas-package-test FAIL non-zero exit status 1
adt-run: & ubtree0t-maas-package-test:  - - - - - - - - - - results - - - - - - - - - -
adt-run1: ** needs_reset, previously=False, requested by run_tests() line 1177
adt-run1: ** needs_reset, previously=True, requested by process_actions() line 2129
adt-run: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ tests done.
adt-run1: ** stop
adt-run1: ** close, scratch=/tmp/adt-run.FUBfUs
+ RC=4
+ [ 4 -eq 20 ]
+ [ 0 -eq 1 ]
+ [ -x /home/ubuntu/adt-export-result ]
+ RES=PASS
+ [ 4 -gt 0 ]
+ RES=FAIL
+ /home/ubuntu/adt-export-result -D /root/adt-log maas FAIL
+ chown -R ubuntu /root/adt-log /var/tmp/testresults
+ chmod og+r /var/log/syslog
+ ls /var/crash/
+ [ -n  ]
+ exit 4
Connection to localhost closed.
+ RET=4
+ [ 0 -eq 1 ]
+ [ 4 -gt 0 ]
+ log_failure_msg adt-run exited with status 4.
+ log_msg Failure: adt-run exited with status 4.\n
+ date +%F %X
+ printf 2014-09-10 21:04:07: Failure: adt-run exited with status 4.\n
2014-09-10 21:04:07: Failure: adt-run exited with status 4.
+ [ 0 -eq 0 ]
+ mkdir -p /home/ubuntu/jenkins-jobs/workspace/precise-adt-maas-daily/results
+ ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o CheckHostIP=no -i /var/tmp/adt/disks/adtkey -p 54323 -tt -o BatchMode=yes -l ubuntu localhost sudo chown -R ubuntu /root/adt-log; find /root/adt-log -type f -empty | xargs rm 2>/dev/null
Warning: Permanently added '[localhost]:54323' (ECDSA) to the list of known hosts.
find: `/root/adt-log': Permission denied
Connection to localhost closed.
+ true
+ scp -r -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o CheckHostIP=no -i /var/tmp/adt/disks/adtkey -P 54323 ubuntu@localhost:/root/adt-log/* /var/crash/*crash /var/log/syslog /var/tmp/testresults /home/ubuntu/jenkins-jobs/workspace/precise-adt-maas-daily/results
+ true
+ log_info_msg Test artifacts copied to /home/ubuntu/jenkins-jobs/workspace/precise-adt-maas-daily/results
+ log_msg Info: Test artifacts copied to /home/ubuntu/jenkins-jobs/workspace/precise-adt-maas-daily/results\n
+ date +%F %X
+ printf 2014-09-10 21:04:10: Info: Test artifacts copied to /home/ubuntu/jenkins-jobs/workspace/precise-adt-maas-daily/results\n
2014-09-10 21:04:10: Info: Test artifacts copied to /home/ubuntu/jenkins-jobs/workspace/precise-adt-maas-daily/results
+ [ -f /home/ubuntu/jenkins-jobs/workspace/precise-adt-maas-daily/results/summary.log ]
+ [ -n 2014-09-10_23-41-10 ]
+ tail -1
+ ls -tr /home/ubuntu/jenkins-jobs/workspace/precise-adt-maas-daily/results/*.result
+ resfile=
+ [ -n  ]
+ log_failure_msg Test didn't end normally. Generating error file
+ log_msg Failure: Test didn't end normally. Generating error file\n
+ date +%F %X
+ printf 2014-09-10 21:04:10: Failure: Test didn't end normally. Generating error file\n
2014-09-10 21:04:10: Failure: Test didn't end normally. Generating error file
+ date +%Y%m%d-%H%M%S
+ errfile=/home/ubuntu/jenkins-jobs/workspace/precise-adt-maas-daily/results/precise_amd64_maas_20140910-210410.error
+ echo precise amd64 maas
+ rsync -a /home/ubuntu/jenkins-jobs/workspace/precise-adt-maas-daily/results/precise_amd64_maas_20140910-210410.error /precise/tmp/
rsync: mkdir "/precise/tmp" failed: No such file or directory (2)
rsync error: error in file IO (code 11) at main.c(674) [Receiver=3.1.0]
+ true
+ [ 0 -eq 0 ]
+ ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o CheckHostIP=no -i /var/tmp/adt/disks/adtkey -p 54323 -tt -o BatchMode=yes -l ubuntu localhost sudo poweroff
Warning: Permanently added '[localhost]:54323' (ECDSA) to the list of known hosts.
Connection to localhost closed.
+ exit 4
+ on_exit
+ log_info_msg Cleaning up
+ log_msg Info: Cleaning up\n
+ date +%F %X
+ printf 2014-09-10 21:04:11: Info: Cleaning up\n
2014-09-10 21:04:11: Info: Cleaning up
+ [ -f /var/tmp/adt/disks/run/precise-amd64-maas-20140910_194133.GaAfs9.img.pid ]
+ cat /var/tmp/adt/disks/run/precise-amd64-maas-20140910_194133.GaAfs9.img.pid
+ kill -9 19384
+ rm -f /var/tmp/adt/disks/run/precise-amd64-maas-20140910_194133.GaAfs9.img.pid
+ rm -f /var/tmp/adt/disks/run/precise-amd64-maas-20140910_194133.GaAfs9.img
+ rm -f /var/lock/adt/ssh.54323.lock
+ rm -f /var/lock/adt/vnc.5911.lock
+ [ -d /tmp/adt-amd64.hMBx8Y ]
+ rm -Rf /tmp/adt-amd64.hMBx8Y
+ rm -f /var/tmp/adt/disks/run/precise-amd64-maas-*.img*
+ find /var/lock/adt -name *.lock -mtime +1
+ exit 4
Build step 'Execute shell' marked build as failure
Archiving artifacts
Email was triggered for: Failure
Sending email for trigger: Failure
Follow ups

Jenkins Fixed - precise-adt-maas-daily 301 - Built with (revno: ) and lp:~maas-maintainers/maas/packaging.precise.sru (revno: )
From: Jenkins Notification, 2014-09-11