ius-coredev team mailing list archive
-
ius-coredev team
-
Mailing list archive
-
Message #00919
Re: Build failed in Jenkins: verify-mirrorlist #2941
Job added to delete files older than 1 day (24 hours):
0 4 * * * /usr/bin/find /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/ -type f -mtime +1 -exec rm -rf {} \;
---
Jeffrey Ness
Linux System Engineer
IT Operations [Development]
Rackspace Hosting & IUS Community
On Jun 13, 2011, at 7:30 AM, wrote:
> After clearing out the cache dmirr is back online
>
> ---
> Jeffrey Ness
> Linux System Engineer
> IT Operations [Development]
> Rackspace Hosting & IUS Community
>
> On Jun 13, 2011, at 7:22 AM, wrote:
>
>> Seems your cache is not being cleaned out
>>
>> [root@322729-dmirr01 ~]# tail /var/log/httpd/error_log
>> [Mon Jun 13 06:57:16 2011] [error] [client 91.66.183.42] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 13 06:57:16 2011] [error] [client 130.83.88.143] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 13 06:57:16 2011] [error] [client 212.147.152.3] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 13 06:57:18 2011] [error] [client 89.212.170.109] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 13 06:57:18 2011] [error] [client 208.185.244.250] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 13 06:57:18 2011] [error] [client 178.18.90.157] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 13 06:57:18 2011] [error] [client 88.149.188.106] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 13 06:57:20 2011] [error] [client 69.164.208.170] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 13 06:57:22 2011] [error] [client 174.121.150.229] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 13 06:58:09 2011] [warn] [client 173.203.5.222] Timeout waiting for output from CGI script /var/www/scripts/health_check.py
>>
>>
>>
>>
>> [root@322729-dmirr01 ~]# tail /var/log/httpd/error_log -f
>> [Mon Jun 13 07:12:32 2011] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
>> [Mon Jun 13 07:12:32 2011] [notice] Digest: generating secret for digest authentication ...
>> [Mon Jun 13 07:12:32 2011] [notice] Digest: done
>> [Mon Jun 13 07:12:32 2011] [notice] Apache/2.2.3 (Red Hat) configured -- resuming normal operations
>> [Mon Jun 13 07:12:40 2011] [error] /var/www/vhosts/dmirr.iuscommunity.org/dmirr-env/lib64/python2.6/site-packages/ToscaWidgets-0.9.8-py2.6.egg/tw/core/view.py:223: DeprecationWarning: object.__new__() takes no parameters
>> [Mon Jun 13 07:12:40 2011] [error] obj = object.__new__(cls, *args, **kw)
>> [Mon Jun 13 07:12:42 2011] [error] /var/www/vhosts/dmirr.iuscommunity.org/dmirr-env/lib64/python2.6/site-packages/TurboGears2-2.0.3-py2.6.egg/tg/configuration.py:501: DeprecationWarning: Cache options should start with cache. instead of cache_
>> [Mon Jun 13 07:12:42 2011] [error] app = CacheMiddleware(app, config)
>> [Mon Jun 13 07:12:43 2011] [error] /var/www/vhosts/dmirr.iuscommunity.org/dmirr-env/lib64/python2.6/site-packages/ToscaWidgets-0.9.8-py2.6.egg/tw/core/view.py:223: DeprecationWarning: object.__new__() takes no parameters
>> [Mon Jun 13 07:12:43 2011] [error] obj = object.__new__(cls, *args, **kw)
>> [Mon Jun 13 07:12:45 2011] [error] /var/www/vhosts/dmirr.iuscommunity.org/dmirr-env/lib64/python2.6/site-packages/TurboGears2-2.0.3-py2.6.egg/tg/configuration.py:501: DeprecationWarning: Cache options should start with cache. instead of cache_
>> [Mon Jun 13 07:12:45 2011] [error] app = CacheMiddleware(app, config)
>> [Mon Jun 13 07:12:47 2011] [error] /var/www/vhosts/dmirr.iuscommunity.org/dmirr-env/lib64/python2.6/site-packages/ToscaWidgets-0.9.8-py2.6.egg/tw/core/view.py:223: DeprecationWarning: object.__new__() takes no parameters
>> [Mon Jun 13 07:12:47 2011] [error] obj = object.__new__(cls, *args, **kw)
>> [Mon Jun 13 07:12:48 2011] [error] /var/www/vhosts/dmirr.iuscommunity.org/dmirr-env/lib64/python2.6/site-packages/TurboGears2-2.0.3-py2.6.egg/tg/configuration.py:501: DeprecationWarning: Cache options should start with cache. instead of cache_
>> [Mon Jun 13 07:12:48 2011] [error] app = CacheMiddleware(app, config)
>> [Mon Jun 13 07:12:51 2011] [error] /var/www/vhosts/dmirr.iuscommunity.org/dmirr-env/lib64/python2.6/site-packages/ToscaWidgets-0.9.8-py2.6.egg/tw/core/view.py:223: DeprecationWarning: object.__new__() takes no parameters
>> [Mon Jun 13 07:12:51 2011] [error] obj = object.__new__(cls, *args, **kw)
>> [Mon Jun 13 07:12:52 2011] [error] /var/www/vhosts/dmirr.iuscommunity.org/dmirr-env/lib64/python2.6/site-packages/TurboGears2-2.0.3-py2.6.egg/tg/configuration.py:501: DeprecationWarning: Cache options should start with cache. instead of cache_
>> [Mon Jun 13 07:12:52 2011] [error] app = CacheMiddleware(app, config)
>>
>>
>>
>> [root@322729-dmirr01 ~]# touch file
>> touch: cannot touch `file': No space left on device
>> [root@322729-dmirr01 ~]# df -h
>> Filesystem Size Used Avail Use% Mounted on
>> /dev/sda5 14G 11G 1.9G 86% /
>> /dev/sda2 2.0G 36M 1.9G 2% /tmp
>> /dev/sda1 99M 24M 71M 26% /boot
>> tmpfs 3.9G 0 3.9G 0% /dev/shm
>> [root@322729-dmirr01 ~]# lsof | grep -i 'deleted'
>> httpd 1965 root 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 1966 dmirr 14wW REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 1968 dmirr 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 1969 dmirr 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 1970 dmirr 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 1971 dmirr 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 1984 dmirr 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 1997 dmirr 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2009 dmirr 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2047 dmirr 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2048 dmirr 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2100 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2101 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2102 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2103 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2104 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2105 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2106 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2107 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2113 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2114 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2115 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2119 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2124 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2127 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2128 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2136 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2138 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2139 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2151 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2155 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2159 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2174 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2175 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2176 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2180 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2186 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2192 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2195 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2196 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2197 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2200 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2201 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> httpd 2208 apache 14w REG 8,5 0 3562339 /var/run/wsgi.1965.0.1.lock (deleted)
>> mysqld 3765 mysql 4u REG 8,5 0 195594 /var/lib/mysqltmp/ibS0qxmg (deleted)
>> mysqld 3765 mysql 5u REG 8,5 0 195595 /var/lib/mysqltmp/ibQXBmK3 (deleted)
>> mysqld 3765 mysql 6u REG 8,5 0 195596 /var/lib/mysqltmp/ibV70b8Q (deleted)
>> mysqld 3765 mysql 7u REG 8,5 0 195597 /var/lib/mysqltmp/ibiA22wE (deleted)
>> mysqld 3765 mysql 11u REG 8,5 0 195601 /var/lib/mysqltmp/ibQ7wd3r (deleted)
>>
>>
>> [root@322729-dmirr01 container_file]# du -h --max-depth=1 /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file
>> 441M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/2
>> 443M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/7
>> 443M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/b
>> 445M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/e
>> 444M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/3
>> 444M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/1
>> 443M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/f
>> 442M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/a
>> 441M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/4
>> 444M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/5
>> 446M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/6
>> 443M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/8
>> 444M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/c
>> 445M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/9
>> 442M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/0
>> 443M /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file/d
>> 7.0G /var/www/vhosts/dmirr.iuscommunity.org/dmirr/data/sessions/container_file
>> ---
>> Jeffrey Ness
>> Linux System Engineer
>> IT Operations [Development]
>> Rackspace Hosting & IUS Community
>>
>> On Jun 13, 2011, at 7:05 AM, <wdierkes@xxxxxxxxxxxxx>
>> <wdierkes@xxxxxxxxxxxxx> wrote:
>>
>>> See <http://jenkins.iuscommunity.org/job/verify-mirrorlist/2941/>
>>>
>>> ------------------------------------------
>>> Started by timer
>>> Building on master
>>> [workspace] $ /bin/sh -xe /tmp/hudson2542008538347314208.sh
>>> + /usr/bin/curl -s 'http://dmirr.iuscommunity.org/mirrorlist?repo=ius-el4&arch=i386'
>>> + grep ius/stable
>>> + exit 1
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~ius-coredev
>>> Post to : ius-coredev@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~ius-coredev
>>> More help : https://help.launchpad.net/ListHelp
>>
>
References
