← Back to team overview

ius-coredev team mailing list archive

  1. ius-coredev team
  2. Mailing list archive
  3. Message #00937

Re: Build failed in Jenkins: verify-mirrorlist #3104

  Thread Previous • Date Previous • • Thread Next 
Ok, dmirr is down again... You are going to need to look in to this as I'm not sure how to even debug a python webapp like this:

[Mon Jun 20 07:46:55 2011] [error]   app = CacheMiddleware(app, config)
[Mon Jun 20 07:54:58 2011] [warn] [client 173.203.5.222] Timeout waiting for output from CGI script /var/www/scripts/health_check.py
[Mon Jun 20 07:54:58 2011] [error] [client 173.203.5.222] Premature end of script headers: health_check.py
[Mon Jun 20 07:56:58 2011] [warn] [client 173.203.5.222] Timeout waiting for output from CGI script /var/www/scripts/health_check.py
[Mon Jun 20 07:57:31 2011] [warn] [client 64.39.5.123] Timeout waiting for output from CGI script /var/www/scripts/health_check.py
[Mon Jun 20 07:57:31 2011] [error] [client 64.39.5.123] Premature end of script headers: health_check.py
[Mon Jun 20 07:59:31 2011] [warn] [client 64.39.5.123] Timeout waiting for output from CGI script /var/www/scripts/health_check.py
[Mon Jun 20 08:11:54 2011] [warn] [client 173.203.5.222] Timeout waiting for output from CGI script /var/www/scripts/health_check.py
[Mon Jun 20 08:11:54 2011] [error] [client 173.203.5.222] Premature end of script headers: health_check.py
[Mon Jun 20 08:13:34 2011] [error] [client 8.20.213.37] Premature end of script headers: dmirr.wsgi
[Mon Jun 20 08:13:38 2011] [error] [client 8.20.213.27] Premature end of script headers: dmirr.wsgi
[Mon Jun 20 08:13:41 2011] [error] [client 8.19.240.68] Premature end of script headers: dmirr.wsgi
[Mon Jun 20 08:13:52 2011] [error] [client 200.131.9.130] Premature end of script headers: dmirr.wsgi
[Mon Jun 20 08:13:54 2011] [warn] [client 173.203.5.222] Timeout waiting for output from CGI script /var/www/scripts/health_check.py
[Mon Jun 20 08:13:55 2011] [error] [client 91.121.180.211] Premature end of script headers: dmirr.wsgi
[Mon Jun 20 08:15:28 2011] [error] [client 79.143.180.49] Premature end of script headers: dmirr.wsgi
[Mon Jun 20 08:15:46 2011] [error] [client 209.134.48.153] Premature end of script headers: dmirr.wsgi
[Mon Jun 20 08:15:53 2011] [error] [client 203.199.72.210] Premature end of script headers: dmirr.wsgi
[Mon Jun 20 08:15:57 2011] [error] [client 128.32.32.220] Premature end of script headers: dmirr.wsgi
[Mon Jun 20 08:16:06 2011] [error] [client 92.247.12.210] Premature end of script headers: dmirr.wsgi

---
Jeffrey Ness
Linux System Engineer
IT Operations [Development]
Rackspace Hosting & IUS Community

On Jun 20, 2011, at 7:48 AM, Jeff Ness wrote:

> Another reload and the app seems to be back up.
> 
> I'm going to keep the error_log parsing in the background to see if it happens again.
> 
> ---
> Jeffrey Ness
> Linux System Engineer
> IT Operations [Development]
> Rackspace Hosting & IUS Community
> 
> On Jun 20, 2011, at 7:45 AM, Jeff Ness wrote:
> 
>> I don't think it has anything to do with logrotate as its happened again....
>> 
>> [root@322729-dmirr01 ~]# tail /var/log/httpd/error_log
>> [Mon Jun 20 07:43:19 2011] [error] [client 209.17.170.226] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 20 07:43:33 2011] [error] [client 128.32.62.158] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 20 07:43:41 2011] [error] [client 72.3.236.90] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 20 07:43:44 2011] [error] [client 60.53.114.7] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 20 07:43:57 2011] [error] [client 193.191.154.73] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 20 07:43:58 2011] [error] [client 121.15.87.66] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 20 07:44:17 2011] [error] [client 137.118.0.254] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 20 07:44:19 2011] [error] [client 67.23.38.216] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 20 07:44:26 2011] [error] [client 128.61.143.100] Premature end of script headers: dmirr.wsgi
>> [Mon Jun 20 07:44:28 2011] [error] [client 85.214.44.166] Premature end of script headers: dmirr.wsgi
>> 
>> 
>> ---
>> Jeffrey Ness
>> Linux System Engineer
>> IT Operations [Development]
>> Rackspace Hosting & IUS Community
>> 
>> On Jun 20, 2011, at 7:26 AM, Bj Dierkes wrote:
>> 
>>> Sounds like maybe something to do with logrotate running on Sunday's.... Hmm.... Once MF is beta I'm going to revisit dMirr.
>>> 
>>> Sent from my iPhone
>>> 
>>> On Jun 20, 2011, at 7:12 AM, "Jeff Ness" <jeffrey.ness@xxxxxxxxxxxxx> wrote:
>>> 
>>>> Seems dmirr had issues again, this time not related to full disk space:
>>>> 
>>>> [Sun Jun 19 22:59:12 2011] [error]   obj = object.__new__(cls, *args, **kw)
>>>> [Sun Jun 19 22:59:13 2011] [error] /var/www/vhosts/dmirr.iuscommunity.org/dmirr-env/lib64/python2.6/site-packages/TurboGears2-2.0.3-py2.6.
>>>> egg/tg/configuration.py:501: DeprecationWarning: Cache options should start with cache. instead of cache_
>>>> [Sun Jun 19 22:59:13 2011] [error]   app = CacheMiddleware(app, config)
>>>> [Sun Jun 19 23:57:07 2011] [error] [client 66.249.72.122] Error - <class 'repoze.what.predicates.NotAuthorizedError'>: 
>>>> [Mon Jun 20 01:15:42 2011] [error] [client 66.249.72.122] Error - <class 'repoze.what.predicates.NotAuthorizedError'>: 
>>>> [Mon Jun 20 01:28:09 2011] [error] [client 213.129.75.1] Premature end of script headers: dmirr.wsgi
>>>> [Mon Jun 20 01:28:12 2011] [error] [client 163.43.163.48] Premature end of script headers: dmirr.wsgi
>>>> [Mon Jun 20 01:28:13 2011] [error] [client 72.72.203.106] Premature end of script headers: dmirr.wsgi
>>>> [Mon Jun 20 01:28:25 2011] [error] [client 193.110.195.10] Premature end of script headers: dmirr.wsgi
>>>> [Mon Jun 20 01:28:26 2011] [error] [client 146.115.38.39] Premature end of script headers: dmirr.wsgi
>>>> [Mon Jun 20 01:28:43 2011] [error] [client 193.85.153.130] Premature end of script headers: dmirr.wsgi
>>>> [Mon Jun 20 01:28:46 2011] [error] [client 61.197.227.129] Premature end of script headers: dmirr.wsgi
>>>> [Mon Jun 20 01:28:47 2011] [error] [client 87.237.215.129] Premature end of script headers: dmirr.wsgi
>>>> [Mon Jun 20 01:28:49 2011] [error] [client 134.99.64.239] Premature end of script headers: dmirr.wsgi
>>>> [Mon Jun 20 01:28:57 2011] [error] [client 196.15.240.2] Premature end of script headers: dmirr.wsgi
>>>> [Mon Jun 20 01:28:58 2011] [error] [client 174.143.56.162] Premature end of script headers: dmirr.wsgi
>>>> [Mon Jun 20 01:29:03 2011] [error] [client 50.17.177.64] Premature end of script headers: dmirr.wsgi
>>>> [Mon Jun 20 01:29:03 2011] [error] [client 69.20.65.82] Premature end of script headers: dmirr.wsgi
>>>> 
>>>> ---
>>>> 
>>>> I issued a httpd restart and dmirr went back to normal.
>>>> 
>>>> ---
>>>> Jeffrey Ness
>>>> Linux System Engineer
>>>> IT Operations [Development]
>>>> Rackspace Hosting & IUS Community
>>>> 
>>>> On Jun 20, 2011, at 2:04 AM, <wdierkes@xxxxxxxxxxxxx>
>>>> <wdierkes@xxxxxxxxxxxxx> wrote:
>>>> 
>>>>> See <http://jenkins.iuscommunity.org/job/verify-mirrorlist/3104/>
>>>>> 
>>>>> ------------------------------------------
>>>>> Started by timer
>>>>> Building on master
>>>>> [workspace] $ /bin/sh -xe /tmp/hudson200383432980641728.sh
>>>>> + /usr/bin/curl -s 'http://dmirr.iuscommunity.org/mirrorlist?repo=ius-el4&arch=i386'
>>>>> + grep ius/stable
>>>>> + exit 1
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> Mailing list: https://launchpad.net/~ius-coredev
>>>>> Post to     : ius-coredev@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~ius-coredev
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>> 
>> 
>> 
>> _______________________________________________
>> Mailing list: https://launchpad.net/~ius-coredev
>> Post to     : ius-coredev@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~ius-coredev
>> More help   : https://help.launchpad.net/ListHelp
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~ius-coredev
> Post to     : ius-coredev@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~ius-coredev
> More help   : https://help.launchpad.net/ListHelp

References

  Thread Previous • Date Previous • • Thread Next