launchpad-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

Trimming the CC back to just launchpad-dev now we're out of zomg mode.

On Tue, Nov 16, 2010 at 1:17 AM, Jonathan Lange <jml@xxxxxxxxxxxxx> wrote:
> On Sat, Nov 13, 2010 at 6:46 PM, Robert Collins
> <robert.collins@xxxxxxxxxxxxx> wrote:
>> On Sun, Nov 14, 2010 at 12:00 AM, Robert Collins
>> <robert.collins@xxxxxxxxxxxxx> wrote:
>>> We had a period - about 36 hours long - where the xmlrpc internal
>>> server was simply saturated and overloaded. During this period folk
>>> would experience a backtrace as described in
>>> https://bugs.launchpad.net/launchpad/+bug/674416.
>>
>
> Thanks so much for escalating this problem. I'm glad to see the big,
> scary stack traces during normal usage of a critical part of Launchpad
> go away.

Strictly speaking I was in violation of policy. I think our policy
here is bong and we should start over.

>> Colin Watson has noted that we have some cleanup to do - we need to
>> trigger a 'branchChanged' event for all the branches which had that
>> event crash.
>>
>
> Has this been done? Where are we tracking this?

No, in the bug. I'm @ training this week - I sure hope someone will
step up and do it. Tim was unwell today or I suspect he would have.

> There's an unreported bug here about the gap in our error handling
> system. No end user should have seen stack traces at all. Instead, we
> should have displayed & recorded an OOPS, and used that OOPS data to
> trigger an alert.

agreed. Could you perhaps file that? Priority high, tagged oops?

-Rob