← Back to team overview

ubuntu-phone team mailing list archive

Re: Status of CI Services - December 11, 2014

 

We need to disable the MP phone testing again. We've temporarily lost the
ability to remote recover the devices and as a result they are all offline
now. As before, only the projects that use phone testing will be disabled.

Other updates:
 - The VMs for some specific unity8 and ubuntu-settings-components MP tests
are back up. The s-jenkins restart was successful.
 - All of the test slaves used for proposed-migration should be available
and running tests again.

Francis


On Thu, Dec 11, 2014 at 10:49 AM, Francis Ginther <
francis.ginther@xxxxxxxxxxxxx> wrote:

> The MP testing service, s-jenkins, will need a restart at 18:00 UTC. The
> restart should take less than 15 minutes.
>
> Also, please be aware that the VPN to access the internal CI services has
> changed. For the new VPN go to
> https://wiki.canonical.com/InformationInfrastructure/IS/HowTo/CompanyOpenVPN
> for instructions.
>
> Updates:
>  - The rtm-dashboard is up again. Results from older test runs are still
> being populated.
>  - The MP testing backlog has been processed. All projects including those
> needing phones to test are enabled. The armhf builders are all back up.
>
> What’s still down:
>  - Emails from jenkins jobs, etc. are not being forwarded (IS has been
> notified).
>  - The VMs for some specific unity8 and ubuntu-settings-components MP
> tests are unavailable. This is expected to be fixed by the s-jenkins
> restart. The test jobs have been disabled until then.
>  - Some of the non-x86 test slaves used for proposed-migration are still
> down (IS has been notified).
>  - The bootspeed and unity7 test systems are still awaiting the correct
> power bricks and/or power cords (IS is working the issue).
>
> Francis
>
> On Wed, Dec 10, 2014 at 6:07 PM, Francis Ginther <
> francis.ginther@xxxxxxxxxxxxx> wrote:
>
>> Since the last update:
>>  - MP testing has been fully enabled. I expect to see some long build
>> queues as the backlog is worked down and we're only at half our normal
>> number of phones available for testing.
>>  - The proposed migration slaves can now talk to ftpmaster.internal which
>> was causing failures earlier today.
>>
>>
>> What's still down:
>>  - The rtm-dashboard for looking at krillin smoke testing results (still
>> a work in progress).
>>  - Some of the non-x86 test slaves used for proposed-migration are still
>> down.
>>
>>
>> Francis
>>
>> On Wed, Dec 10, 2014 at 12:00 PM, Francis Ginther <
>> francis.ginther@xxxxxxxxxxxxx> wrote:
>>
>>> Here's the latest on what is now working:
>>>
>>> - The krillin build process is now fully enabled and builds are working.
>>>
>>> - Smoke testing is running for both mako and krilin devices. Capacity is
>>> limited as we have several makos that need attention from the Data Center
>>> Engineers (who are also busy moving other kit).
>>>
>>>
>>> And what's still offline:
>>>
>>> - MP testing is disabled, but should be re-enabled in the next 2-3
>>> hours. This will initially be at very low capacity as several of the armhf
>>> builders were lost during the move. They are being rebuilt. Phones for
>>> testing MPs are also limited.
>>>
>>> - The rtm-dashboard for looking at krillin smoke testing results (work
>>> in progress).
>>>
>>> - The proposed-migration test slaves are still having network
>>> connectivity issues (IS has been notified).
>>>
>>> Francis
>>>
>>> On Tue, Dec 9, 2014 at 5:57 PM, Francis Ginther <
>>> francis.ginther@xxxxxxxxxxxxx> wrote:
>>>
>>>> The CI services are starting to come back online. Much is still
>>>> disabled, but I expect to get the smoke testing enabled tonight. The
>>>> krillin builds are also partially enabled, just need to see if the rest of
>>>> the system-image process is properly networked. The krillin builds are
>>>> currently a priority.
>>>>
>>>> The MP testing is currently still disabled while we work with IS to
>>>> update the network and firewall connections to the build nodes. We also
>>>> need to rebuild some of the armhf build nodes. The current ETA is to have
>>>> this enabled tomorrow.
>>>>
>>>> Francis
>>>> --
>>>> Francis Ginther
>>>> Canonical - Ubuntu Engineering - Continuous Integration Team
>>>>
>>>
>>>
>>>
>>> --
>>> Francis Ginther
>>> Canonical - Ubuntu Engineering - Continuous Integration Team
>>>
>>
>>
>>
>> --
>> Francis Ginther
>> Canonical - Ubuntu Engineering - Continuous Integration Team
>>
>
>
>
> --
> Francis Ginther
> Canonical - Ubuntu Engineering - Continuous Integration Team
>



-- 
Francis Ginther
Canonical - Ubuntu Engineering - Continuous Integration Team

Follow ups

References