← Back to team overview

maas-builds team mailing list archive

Re: maas-utopic-trunk - Build # 566 - Failure! - Built with lp:maas (revno: 4064) and lp:~maas-maintainers/maas/packaging (revno: 409)

 

I seriously think it would be better if we created a separate job for
mirroring the images, and the CI could just use the downloaded mirror.

Mike

On Thu, Jul 9, 2015 at 5:07 PM, Blake Rouse <blake.rouse@xxxxxxxxxxxxx>
wrote:

> Haha. We don't need another, we got this one. :-)
> On Jul 9, 2015 8:02 PM, "Mike Pontillo" <mike.pontillo@xxxxxxxxxxxxx>
> wrote:
>
>> Then we need another CI to test that! ;-)
>>
>> On Thu, Jul 9, 2015 at 4:53 PM, Blake Rouse <blake.rouse@xxxxxxxxxxxxx>
>> wrote:
>>
>>> I disagree as using the maas.ubuntu.com images makes sure that service
>>> is working as everyone else relies on it.
>>> On Jul 9, 2015 7:37 PM, "Mike Pontillo" <mike.pontillo@xxxxxxxxxxxxx>
>>> wrote:
>>>
>>>> Maybe we should change the CI to use a local mirror of the images, to
>>>> make the CI faster and eliminate side effects from network latency.
>>>>
>>>> Mike
>>>>
>>>> On Tue, Jul 7, 2015 at 7:38 AM, Andres Rodriguez <
>>>> andres.rodriguez@xxxxxxxxxxxxx> wrote:
>>>>
>>>>> Yeah, so then the only issues we need to figure out are:
>>>>>
>>>>> 1. Why is the region trying to access clusterd.conf
>>>>> 2. Why the tests failing to pass? Is it not importing for another
>>>>> issue or is it just taking longer than expected, since judging by the logs,
>>>>> both the cluster and region are up.
>>>>>
>>>>> On Tue, Jul 7, 2015 at 9:33 AM, Gavin Panella <
>>>>> gavin.panella@xxxxxxxxxxxxx> wrote:
>>>>>
>>>>>> On 7 July 2015 at 13:47, Andres Rodriguez
>>>>>> <andres.rodriguez@xxxxxxxxxxxxx> wrote:
>>>>>> > Hi Gavin,
>>>>>> >>
>>>>>> >>
>>>>>> >> > http://162.213.35.104:8080/job/maas-utopic-trunk/576/console
>>>>>> >> >
>>>>>> >> >
>>>>>> http://162.213.35.104:8080/job/maas-utopic-trunk/576/artifact/results/artifacts/maas-logs/var/log/maas/regiond.log
>>>>>> >> >
>>>>>> >> >
>>>>>> http://162.213.35.104:8080/job/maas-utopic-trunk/576/artifact/results/artifacts/maas-logs/var/log/maas/clusterd.log
>>>>>> >> >
>>>>>> >> > So, the last issue seems liek the test does not wait long enough
>>>>>> for
>>>>>> >> > the images to be present, however, that's caused for other
>>>>>> things (see
>>>>>> >> > regiond.log):
>>>>>> >> >
>>>>>> >> > 1. OSError: [Errno 13] Permission denied:
>>>>>> '/etc/maas/regiond.conf'
>>>>>> >> > 2. django.db.utils.OperationalError: fe_sendauth: no password
>>>>>> supplied
>>>>>> >> > 3. exceptions.OSError: [Errno 13] Permission denied:
>>>>>> >> > '/etc/maas/clusterd.conf'
>>>>>> >> >
>>>>>> >> > So, by the looks of it, it seems that after the installation goes
>>>>>> >> > through the regiond.conf is not with the correct permissions.
>>>>>> >> > Packaging itself does not configure nor changes the permissions.
>>>>>> So
>>>>>> >> > questions are:
>>>>>> >> > 1. Why are the daemons not creating the configs with correct
>>>>>> >> > permissions?
>>>>>> >>
>>>>>> >> I'm investigating this first. This was working, for the region. At
>>>>>> the
>>>>>> >> time I had to make several changes in trunk and in packaging to
>>>>>> get this
>>>>>> >> working. I haven't yet figured out what has made it regress.
>>>>>> >>
>>>>>> >
>>>>>> > After some investigation, the permissions issue is only seen for a
>>>>>> > while, but it gets to the point that it is correct and the daemons
>>>>>> can
>>>>>> > read it just fine.
>>>>>>
>>>>>> I've figured out a fix for this.
>>>>>>
>>>>>> >
>>>>>> >>
>>>>>> >> > 2. What causes the permissions to be changed correctly later on?
>>>>>> >> >
>>>>>> >> > Then, after the daemon successfully loads regiond.conf, no
>>>>>> password is
>>>>>> >> > supplied to the DB:
>>>>>> >> > 3. So, why does the daemon seem not to have the DB password if
>>>>>> it was
>>>>>> >> > supposed to be set?
>>>>>> >>
>>>>>> >> This is probably a symptom of #1.
>>>>>>
>>>>>> The database password is configured by
>>>>>> maas-region-controller.postinst,
>>>>>> but is needed by maas-region-controller-min which is started before
>>>>>> the
>>>>>> database has been configured. The errors do eventually go away.
>>>>>>
>>>>>> >>
>>>>>> >> >
>>>>>> >> > Last, but not least, the regiond is trying to access
>>>>>> cluster.conf:
>>>>>> >> > 4. Why is the region daemon trying to access cluster.conf ? This
>>>>>> >> > should not be happening at all.
>>>>>> >>
>>>>>> >> This looks like a bug, but I'm not 100% sure about that.
>>>>>> >
>>>>>> >
>>>>>> > yup. IIRC, i mentioned this before. For whatever reason the region
>>>>>> is
>>>>>> > trying to access this.
>>>>>> >>
>>>>>> >>
>>>>>> >> >
>>>>>> >> > Gavin, can you please look at the above and fix asap?
>>>>>> >> >
>>>>>> >> > Thanks.
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > --
>>>>>> > Andres Rodriguez
>>>>>> > Engineering Manager, MAAS & OIL DevOps
>>>>>> > Canonical USA, Inc.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Andres Rodriguez
>>>>> Engineering Manager, MAAS & OIL DevOps
>>>>> Canonical USA, Inc.
>>>>>
>>>>> --
>>>>> Mailing list: https://launchpad.net/~maas-builds
>>>>> Post to     : maas-builds@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~maas-builds
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>>
>>>> --
>>>> Mailing list: https://launchpad.net/~maas-builds
>>>> Post to     : maas-builds@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~maas-builds
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>

Follow ups

References