maas-devel team mailing list archive

[Diogo Matsubara <matsubara@xxxxxxxxxxxxx>]

On Thu, Apr 19, 2012 at 5:31 PM, Michael Hughes <itismike@xxxxxxxxx> wrote:
> On Thu, Apr 19, 2012 at 3:02 PM, Diogo Matsubara
> <matsubara@xxxxxxxxxxxxx> wrote:
>> On Thu, Apr 19, 2012 at 4:40 PM, Michael Hughes <itismike@xxxxxxxxx> wrote:
>>> Okay I'll hang tight on the reinstall. Yes, I added an ssh key.
>>> [edit - I just re-read your comment a little closer. You want me to
>>> add an ssh key for the ADMIN user? As in root?]
>>>
>>
>> For the user created by `sudo maas createsuperuser`. Are you logging
>> into MAAS using that user?
>
> I am logging into the web interface as that user, yes. The ssh key I
> generated was done within my normal user account (not root, not
> 'superuser'... does superuser even has login privileges via ssh?)

This is correct. We just found a bug in the documentation. An
important part of the step after generating your ssh-key is to add
that key in your user's MAAS preferences page. The same page used to
get the API key you used on your .juju/environment.yaml

So what I think is happening:

- node boots
- node talks to maas server and identifies itself
- maas server sees the user associated with that node
- node ask for the public ssh key for that user
- It can't find the key
  maas.log: MAASAPINotFound: No registered public keys
  access.log: 172.23.1.161 - - [19/Apr/2012:11:38:32 -0500] "GET
/MAAS/metadata//2012-03-01/meta-data/public-keys HTTP/1.1" 404 249 "-"
"Python-urllib/2.7"
- it continues booting but the nodes become inacessible to that user
since there's no user/pass set and auth should happen through pubkey
authentication

If you have registered your ssh key into the MAAS interface, then it's
probably another bug.

>
>>
>>> Here's select parts of my history:
>>>
>>> sudo aptitude [updates were applied]
>>> sudo maas createsuperuser
>>> sudo apt-get install maas-dhcp
>>> sudo vi /etc/network/interfaces
>>> sudo ifup eth1
>>> sudo maas-import-isos
>>> sudo mount /var/lib/maas/ephemeral/precise/server/amd64/20120328/disk.img /mnt/
>>> sudo su -
>>> chroot /mnt
>>> apt-get update
>>> apt-get install cloud-init
>>> exit
>>> exit
>>> sudo umount /mnt
>>> sudo apt-get install juju
>>> mkdir .juju
>>> vi .juju/environments.yaml
>>> ssh-keygen
>>> juju bootstrap
>>>  - [encountered errors here, so tried adding PPAs]
>>> sudo apt-get install python-software-properties
>>> sudo add-apt-repository ppa:juju/pkgs
>>> sudo apt-get update
>>> sudo aptitude [updates were applied]
>>> juju bootstrap
>>>  - [encountered errors here]
>>> cat .juju/environments.yaml
>>> juju --verbose bootstrap
>>> juju bootstrap
>>> sudo cat /var/log/maas/maas.log
>>> sudo cat /var/log/apache2/access.log
>>> sudo cat /var/log/apache2/error.log
>>>
>>> I just created a 4th node and 5th nodes and each said "Commissioning"
>>> in the GUI before I booted them. They booted normally, generating a
>>
>> How are you creating the nodes? Are you creating a new VM and then
>> adding them through the MAAS UI? Or you create the VM, pxe boot it and
>> let MAAS enlist them automatically?
>
> I haven't figured out how to set it to auto-create nodes yet, so I'm
> just entering the MAC addresses into the MAAS web interface. Full
> automation is the end-goal though :)
>
>>
>>> SSH host key and reported:
>>> cloud-init boot finished at Thu... Up 23.35 seconds.
>>> Followed shortly by:
>>> Skipping profile in /etc/aparmor.d/disable: usr.sbin.rsyslogd
>>> landscape-client is not configured...
>>>
>>
>> Can you ssh into the instance?
>
> No:
> $ ssh 172.23.1.5
> Permission denied (publickey).
> $
>
> I couldn't ssh into it anyway could I? I don't know the
> username/password that the nodes use.

Yes, you can only ssh into the node using public key authentication. I
don't think there's a user/pass set

>
>> What does ssh -vvv <ip-of-a-node> reports?
>
> http://pastebin.ubuntu.com/937427/
>
>>
>>> The Nodes screen indicated these new nodes were "Commissioning" until
>>> I clicked home and back into Nodes. Then they indicated "Ready." I did
>>> not see the word "Declared."
>>
>> Can you run:
>>
>> sudo maas shell
>> from maasserver.models import Node
>> [node.owner for node in Node.objects.all()]
>>
>> and paste back the result?
>>
>
> $ sudo maas shell
> Python 2.7.3 (default, Apr 10 2012, 12:29:04)
> [GCC 4.6.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> (InteractiveConsole)
>>>> from maasserver.models import Node
>>>> [node.owner for node in Node.objects.all()]
> [<User: mike>, <User: mike>, <User: mike>, <User: mike>, <User: mike>]
>>>>
>
>
>> Thanks,
>>
>> Diogo
>>>
>>>
>>>
>>> On Thu, Apr 19, 2012 at 2:04 PM, Diogo Matsubara
>>> <matsubara@xxxxxxxxxxxxx> wrote:
>>>> On Thu, Apr 19, 2012 at 3:48 PM, Michael Hughes <itismike@xxxxxxxxx> wrote:
>>>>> maas.log ==> http://pastebin.ubuntu.com/937067/
>>>>> access.log ==> http://pastebin.ubuntu.com/937084/
>>>>> error.log ==> http://pastebin.ubuntu.com/937268/
>>>>>
>>>>> I don't think I'm doing anything special. Maybe I'll try reinstalling
>>>>> the base OS again and start from the beginning.
>>>>>
>>>>
>>>> Please don't reinstall. I think I've run into the same bug as you.
>>>>
>>>> Did you add a ssh key for the admin user?
>>>> Could you check in any node edit page if the node has a owner assigned?
>>>> You said that they are skipping the commissioning state and going
>>>> directly from Declared to Ready? Is this still happening?
>>>>
>>>> Thank you and sorry for all this inconvenience.
>>>>
>>>>> On Thu, Apr 19, 2012 at 11:31 AM, Diogo Matsubara
>>>>> <matsubara@xxxxxxxxxxxxx> wrote:
>>>>>> On Thu, Apr 19, 2012 at 1:22 PM, Michael Hughes <itismike@xxxxxxxxx> wrote:
>>>>>>> Thanks Diogo! I added the reference to the Ephemeral bug to the troubleshooter.
>>>>>>>
>>>>>>
>>>>>> Thank you!
>>>>>>
>>>>>>> Verbose provides a few more lines at the bottom:
>>>>>>> ~$ juju --verbose bootstrap
>>>>>>> 2012-04-19 10:43:15,254 DEBUG Initializing juju bootstrap runtime
>>>>>>> ...
>>>>>>> Traceback (most recent call last):
>>>>>>> Failure: juju.errors.ProviderInteractionError: Unexpected TimeoutError
>>>>>>> interacting with provider: User timeout caused connection failure.
>>>>>>> 2012-04-19 10:43:45,297 ERROR Traceback (most recent call last):
>>>>>>> Failure: juju.errors.ProviderInteractionError: Unexpected TimeoutError
>>>>>>> interacting with provider: User timeout caused connection failure.
>>>>>>>
>>>>>>> Unexpected TimeoutError interacting with provider: User timeout caused
>>>>>>> connection failure.
>>>>>>> 2012-04-19 10:43:45,299 ERROR Unexpected TimeoutError interacting with
>>>>>>> provider: User timeout caused connection failure.
>>>>>>> ~$
>>>>>>
>>>>>> When you bootstrap, do you see anything in the maas.log? How about
>>>>>> apache's access.log and error.log?
>>>>>> /var/log/maas/maas.log
>>>>>> /var/log/apache2/access.log
>>>>>> /var/log/apache2/error.log
>>>>>>
>>>>>> Could you paste those logs on pastebin.ubuntu.com?
>>>>>> I'm asking the juju developers what causes this kinds of errors and
>>>>>> will get back to you.
>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Mike
>>>>>>>
>>>>>>> On Thu, Apr 19, 2012 at 9:45 AM, Diogo Matsubara
>>>>>>> <matsubara@xxxxxxxxxxxxx> wrote:
>>>>>>>> Oh, forgot to say, feel free to update
>>>>>>>> https://wiki.ubuntu.com/ServerTeam/MAAS/Troubleshooting with any
>>>>>>>> further debugging tips you have.
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>> Diogo
>>>>>>>>
>>>>>>>> On Thu, Apr 19, 2012 at 11:37 AM, Diogo Matsubara
>>>>>>>> <matsubara@xxxxxxxxxxxxx> wrote:
>>>>>>>>> Hi MIchael,
>>>>>>>>>
>>>>>>>>> On Thu, Apr 19, 2012 at 10:32 AM, Michael Hughes <itismike@xxxxxxxxx> wrote:
>>>>>>>>>> Hi Jeroen. Thanks for the idea but I don't believe I've done anything
>>>>>>>>>> fancy in my setup that would require manipulating the database by
>>>>>>>>>> hand. My MAAS server has two NICs: one public which connects to the
>>>>>>>>>> Internet and one private. Each node has a single NIC which is
>>>>>>>>>> connected to that private NIC via a switch. Isn't this a pretty
>>>>>>>>>> standard setup for a group of machines with a dedicated DHCP server
>>>>>>>>>> like MAAS employs?
>>>>>>>>>>
>>>>>>>>>> Further, while I was not able to reconfigure the gateway address for
>>>>>>>>>> the nodes by running dpkg-reconfigure maas-dhcp, I performed a full
>>>>>>>>>> reinstall (using the steps below) the proper gateway settings
>>>>>>>>>> (172.x.x.x) stuck just fine. Nodes now boot and appear to install a
>>>>>>>>>> base system.
>>>>>>>>>>
>>>>>>>>>> There is still no 'Accept and Commission' button in the GUI for me,
>>>>>>>>>> but the nodes now report as "Ready" rather than "Commissioning."  My
>>>>>>>>>> next challenge is that Juju isn't bootstrapping but since I've made
>>>>>>>>>> some progress with MAAS I figured I'd spell out what has worked for me
>>>>>>>>>> to this point:
>>>>>>>>>>
>>>>>>>>>>  • Environment: 64-bit Virtualbox running on Windows 7 host
>>>>>>>>>>  • 4 64-bit guest VMs consisting of:
>>>>>>>>>>     - 1 MAAS server with two NICs - one bridged to the Internet, one
>>>>>>>>>> Internal Network (intnet)
>>>>>>>>>>     - 3 nodes each with one NIC set to Internal Network (intnet)
>>>>>>>>>>  • Installed yesterday's daily build of precise-server-amd64.iso
>>>>>>>>>> (18-Apr-2012 06:40) onto the MAAS server following this wiki:
>>>>>>>>>> https://wiki.ubuntu.com/ServerTeam/MAAS
>>>>>>>>>>  • Overrode the default gateway address for nodes and plugged in my
>>>>>>>>>> private IP address (172.x.x.x) [BTW, the language at this step is
>>>>>>>>>> worded to suggest leaving the guessed IP address as-is, which was
>>>>>>>>>> incorrect for my vanilla(?) setup. Perhaps this needs tweaking. Just
>>>>>>>>>> remove the language and describe the gateway a little better ("path to
>>>>>>>>>> your MAAS server" rather than "path to the Internet",) and let the
>>>>>>>>>> admin figure out which IP is appropriate)
>>>>>>>>>>  • Copied down the MAAS management URL when provided
>>>>>>>>>>  • Bring the system up-to-date with sudo apt-get update and sudo apt-get install
>>>>>>>>>>  • Edit /etc/network/interfaces to add an IP address for my internal
>>>>>>>>>> network; ifup eth1
>>>>>>>>>>  • Return to the wiki to complete the createsuperuser and import-isos steps
>>>>>>>>>>  • Follow the steps in comment #2 of this bug to bring ephemerals
>>>>>>>>>> current: https://bugs.launchpad.net/ubuntu/+source/maas/+bug/981845
>>>>>>>>>>  • Return to the wiki to add nodes
>>>>>>>>>>
>>>>>>>>>> At this point, I started the nodes and the nodes boot well past the
>>>>>>>>>> previous stoppage. An auto-login processes, SSH keys are generated,
>>>>>>>>>> and the tty output on the nodes stops with 'landscape-client is not
>>>>>>>>>> configured, please run landscape-config.'
>>>>>>>>>>
>>>>>>>>>> The GUI reports that all nodes are "Ready," though none are ready for
>>>>>>>>>> deployment:
>>>>>>>>>> "3 nodes in this MAAS
>>>>>>>>>> 0 nodes reserved for named deployment.
>>>>>>>>>> 0 retired nodes not represented."
>>>>>>>>>
>>>>>>>>> Ready means they're ready for deployment. I think the message 0 nodes
>>>>>>>>> reserved for named deployment is confusing you here. (This has been
>>>>>>>>> fixed recently: https://bugs.launchpad.net/maas/+bug/979902)
>>>>>>>>> Named deployment is when you want to deploy to a specific node, using
>>>>>>>>> its hostname as a constraint.
>>>>>>>>> Something like `juju deploy --constraints
>>>>>>>>> "maas-name=node-00e081ddd11b.local" mysql`
>>>>>>>>> More info about juju constraints can be found here:
>>>>>>>>> https://juju.ubuntu.com/docs/constraints.html
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Clicking "Start node" in the node editing screen shows "Node Started"
>>>>>>>>>> in the notification area, yet no changes are observed on the node's
>>>>>>>>>> tty.
>>>>>>>>>
>>>>>>>>> Did you configure virsh as the power type for your nodes? I think that
>>>>>>>>> needs to be set before you can start a node.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  • Continue to the juju wiki. However, since I'm using 12.04 I did not
>>>>>>>>>> add the PPA as indicated in the 'Before you get going' section.
>>>>>>>>>>
>>>>>>>>>> Running juju on the command-line of the MAAS server returns this error:
>>>>>>>>>> ==
>>>>>>>>>> $ juju bootstrap
>>>>>>>>>> 2012-04-18 18:23:50,052 INFO Bootstrapping environment 'maas' (origin:
>>>>>>>>>> distro type: maas)...
>>>>>>>>>> Unhandled Error
>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/juju/control/__init__.py",
>>>>>>>>>> line 188, in main
>>>>>>>>>>    options.command(options)
>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/juju/control/command.py",
>>>>>>>>>> line 44, in __call__
>>>>>>>>>>    reactor.run()
>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py",
>>>>>>>>>> line 1169, in run
>>>>>>>>>>    self.mainLoop()
>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py",
>>>>>>>>>> line 1178, in mainLoop
>>>>>>>>>>    self.runUntilCurrent()
>>>>>>>>>> --- <exception caught here> ---
>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py",
>>>>>>>>>> line 800, in runUntilCurrent
>>>>>>>>>>    call.func(*call.args, **call.kw)
>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/twisted/internet/tcp.py",
>>>>>>>>>> line 362, in resolveAddress
>>>>>>>>>>    self._setRealAddress(self.addr[0])
>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/twisted/internet/tcp.py",
>>>>>>>>>> line 369, in _setRealAddress
>>>>>>>>>>    self.doConnect()
>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/twisted/internet/tcp.py",
>>>>>>>>>> line 395, in doConnect
>>>>>>>>>>    connectResult = self.socket.connect_ex(self.realAddress)
>>>>>>>>>>  File "/usr/lib/python2.7/socket.py", line 224, in meth
>>>>>>>>>>    return getattr(self._sock,name)(*args)
>>>>>>>>>> exceptions.TypeError: an integer is required
>>>>>>>>>> Unexpected TimeoutError interacting with provider: User timeout caused
>>>>>>>>>> connection failure.
>>>>>>>>>> 2012-04-18 18:24:20,090 ERROR Unexpected TimeoutError interacting with
>>>>>>>>>> provider: User timeout caused connection failure.
>>>>>>>>>> $
>>>>>>>>>> ==
>>>>>>>>>>
>>>>>>>>>> I've attempted both with and without the juju PPA but it still
>>>>>>>>>> produces the same results.
>>>>>>>>>
>>>>>>>>> Does juju --verbose bootstrap gives you any more info?
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I hope this feedback is helpful and am excited to continue working on
>>>>>>>>>> this in anticipation of the 12.04 release!
>>>>>>>>>
>>>>>>>>> It is very useful, please keep it coming! :-)
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>> Diogo
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -Mike
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 18, 2012 at 11:25 PM, Jeroen Vermeulen <jtv@xxxxxxxxxxxxx> wrote:
>>>>>>>>>>> On 2012-04-19 01:30, Michael Hughes wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Maybe I'm using am incompatible base OS. What are others testing MAAS
>>>>>>>>>>>> with?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> In the development team we all run 12.04.
>>>>>>>>>>>
>>>>>>>>>>> Looks like you to change the maas_url setting to adapt to your setup. This
>>>>>>>>>>> setting tells nodes where they can reach the MAAS service.  The URL's
>>>>>>>>>>> hostname part defaults to the IP address of whatever interface your server
>>>>>>>>>>> uses for its default route, but it sounds like that's not going to be right
>>>>>>>>>>> for your network.
>>>>>>>>>>>
>>>>>>>>>>> I don't think we have the UI for such settings accessible yet.  But you can
>>>>>>>>>>> change the setting directly in the database:
>>>>>>>>>>>
>>>>>>>>>>>    INSERT INTO maasserver_config (name, value)
>>>>>>>>>>>    VALUES ('maas_url', 'http://<ip>/MAAS/');
>>>>>>>>>>>
>>>>>>>>>>> The <ip> is the address by which the nodes can reach the server.  The
>>>>>>>>>>> “MAAS/” path is as what you'd use to get to the MAAS user interface (we use
>>>>>>>>>>> just “/” on development setups).
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Jeroen
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Mailing list: https://launchpad.net/~maas-devel
>>>>>>>>>> Post to     : maas-devel@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>> Unsubscribe : https://launchpad.net/~maas-devel
>>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Diogo M. Matsubara
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Diogo M. Matsubara
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Diogo M. Matsubara
>>>>
>>>>
>>>>
>>>> --
>>>> Diogo M. Matsubara
>>
>>
>>
>> --
>> Diogo M. Matsubara



-- 
Diogo M. Matsubara