maas-devel team mailing list archive

[Diogo Matsubara <matsubara@xxxxxxxxxxxxx>]

On Thu, Apr 19, 2012 at 6:09 PM, Michael Hughes <itismike@xxxxxxxxx> wrote:
> Funny - I think I was on to something a week ago when I added this to the wiki:
> https://wiki.ubuntu.com/ServerTeam/MAAS/Juju?action=recall&rev=10#Generating_SSH_keypair_for_Juju
>
> It annoys me that some heavy-handed editor reverted my contributions
> rather than confirming or enhancing them.
>

Sorry about that. I just updated the instructions to include a step to
add the ssh-key to the MAAS user account.

After adding to your account, did it work?

> On Thu, Apr 19, 2012 at 3:53 PM, Diogo Matsubara
> <matsubara@xxxxxxxxxxxxx> wrote:
>> On Thu, Apr 19, 2012 at 5:31 PM, Michael Hughes <itismike@xxxxxxxxx> wrote:
>>> On Thu, Apr 19, 2012 at 3:02 PM, Diogo Matsubara
>>> <matsubara@xxxxxxxxxxxxx> wrote:
>>>> On Thu, Apr 19, 2012 at 4:40 PM, Michael Hughes <itismike@xxxxxxxxx> wrote:
>>>>> Okay I'll hang tight on the reinstall. Yes, I added an ssh key.
>>>>> [edit - I just re-read your comment a little closer. You want me to
>>>>> add an ssh key for the ADMIN user? As in root?]
>>>>>
>>>>
>>>> For the user created by `sudo maas createsuperuser`. Are you logging
>>>> into MAAS using that user?
>>>
>>> I am logging into the web interface as that user, yes. The ssh key I
>>> generated was done within my normal user account (not root, not
>>> 'superuser'... does superuser even has login privileges via ssh?)
>>
>> This is correct. We just found a bug in the documentation. An
>> important part of the step after generating your ssh-key is to add
>> that key in your user's MAAS preferences page. The same page used to
>> get the API key you used on your .juju/environment.yaml
>>
>> So what I think is happening:
>>
>> - node boots
>> - node talks to maas server and identifies itself
>> - maas server sees the user associated with that node
>> - node ask for the public ssh key for that user
>> - It can't find the key
>>  maas.log: MAASAPINotFound: No registered public keys
>>  access.log: 172.23.1.161 - - [19/Apr/2012:11:38:32 -0500] "GET
>> /MAAS/metadata//2012-03-01/meta-data/public-keys HTTP/1.1" 404 249 "-"
>> "Python-urllib/2.7"
>> - it continues booting but the nodes become inacessible to that user
>> since there's no user/pass set and auth should happen through pubkey
>> authentication
>>
>> If you have registered your ssh key into the MAAS interface, then it's
>> probably another bug.
>>
>>>
>>>>
>>>>> Here's select parts of my history:
>>>>>
>>>>> sudo aptitude [updates were applied]
>>>>> sudo maas createsuperuser
>>>>> sudo apt-get install maas-dhcp
>>>>> sudo vi /etc/network/interfaces
>>>>> sudo ifup eth1
>>>>> sudo maas-import-isos
>>>>> sudo mount /var/lib/maas/ephemeral/precise/server/amd64/20120328/disk.img /mnt/
>>>>> sudo su -
>>>>> chroot /mnt
>>>>> apt-get update
>>>>> apt-get install cloud-init
>>>>> exit
>>>>> exit
>>>>> sudo umount /mnt
>>>>> sudo apt-get install juju
>>>>> mkdir .juju
>>>>> vi .juju/environments.yaml
>>>>> ssh-keygen
>>>>> juju bootstrap
>>>>>  - [encountered errors here, so tried adding PPAs]
>>>>> sudo apt-get install python-software-properties
>>>>> sudo add-apt-repository ppa:juju/pkgs
>>>>> sudo apt-get update
>>>>> sudo aptitude [updates were applied]
>>>>> juju bootstrap
>>>>>  - [encountered errors here]
>>>>> cat .juju/environments.yaml
>>>>> juju --verbose bootstrap
>>>>> juju bootstrap
>>>>> sudo cat /var/log/maas/maas.log
>>>>> sudo cat /var/log/apache2/access.log
>>>>> sudo cat /var/log/apache2/error.log
>>>>>
>>>>> I just created a 4th node and 5th nodes and each said "Commissioning"
>>>>> in the GUI before I booted them. They booted normally, generating a
>>>>
>>>> How are you creating the nodes? Are you creating a new VM and then
>>>> adding them through the MAAS UI? Or you create the VM, pxe boot it and
>>>> let MAAS enlist them automatically?
>>>
>>> I haven't figured out how to set it to auto-create nodes yet, so I'm
>>> just entering the MAC addresses into the MAAS web interface. Full
>>> automation is the end-goal though :)
>>>
>>>>
>>>>> SSH host key and reported:
>>>>> cloud-init boot finished at Thu... Up 23.35 seconds.
>>>>> Followed shortly by:
>>>>> Skipping profile in /etc/aparmor.d/disable: usr.sbin.rsyslogd
>>>>> landscape-client is not configured...
>>>>>
>>>>
>>>> Can you ssh into the instance?
>>>
>>> No:
>>> $ ssh 172.23.1.5
>>> Permission denied (publickey).
>>> $
>>>
>>> I couldn't ssh into it anyway could I? I don't know the
>>> username/password that the nodes use.
>>
>> Yes, you can only ssh into the node using public key authentication. I
>> don't think there's a user/pass set
>>
>>>
>>>> What does ssh -vvv <ip-of-a-node> reports?
>>>
>>> http://pastebin.ubuntu.com/937427/
>>>
>>>>
>>>>> The Nodes screen indicated these new nodes were "Commissioning" until
>>>>> I clicked home and back into Nodes. Then they indicated "Ready." I did
>>>>> not see the word "Declared."
>>>>
>>>> Can you run:
>>>>
>>>> sudo maas shell
>>>> from maasserver.models import Node
>>>> [node.owner for node in Node.objects.all()]
>>>>
>>>> and paste back the result?
>>>>
>>>
>>> $ sudo maas shell
>>> Python 2.7.3 (default, Apr 10 2012, 12:29:04)
>>> [GCC 4.6.3] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>> (InteractiveConsole)
>>>>>> from maasserver.models import Node
>>>>>> [node.owner for node in Node.objects.all()]
>>> [<User: mike>, <User: mike>, <User: mike>, <User: mike>, <User: mike>]
>>>>>>
>>>
>>>
>>>> Thanks,
>>>>
>>>> Diogo
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Apr 19, 2012 at 2:04 PM, Diogo Matsubara
>>>>> <matsubara@xxxxxxxxxxxxx> wrote:
>>>>>> On Thu, Apr 19, 2012 at 3:48 PM, Michael Hughes <itismike@xxxxxxxxx> wrote:
>>>>>>> maas.log ==> http://pastebin.ubuntu.com/937067/
>>>>>>> access.log ==> http://pastebin.ubuntu.com/937084/
>>>>>>> error.log ==> http://pastebin.ubuntu.com/937268/
>>>>>>>
>>>>>>> I don't think I'm doing anything special. Maybe I'll try reinstalling
>>>>>>> the base OS again and start from the beginning.
>>>>>>>
>>>>>>
>>>>>> Please don't reinstall. I think I've run into the same bug as you.
>>>>>>
>>>>>> Did you add a ssh key for the admin user?
>>>>>> Could you check in any node edit page if the node has a owner assigned?
>>>>>> You said that they are skipping the commissioning state and going
>>>>>> directly from Declared to Ready? Is this still happening?
>>>>>>
>>>>>> Thank you and sorry for all this inconvenience.
>>>>>>
>>>>>>> On Thu, Apr 19, 2012 at 11:31 AM, Diogo Matsubara
>>>>>>> <matsubara@xxxxxxxxxxxxx> wrote:
>>>>>>>> On Thu, Apr 19, 2012 at 1:22 PM, Michael Hughes <itismike@xxxxxxxxx> wrote:
>>>>>>>>> Thanks Diogo! I added the reference to the Ephemeral bug to the troubleshooter.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Thank you!
>>>>>>>>
>>>>>>>>> Verbose provides a few more lines at the bottom:
>>>>>>>>> ~$ juju --verbose bootstrap
>>>>>>>>> 2012-04-19 10:43:15,254 DEBUG Initializing juju bootstrap runtime
>>>>>>>>> ...
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> Failure: juju.errors.ProviderInteractionError: Unexpected TimeoutError
>>>>>>>>> interacting with provider: User timeout caused connection failure.
>>>>>>>>> 2012-04-19 10:43:45,297 ERROR Traceback (most recent call last):
>>>>>>>>> Failure: juju.errors.ProviderInteractionError: Unexpected TimeoutError
>>>>>>>>> interacting with provider: User timeout caused connection failure.
>>>>>>>>>
>>>>>>>>> Unexpected TimeoutError interacting with provider: User timeout caused
>>>>>>>>> connection failure.
>>>>>>>>> 2012-04-19 10:43:45,299 ERROR Unexpected TimeoutError interacting with
>>>>>>>>> provider: User timeout caused connection failure.
>>>>>>>>> ~$
>>>>>>>>
>>>>>>>> When you bootstrap, do you see anything in the maas.log? How about
>>>>>>>> apache's access.log and error.log?
>>>>>>>> /var/log/maas/maas.log
>>>>>>>> /var/log/apache2/access.log
>>>>>>>> /var/log/apache2/error.log
>>>>>>>>
>>>>>>>> Could you paste those logs on pastebin.ubuntu.com?
>>>>>>>> I'm asking the juju developers what causes this kinds of errors and
>>>>>>>> will get back to you.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Mike
>>>>>>>>>
>>>>>>>>> On Thu, Apr 19, 2012 at 9:45 AM, Diogo Matsubara
>>>>>>>>> <matsubara@xxxxxxxxxxxxx> wrote:
>>>>>>>>>> Oh, forgot to say, feel free to update
>>>>>>>>>> https://wiki.ubuntu.com/ServerTeam/MAAS/Troubleshooting with any
>>>>>>>>>> further debugging tips you have.
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>>
>>>>>>>>>> Diogo
>>>>>>>>>>
>>>>>>>>>> On Thu, Apr 19, 2012 at 11:37 AM, Diogo Matsubara
>>>>>>>>>> <matsubara@xxxxxxxxxxxxx> wrote:
>>>>>>>>>>> Hi MIchael,
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Apr 19, 2012 at 10:32 AM, Michael Hughes <itismike@xxxxxxxxx> wrote:
>>>>>>>>>>>> Hi Jeroen. Thanks for the idea but I don't believe I've done anything
>>>>>>>>>>>> fancy in my setup that would require manipulating the database by
>>>>>>>>>>>> hand. My MAAS server has two NICs: one public which connects to the
>>>>>>>>>>>> Internet and one private. Each node has a single NIC which is
>>>>>>>>>>>> connected to that private NIC via a switch. Isn't this a pretty
>>>>>>>>>>>> standard setup for a group of machines with a dedicated DHCP server
>>>>>>>>>>>> like MAAS employs?
>>>>>>>>>>>>
>>>>>>>>>>>> Further, while I was not able to reconfigure the gateway address for
>>>>>>>>>>>> the nodes by running dpkg-reconfigure maas-dhcp, I performed a full
>>>>>>>>>>>> reinstall (using the steps below) the proper gateway settings
>>>>>>>>>>>> (172.x.x.x) stuck just fine. Nodes now boot and appear to install a
>>>>>>>>>>>> base system.
>>>>>>>>>>>>
>>>>>>>>>>>> There is still no 'Accept and Commission' button in the GUI for me,
>>>>>>>>>>>> but the nodes now report as "Ready" rather than "Commissioning."  My
>>>>>>>>>>>> next challenge is that Juju isn't bootstrapping but since I've made
>>>>>>>>>>>> some progress with MAAS I figured I'd spell out what has worked for me
>>>>>>>>>>>> to this point:
>>>>>>>>>>>>
>>>>>>>>>>>>  • Environment: 64-bit Virtualbox running on Windows 7 host
>>>>>>>>>>>>  • 4 64-bit guest VMs consisting of:
>>>>>>>>>>>>     - 1 MAAS server with two NICs - one bridged to the Internet, one
>>>>>>>>>>>> Internal Network (intnet)
>>>>>>>>>>>>     - 3 nodes each with one NIC set to Internal Network (intnet)
>>>>>>>>>>>>  • Installed yesterday's daily build of precise-server-amd64.iso
>>>>>>>>>>>> (18-Apr-2012 06:40) onto the MAAS server following this wiki:
>>>>>>>>>>>> https://wiki.ubuntu.com/ServerTeam/MAAS
>>>>>>>>>>>>  • Overrode the default gateway address for nodes and plugged in my
>>>>>>>>>>>> private IP address (172.x.x.x) [BTW, the language at this step is
>>>>>>>>>>>> worded to suggest leaving the guessed IP address as-is, which was
>>>>>>>>>>>> incorrect for my vanilla(?) setup. Perhaps this needs tweaking. Just
>>>>>>>>>>>> remove the language and describe the gateway a little better ("path to
>>>>>>>>>>>> your MAAS server" rather than "path to the Internet",) and let the
>>>>>>>>>>>> admin figure out which IP is appropriate)
>>>>>>>>>>>>  • Copied down the MAAS management URL when provided
>>>>>>>>>>>>  • Bring the system up-to-date with sudo apt-get update and sudo apt-get install
>>>>>>>>>>>>  • Edit /etc/network/interfaces to add an IP address for my internal
>>>>>>>>>>>> network; ifup eth1
>>>>>>>>>>>>  • Return to the wiki to complete the createsuperuser and import-isos steps
>>>>>>>>>>>>  • Follow the steps in comment #2 of this bug to bring ephemerals
>>>>>>>>>>>> current: https://bugs.launchpad.net/ubuntu/+source/maas/+bug/981845
>>>>>>>>>>>>  • Return to the wiki to add nodes
>>>>>>>>>>>>
>>>>>>>>>>>> At this point, I started the nodes and the nodes boot well past the
>>>>>>>>>>>> previous stoppage. An auto-login processes, SSH keys are generated,
>>>>>>>>>>>> and the tty output on the nodes stops with 'landscape-client is not
>>>>>>>>>>>> configured, please run landscape-config.'
>>>>>>>>>>>>
>>>>>>>>>>>> The GUI reports that all nodes are "Ready," though none are ready for
>>>>>>>>>>>> deployment:
>>>>>>>>>>>> "3 nodes in this MAAS
>>>>>>>>>>>> 0 nodes reserved for named deployment.
>>>>>>>>>>>> 0 retired nodes not represented."
>>>>>>>>>>>
>>>>>>>>>>> Ready means they're ready for deployment. I think the message 0 nodes
>>>>>>>>>>> reserved for named deployment is confusing you here. (This has been
>>>>>>>>>>> fixed recently: https://bugs.launchpad.net/maas/+bug/979902)
>>>>>>>>>>> Named deployment is when you want to deploy to a specific node, using
>>>>>>>>>>> its hostname as a constraint.
>>>>>>>>>>> Something like `juju deploy --constraints
>>>>>>>>>>> "maas-name=node-00e081ddd11b.local" mysql`
>>>>>>>>>>> More info about juju constraints can be found here:
>>>>>>>>>>> https://juju.ubuntu.com/docs/constraints.html
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Clicking "Start node" in the node editing screen shows "Node Started"
>>>>>>>>>>>> in the notification area, yet no changes are observed on the node's
>>>>>>>>>>>> tty.
>>>>>>>>>>>
>>>>>>>>>>> Did you configure virsh as the power type for your nodes? I think that
>>>>>>>>>>> needs to be set before you can start a node.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>  • Continue to the juju wiki. However, since I'm using 12.04 I did not
>>>>>>>>>>>> add the PPA as indicated in the 'Before you get going' section.
>>>>>>>>>>>>
>>>>>>>>>>>> Running juju on the command-line of the MAAS server returns this error:
>>>>>>>>>>>> ==
>>>>>>>>>>>> $ juju bootstrap
>>>>>>>>>>>> 2012-04-18 18:23:50,052 INFO Bootstrapping environment 'maas' (origin:
>>>>>>>>>>>> distro type: maas)...
>>>>>>>>>>>> Unhandled Error
>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/juju/control/__init__.py",
>>>>>>>>>>>> line 188, in main
>>>>>>>>>>>>    options.command(options)
>>>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/juju/control/command.py",
>>>>>>>>>>>> line 44, in __call__
>>>>>>>>>>>>    reactor.run()
>>>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py",
>>>>>>>>>>>> line 1169, in run
>>>>>>>>>>>>    self.mainLoop()
>>>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py",
>>>>>>>>>>>> line 1178, in mainLoop
>>>>>>>>>>>>    self.runUntilCurrent()
>>>>>>>>>>>> --- <exception caught here> ---
>>>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py",
>>>>>>>>>>>> line 800, in runUntilCurrent
>>>>>>>>>>>>    call.func(*call.args, **call.kw)
>>>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/twisted/internet/tcp.py",
>>>>>>>>>>>> line 362, in resolveAddress
>>>>>>>>>>>>    self._setRealAddress(self.addr[0])
>>>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/twisted/internet/tcp.py",
>>>>>>>>>>>> line 369, in _setRealAddress
>>>>>>>>>>>>    self.doConnect()
>>>>>>>>>>>>  File "/usr/lib/python2.7/dist-packages/twisted/internet/tcp.py",
>>>>>>>>>>>> line 395, in doConnect
>>>>>>>>>>>>    connectResult = self.socket.connect_ex(self.realAddress)
>>>>>>>>>>>>  File "/usr/lib/python2.7/socket.py", line 224, in meth
>>>>>>>>>>>>    return getattr(self._sock,name)(*args)
>>>>>>>>>>>> exceptions.TypeError: an integer is required
>>>>>>>>>>>> Unexpected TimeoutError interacting with provider: User timeout caused
>>>>>>>>>>>> connection failure.
>>>>>>>>>>>> 2012-04-18 18:24:20,090 ERROR Unexpected TimeoutError interacting with
>>>>>>>>>>>> provider: User timeout caused connection failure.
>>>>>>>>>>>> $
>>>>>>>>>>>> ==
>>>>>>>>>>>>
>>>>>>>>>>>> I've attempted both with and without the juju PPA but it still
>>>>>>>>>>>> produces the same results.
>>>>>>>>>>>
>>>>>>>>>>> Does juju --verbose bootstrap gives you any more info?
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I hope this feedback is helpful and am excited to continue working on
>>>>>>>>>>>> this in anticipation of the 12.04 release!
>>>>>>>>>>>
>>>>>>>>>>> It is very useful, please keep it coming! :-)
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> Diogo
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -Mike
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Apr 18, 2012 at 11:25 PM, Jeroen Vermeulen <jtv@xxxxxxxxxxxxx> wrote:
>>>>>>>>>>>>> On 2012-04-19 01:30, Michael Hughes wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Maybe I'm using am incompatible base OS. What are others testing MAAS
>>>>>>>>>>>>>> with?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> In the development team we all run 12.04.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Looks like you to change the maas_url setting to adapt to your setup. This
>>>>>>>>>>>>> setting tells nodes where they can reach the MAAS service.  The URL's
>>>>>>>>>>>>> hostname part defaults to the IP address of whatever interface your server
>>>>>>>>>>>>> uses for its default route, but it sounds like that's not going to be right
>>>>>>>>>>>>> for your network.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I don't think we have the UI for such settings accessible yet.  But you can
>>>>>>>>>>>>> change the setting directly in the database:
>>>>>>>>>>>>>
>>>>>>>>>>>>>    INSERT INTO maasserver_config (name, value)
>>>>>>>>>>>>>    VALUES ('maas_url', 'http://<ip>/MAAS/');
>>>>>>>>>>>>>
>>>>>>>>>>>>> The <ip> is the address by which the nodes can reach the server.  The
>>>>>>>>>>>>> “MAAS/” path is as what you'd use to get to the MAAS user interface (we use
>>>>>>>>>>>>> just “/” on development setups).
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Jeroen
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Mailing list: https://launchpad.net/~maas-devel
>>>>>>>>>>>> Post to     : maas-devel@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>> Unsubscribe : https://launchpad.net/~maas-devel
>>>>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Diogo M. Matsubara
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Diogo M. Matsubara
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Diogo M. Matsubara
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Diogo M. Matsubara
>>>>
>>>>
>>>>
>>>> --
>>>> Diogo M. Matsubara
>>
>>
>>
>> --
>> Diogo M. Matsubara



-- 
Diogo M. Matsubara