← Back to team overview

openstack team mailing list archive

Re: Can't ssh into instance

 

Thank you very much guys  Sébastien and Leander

It was very interesting case.

I thought if I see this lines in compute.log then sshkey  injection was
successfully

1fc4e8b132647ec] Running cmd (subprocess): sudo nova-rootwrap qemu-nbd
-c /dev/nbd15 /var/lib/nova/instances/instance-0000000c/disk from
(pid=1208) execute /usr/lib/python2.7/dist-packages/nova/utils.py:219
2012-05-24 13:25:00 DEBUG nova.utils
[req-5c28365b-7b33-4788-b044-d0144efaeeb8
b568e6c7911042339705042bb75ee1a7 1a4f83e3463b403f91fc4e8b132647ec]
Running cmd (subprocess): sudo nova-rootwrap kpartx -a /dev/nbd15 from
(pid=1208) execute /usr/lib/python2.7/dist-packages/nova/utils.py:219
2012-05-24 13:25:00 DEBUG nova.utils
[req-5c28365b-7b33-4788-b044-d0144efaeeb8
b568e6c7911042339705042bb75ee1a7 1a4f83e3463b403f91fc4e8b132647ec]
Running cmd (subprocess): sudo nova-rootwrap mount /dev/mapper/nbd15p1
/tmp/tmpffuOQJ from (pid=1208) execute
/usr/lib/python2.7/dist-packages/nova/utils.py:219
2012-05-24 13:25:00 DEBUG nova.utils
[req-5c28365b-7b33-4788-b044-d0144efaeeb8
b568e6c7911042339705042bb75ee1a7 1a4f83e3463b403f91fc4e8b132647ec]
Running cmd (subprocess): sudo nova-rootwrap mkdir -p
/tmp/tmpffuOQJ/root/.ssh from (pid=1208) execute
/usr/lib/python2.7/dist-packages/nova/utils.py:219
2012-05-24 13:25:00 DEBUG nova.utils
[req-5c28365b-7b33-4788-b044-d0144efaeeb8
b568e6c7911042339705042bb75ee1a7 1a4f83e3463b403f91fc4e8b132647ec]
Running cmd (subprocess): sudo nova-rootwrap chown root
/tmp/tmpffuOQJ/root/.ssh from (pid=1208) execute
/usr/lib/python2.7/dist-packages/nova/utils.py:219
2012-05-24 13:25:00 DEBUG nova.utils
[req-5c28365b-7b33-4788-b044-d0144efaeeb8
b568e6c7911042339705042bb75ee1a7 1a4f83e3463b403f91fc4e8b132647ec]
Running cmd (subprocess): sudo nova-rootwrap chmod 700
/tmp/tmpffuOQJ/root/.ssh from (pid=1208) execute
/usr/lib/python2.7/dist-packages/nova/utils.py:219
2012-05-24 13:25:00 DEBUG nova.utils
[req-5c28365b-7b33-4788-b044-d0144efaeeb8
b568e6c7911042339705042bb75ee1a7 1a4f83e3463b403f91fc4e8b132647ec]
Running cmd (subprocess): sudo nova-rootwrap tee -a
/tmp/tmpffuOQJ/root/.ssh/authorized_keys from (pid=1208) execute
/usr/lib/python2.7/dist-packages/nova/utils.py:219
2012-05-24 13:25:00 DEBUG nova.utils
[req-5c28365b-7b33-4788-b044-d0144efaeeb8
b568e6c7911042339705042bb75ee1a7 1a4f83e3463b



it's from Leander compute log

http://paste.openstack.org/show/18149/


And I was sure I have few vms  with metadata server access issue and
without ssh connection issue .


It looks like I need some additional tests and need more time for code
reading : - )

And by the way if we did not have ssh key in authorised key I thought
we would have different output from ssh client

something like this

debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA b8:7e:bd:80:ae:72:61:51:8f:d0:fc:e1:7d:47:81:a6
debug1: Host '10.1.0.7' is known and matches the ECDSA host key.
debug1: Found key in /root/.ssh/known_hosts:2
debug1: ssh_ecdsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: rhelkey.pem
debug1: read PEM private key done: type RSA
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
Permission denied (publickey).


Leander ouput was looking strange for me:


ssh -i testkey.pem root@10.1.1.3 -v

OpenSSH_5.9p1 Debian-5ubuntu1, OpenSSL 1.0.1 14 Mar 2012

debug1: Reading configuration data /etc/ssh/ssh_config

debug1: /etc/ssh/ssh_config line 19: Applying options for *

debug1: Connecting to 10.1.1.3 [10.1.1.3] port 22.

debug1: Connection established.

debug1: identity file testkey.pem type -1

debug1: identity file testkey.pem-cert type -1

debug1: Remote protocol version 2.0, remote software version
OpenSSH_5.8p1 Debian-7ubuntu1

debug1: match: OpenSSH_5.8p1 Debian-7ubuntu1 pat OpenSSH*

debug1: Enabling compatibility mode for protocol 2.0

debug1: Local version string SSH-2.0-OpenSSH_5.9p1 Debian-5ubuntu1

debug1: SSH2_MSG_KEXINIT sent

Read from socket failed: Connection reset by peer



" Connection reset by peer " after "debug1: SSH2_MSG_KEXINIT sent" it
looks like network issue.


Anyway thank you very much.



On Fri, May 25, 2012 at 3:19 AM, Sébastien Han <han.sebastien@xxxxxxxxx>wrote:

> Ok ok, no offense in my previous emails :)
> Good to know that everything is working.
>
> Cheers.
>
>
>
>
> On Fri, May 25, 2012 at 1:00 AM, Leander Bessa Beernaert <
> leanderbb@xxxxxxxxx> wrote:
>
>> I'm in no way saying that openstack is to blame for the current problem,
>> but it occurred to me that my install could have script failed somewhere
>> along the way without me knowing (i might have forgotten to mention that in
>> the previous post :s).
>>
>> Adding those two lines solved my problem.  I've already mentioned it
>> earlier, perhaps the mail wasn't sent to the list. And yes, i'm still using
>> a all-in-one setup for now.
>>
>>  Thanks for the tip.
>>
>>
>> On Thu, May 24, 2012 at 9:03 PM, Sébastien Han <han.sebastien@xxxxxxxxx>wrote:
>>
>>> Why did you reinstall everything?
>>> There is no "just in case", I mean you solved your issue, it was from
>>> your configuration not from openstack :)
>>>
>>> It's a routing issue, same as earlier.
>>> Check again those parameters, specially the first one:
>>>
>>>    - --routing_source_ip=IP_CURRENT_NODE
>>>    - --my_ip=IP_CURRENT_NODE
>>>
>>> Still in all-in-one setup?
>>>
>>> Cheers :)
>>>
>>> On Thu, May 24, 2012 at 8:40 PM, Matt Joyce <matt.joyce@xxxxxxxxxxxxxxxx
>>> > wrote:
>>>
>>>> First rule of security group.  Do not talk about security group.  <j/k>
>>>>
>>>>
>>>> On Thu, May 24, 2012 at 9:35 AM, Rogerio Goncalves <rogerlz@xxxxxxxxx>wrote:
>>>>
>>>>> Maybe you missed the rules of security group?
>>>>>
>>>>> Rogério Gonçalves
>>>>> rogerlz@xxxxxxxxx
>>>>> Cel: (11) 8840-9790
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, May 24, 2012 at 12:12 PM, Leander Bessa Beernaert <
>>>>> leanderbb@xxxxxxxxx> wrote:
>>>>>
>>>>>> I've formatted the host machine and reinstalled openstack, just in
>>>>>> case. Now i am only getting "connection refused".
>>>>>>
>>>>>>
>>>>>> On Thu, May 24, 2012 at 3:01 PM, Leander Bessa Beernaert <
>>>>>> leanderbb@xxxxxxxxx> wrote:
>>>>>>
>>>>>>> Compute log: Log: http://paste.openstack.org/show/18149/
>>>>>>>
>>>>>>> I've tried bot root and ubuntu as account names (ssh -i key.pem
>>>>>>> root@10.1.2.3 and  ssh -i key.pem ubuntu@10.1.2.3) and the result
>>>>>>> is still "Read from socket failed: Connection reset by peer"
>>>>>>>
>>>>>>>
>>>>>>> On Thu, May 24, 2012 at 1:57 PM, Leander Bessa Beernaert <
>>>>>>> leanderbb@xxxxxxxxx> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> Complete log: http://paste.openstack.org/show/18144/
>>>>>>>>
>>>>>>>> On Thu, May 24, 2012 at 1:49 PM, Anton Haldin <
>>>>>>>> ahaldin@xxxxxxxxxxxxxxxx> wrote:
>>>>>>>>
>>>>>>>>> you can try to look in  console.log for this instance
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, May 24, 2012 at 4:41 PM, Leander Bessa Beernaert <
>>>>>>>>> leanderbb@xxxxxxxxx> wrote:
>>>>>>>>>
>>>>>>>>>> No, at the moment the vnc console isn't working yet. I haven't
>>>>>>>>>> gotten that far yet.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, May 24, 2012 at 1:29 PM, Anton Haldin <
>>>>>>>>>> ahaldin@xxxxxxxxxxxxxxxx> wrote:
>>>>>>>>>>
>>>>>>>>>>> t can be an issue on OS side in instance ?
>>>>>>>>>>>
>>>>>>>>>>> do you have vnc access for this instance?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, May 24, 2012 at 2:56 PM, Leander Bessa Beernaert <
>>>>>>>>>>> leanderbb@xxxxxxxxx> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello,
>>>>>>>>>>>>
>>>>>>>>>>>> I'm having trouble sshing into the created instances. At first
>>>>>>>>>>>> i was getting the following error:
>>>>>>>>>>>>
>>>>>>>>>>>> ssh -i testkey.pem root@10.1.1.3 -v
>>>>>>>>>>>>>
>>>>>>>>>>>>> OpenSSH_5.9p1 Debian-5ubuntu1, OpenSSL 1.0.1 14 Mar 2012
>>>>>>>>>>>>>
>>>>>>>>>>>>> debug1: Reading configuration data /etc/ssh/ssh_config
>>>>>>>>>>>>>
>>>>>>>>>>>>> debug1: /etc/ssh/ssh_config line 19: Applying options for *
>>>>>>>>>>>>>
>>>>>>>>>>>>> debug1: Connecting to 10.1.1.3 [10.1.1.3] port 22.
>>>>>>>>>>>>>
>>>>>>>>>>>>> debug1: Connection established.
>>>>>>>>>>>>>
>>>>>>>>>>>>> debug1: identity file testkey.pem type -1
>>>>>>>>>>>>>
>>>>>>>>>>>>> debug1: identity file testkey.pem-cert type -1
>>>>>>>>>>>>>
>>>>>>>>>>>>> debug1: Remote protocol version 2.0, remote software version
>>>>>>>>>>>>>> OpenSSH_5.8p1 Debian-7ubuntu1
>>>>>>>>>>>>>
>>>>>>>>>>>>> debug1: match: OpenSSH_5.8p1 Debian-7ubuntu1 pat OpenSSH*
>>>>>>>>>>>>>
>>>>>>>>>>>>> debug1: Enabling compatibility mode for protocol 2.0
>>>>>>>>>>>>>
>>>>>>>>>>>>> debug1: Local version string SSH-2.0-OpenSSH_5.9p1
>>>>>>>>>>>>>> Debian-5ubuntu1
>>>>>>>>>>>>>
>>>>>>>>>>>>> debug1: SSH2_MSG_KEXINIT sent
>>>>>>>>>>>>>
>>>>>>>>>>>>> Read from socket failed: Connection reset by peer
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> I then proceeded to reboot the machine and all it's services.
>>>>>>>>>>>> However, now i can't even get that far. I'm alway faced with a connection
>>>>>>>>>>>> refused.
>>>>>>>>>>>>
>>>>>>>>>>>> I've added the permissions for port 22 and icmp in the default
>>>>>>>>>>>> security group and i'm also able to ping the instances.
>>>>>>>>>>>>
>>>>>>>>>>>> I'm using the openstack packages provided with ubuntu 12.04.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>>
>>>>>>>>>>>> Leander
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Mailing list: https://launchpad.net/~openstack
>>>>>>>>>>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>> Unsubscribe : https://launchpad.net/~openstack
>>>>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Mailing list: https://launchpad.net/~openstack
>>>>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>>>>>> Unsubscribe : https://launchpad.net/~openstack
>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list: https://launchpad.net/~openstack
>>>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~openstack
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~openstack
>>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~openstack
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>

Follow ups

References