kernel-packages team mailing list archive

[bugproxy <bugproxy@xxxxxxxxxx>]

This seems to be a timing issue between network coming up and kdump-tools service
trying to access the remote target with ssh because when I get into emergency shell
after failure, this is what I get to see..

---
[FAILED] Failed to start Kernel crash dump capture service.
See 'systemctl status kdump-tools.service' for details.
Welcome to emergency mode! After logging in, type "journalctl -xb" to view
system logs, "systemctl reboot" to reboot, "systemctl default" or ^D to
try again to boot into default mode.
Give root password for maintenance
(or press Control-D to continue):
root@pinelp2:~# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 56:2d:60:47:c4:02 brd ff:ff:ff:ff:ff:ff
3: enP257p80s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq portid 40f2e93188c0 state UP group default qlen 1000
link/ether 40:f2:e9:31:88:c0 brd ff:ff:ff:ff:ff:ff
inet 10.33.11.58/16 brd 10.33.255.255 scope global enP257p80s0f0
valid_lft forever preferred_lft forever
inet6 2002:903:15f:1130:89c:e5b:6d8c:753f/64 scope global temporary dynamic
valid_lft 604746sec preferred_lft 85746sec
inet6 2002:903:15f:1130:42f2:e9ff:fe31:88c0/64 scope global mngtmpaddr dynamic
valid_lft 2591946sec preferred_lft 604746sec
inet6 2002:926:3e2:1130:89c:e5b:6d8c:753f/64 scope global temporary dynamic
valid_lft 604746sec preferred_lft 85746sec
inet6 2002:926:3e2:1130:42f2:e9ff:fe31:88c0/64 scope global mngtmpaddr dynamic
valid_lft 2591946sec preferred_lft 604746sec
inet6 fe80::42f2:e9ff:fe31:88c0/64 scope link
valid_lft forever preferred_lft forever
4: enP257p80s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop portid 40f2e93188c2 state DOWN group default qlen 1000
link/ether 40:f2:e9:31:88:c2 brd ff:ff:ff:ff:ff:ff
root@pinelp2:~# hostname -I
10.33.11.58 2002:903:15f:1130:89c:e5b:6d8c:753f 2002:903:15f:1130:42f2:e9ff:fe31:88c0 2002:926:3e2:1130:89c:e5b:6d8c:753f 2002:926:3e2:1130:42f2:e9ff:fe31:88c0
root@pinelp2:~# ping 10.33.31.113
PING 10.33.31.113 (10.33.31.113) 56(84) bytes of data.
64 bytes from 10.33.31.113: icmp_seq=1 ttl=64 time=0.399 ms
64 bytes from 10.33.31.113: icmp_seq=2 ttl=64 time=0.239 ms
64 bytes from 10.33.31.113: icmp_seq=3 ttl=64 time=0.234 ms

--- 10.33.31.113 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.234/0.290/0.399/0.079 ms
root@pinelp2:~#
root@pinelp2:~#
root@pinelp2:~#
root@pinelp2:~# ssh -i /root/.ssh/kdump_id_rsa root@10.33.31.113 mkdir -p /var/crash/10.33.11.58-test
root@pinelp2:~#
---

Most importantly, ssh command to create dir on remote target succeeds
while that is what has failed leading to the below error message in the
first place

"Starting kdump-tools: ssh: connect to host 10.33.31.113 port 22:
Network is unreachable"

Looking into ways to fix that up.

Thanks
Hari

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to makedumpfile in Ubuntu.
https://bugs.launchpad.net/bugs/1571590

Title:
  pVM:pinelp2:ubuntu 16.04: Network is unreachable when using ssh in
  kdump

Status in makedumpfile package in Ubuntu:
  New

Bug description:
  Problem Description
  ============================
  When dumping to ssh target , kdump failed:

           Starting Kernel crash dump capture service...
  [   18.542292] kdump-tools[2748]: Starting kdump-tools: ssh: connect to host 10.33.31.113 port 22: Network is unreachable
  [   18.542894] kdump-tools[2748]:  * kdump-tools: Unable to reach remote server root@10.33.31.113. No reason to continue
  [   18.613376] kdump-tools[2748]: Mon, 11 Apr 2016 01:23:55 -0500
  [   19.310893] kdump-tools[2748]: Rebooting.
  [   19.652714] reboot: Restarting system

  But the weired thing is that nfs target works on the same system.
   
  ---uname output---
  Linux pinelp2 4.4.0-18-generic #34-Ubuntu SMP Wed Apr 6 14:00:30 UTC 2016 ppc64le ppc64le ppc64le GNU/Linux
   
  Steps to Reproduce
  ===================================
  1. install latest ubuntu 16.04 on pinelp2
  2. config kdump to use ssh:
  SSH="root@10.33.31.113"
  3. run "kdump-config propagate" as root 
  4. trigger kdump

   
  Userspace tool common name: kdump-tools 
   
  The userspace tool has the following bit modes: 64-bit 

  Userspace rpm: version 1:1.5.9-5

  == Comment: #8 - Ping Tian Han <pthan@xxxxxxxxxx> - 2016-04-17 21:58:52 ==
  (In reply to comment #6)
  > I am able to reproduce this issue as well. 
  > Ping, what is you try to dump on some other machine with ssh ? Does it still
  > complain the same ? Is the ssh failing to connect to the network ?

  I saw the same problem when dumping to medlp4 with ssh:

           Starting Kernel crash dump capture service...
  [   17.293028] kdump-tools[2733]: Starting kdump-tools: ssh: connect to host 10.33.7.181 port 22: Network is unreachable
  [   17.293692] kdump-tools[2733]:  * kdump-tools: Unable to reach remote server root@10.33.7.181. No reason to continue
  [   17.405333] kdump-tools[2733]: Sun, 17 Apr 2016 20:56:59 -0500
  [   18.536587] kdump-tools[2733]: Rebooting.
  [   18.903280] reboot: Restarting system

  but I can ssh log noto medlp4 before triggering kdump.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/makedumpfile/+bug/1571590/+subscriptions