Kernel panic - not syncing: An NMI occurred on HP Dl360g6
Hello, can somebody help me getting to the bottom of this panic? vdsm.log was empty (stopped 10min prior panic) and is rotated away already (forgot to save it...) Might this be [1]? However, I do not have an active subscription so I cannot acccess the helpdoc. Thanks! [1] https://access.redhat.com/solutions/442193 [64457.990217] kvm: zapping shadow pages for mmio generation wraparound [64469.916912] kvm [21434]: vcpu0 unhandled rdmsr: 0x1ad [71035.222120] ixgbe 0000:04:00.0 ens1f0: NIC Link is Down [71035.223699] storageA: port 1(ens1f0.2) entered disabled state [71035.223932] storageB: port 1(ens1f0.3) entered disabled state [71035.224173] server: port 1(ens1f0.10) entered disabled state [71035.224447] workstations: port 1(ens1f0.11) entered disabled state [71035.572619] bnx2 0000:02:00.0 enp2s0f0: NIC Copper Link is Down [71036.225789] ovirtmgmt: port 1(enp2s0f0) entered disabled state [71091.146266] hpwdt: Unexpected close, not stopping watchdog! [71138.579788] ovirtmgmt: port 2(vnet0) entered disabled state [71138.580067] device vnet0 left promiscuous mode [71138.580079] ovirtmgmt: port 2(vnet0) entered disabled state [71146.413292] Kernel panic - not syncing: An NMI occurred. Depending on your system the reason for the NMI is logged in any one of the following resources: 1. Integrated Management Log (IML) 2. OA Syslog 3. OA Forward Progress Log 4. iLO Event Log [71146.739719] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G I -------------- 3.10.0-229.4.2.el7.x86_64 #1 [71146.858234] Hardware name: HP ProLiant DL360 G6, BIOS P64 07/02/2013 [71146.934129] ffffffffa026d2d8 f7d5e6c540f56fc7 ffff8807ffc05de0 ffffffff81604eaa [71147.022535] ffff8807ffc05e60 ffffffff815fe71e 0000000000000008 ffff8807ffc05e70 [71147.111065] ffff8807ffc05e10 f7d5e6c540f56fc7 ffffffff8101bad9 ffffc9000a806072 [71147.199595] Call Trace: [71147.228779] <NMI> [<ffffffff81604eaa>] dump_stack+0x19/0x1b [71147.297491] [<ffffffff815fe71e>] panic+0xd8/0x1e7 [71147.354746] [<ffffffff8101bad9>] ? sched_clock+0x9/0x10 [71147.418233] [<ffffffffa026c8ed>] hpwdt_pretimeout+0xdd/0xe0 [hpwdt] [71147.494192] [<ffffffff8160d6d9>] nmi_handle.isra.0+0x69/0xb0 [71147.562873] [<ffffffff8160d8dd>] do_nmi+0x1bd/0x340 [71147.622201] [<ffffffff8160cb31>] end_repeat_nmi+0x1e/0x2e [71147.687768] [<ffffffff8133d4d4>] ? intel_idle+0xe4/0x170 [71147.752291] [<ffffffff8133d4d4>] ? intel_idle+0xe4/0x170 [71147.816818] [<ffffffff8133d4d4>] ? intel_idle+0xe4/0x170 [71147.881343] <<EOE>> [<ffffffff814aaa20>] cpuidle_enter_state+0x40/0xc0 [71147.961483] [<ffffffff814aab65>] cpuidle_idle_call+0xc5/0x200 [71148.031208] [<ffffffff8101d21e>] arch_cpu_idle+0xe/0x30 [71148.094694] [<ffffffff810c6985>] cpu_startup_entry+0xf5/0x290 [71148.164420] [<ffffffff815f33e7>] rest_init+0x77/0x80 [71148.224791] [<ffffffff81a45057>] start_kernel+0x429/0x44a [71148.290354] [<ffffffff81a44a37>] ? repair_env_string+0x5c/0x5c [71148.361120] [<ffffffff81a44120>] ? early_idt_handlers+0x120/0x120 [71148.434999] [<ffffffff81a445ee>] x86_64_start_reservations+0x2a/0x2c [71148.511997] [<ffffffff81a44742>] x86_64_start_kernel+0x152/0x175 -- Daniel Helgenberger m box bewegtbild GmbH P: +49/30/2408781-22 F: +49/30/2408781-10 ACKERSTR. 19 D-10115 BERLIN www.m-box.de www.monkeymen.tv Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
Hi, If you've had a crash caused by an NMI it can often be a hardware issue. Maybe worth running some H/W diagnostics. Cheers Alex On 27/05/15 15:40, Daniel Helgenberger wrote:
Hello,
can somebody help me getting to the bottom of this panic? vdsm.log was empty (stopped 10min prior panic) and is rotated away already (forgot to save it...)
Might this be [1]? However, I do not have an active subscription so I cannot acccess the helpdoc.
Thanks!
[1] https://access.redhat.com/solutions/442193
[64457.990217] kvm: zapping shadow pages for mmio generation wraparound [64469.916912] kvm [21434]: vcpu0 unhandled rdmsr: 0x1ad [71035.222120] ixgbe 0000:04:00.0 ens1f0: NIC Link is Down [71035.223699] storageA: port 1(ens1f0.2) entered disabled state [71035.223932] storageB: port 1(ens1f0.3) entered disabled state [71035.224173] server: port 1(ens1f0.10) entered disabled state [71035.224447] workstations: port 1(ens1f0.11) entered disabled state [71035.572619] bnx2 0000:02:00.0 enp2s0f0: NIC Copper Link is Down [71036.225789] ovirtmgmt: port 1(enp2s0f0) entered disabled state [71091.146266] hpwdt: Unexpected close, not stopping watchdog! [71138.579788] ovirtmgmt: port 2(vnet0) entered disabled state [71138.580067] device vnet0 left promiscuous mode [71138.580079] ovirtmgmt: port 2(vnet0) entered disabled state [71146.413292] Kernel panic - not syncing: An NMI occurred. Depending on your system the reason for the NMI is logged in any one of the following resources: 1. Integrated Management Log (IML) 2. OA Syslog 3. OA Forward Progress Log 4. iLO Event Log [71146.739719] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G I -------------- 3.10.0-229.4.2.el7.x86_64 #1 [71146.858234] Hardware name: HP ProLiant DL360 G6, BIOS P64 07/02/2013 [71146.934129] ffffffffa026d2d8 f7d5e6c540f56fc7 ffff8807ffc05de0 ffffffff81604eaa [71147.022535] ffff8807ffc05e60 ffffffff815fe71e 0000000000000008 ffff8807ffc05e70 [71147.111065] ffff8807ffc05e10 f7d5e6c540f56fc7 ffffffff8101bad9 ffffc9000a806072 [71147.199595] Call Trace: [71147.228779] <NMI> [<ffffffff81604eaa>] dump_stack+0x19/0x1b [71147.297491] [<ffffffff815fe71e>] panic+0xd8/0x1e7 [71147.354746] [<ffffffff8101bad9>] ? sched_clock+0x9/0x10 [71147.418233] [<ffffffffa026c8ed>] hpwdt_pretimeout+0xdd/0xe0 [hpwdt] [71147.494192] [<ffffffff8160d6d9>] nmi_handle.isra.0+0x69/0xb0 [71147.562873] [<ffffffff8160d8dd>] do_nmi+0x1bd/0x340 [71147.622201] [<ffffffff8160cb31>] end_repeat_nmi+0x1e/0x2e [71147.687768] [<ffffffff8133d4d4>] ? intel_idle+0xe4/0x170 [71147.752291] [<ffffffff8133d4d4>] ? intel_idle+0xe4/0x170 [71147.816818] [<ffffffff8133d4d4>] ? intel_idle+0xe4/0x170 [71147.881343] <<EOE>> [<ffffffff814aaa20>] cpuidle_enter_state+0x40/0xc0 [71147.961483] [<ffffffff814aab65>] cpuidle_idle_call+0xc5/0x200 [71148.031208] [<ffffffff8101d21e>] arch_cpu_idle+0xe/0x30 [71148.094694] [<ffffffff810c6985>] cpu_startup_entry+0xf5/0x290 [71148.164420] [<ffffffff815f33e7>] rest_init+0x77/0x80 [71148.224791] [<ffffffff81a45057>] start_kernel+0x429/0x44a [71148.290354] [<ffffffff81a44a37>] ? repair_env_string+0x5c/0x5c [71148.361120] [<ffffffff81a44120>] ? early_idt_handlers+0x120/0x120 [71148.434999] [<ffffffff81a445ee>] x86_64_start_reservations+0x2a/0x2c [71148.511997] [<ffffffff81a44742>] x86_64_start_kernel+0x152/0x175
-- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. "Transact" is operated by Integrated Financial Arrangements plc. 29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608 5300. (Registered office: as above; Registered in England and Wales under number: 3727592). Authorised and regulated by the Financial Conduct Authority (entered on the Financial Services Register; no. 190856).
On 27.05.2015 16:50, Alex Crow wrote:
Hi, Hello,
thanks for getting back to me.
If you've had a crash caused by an NMI it can often be a hardware issue.
The first thing I checked is IML logs - nothing there. These machines are quite resilient and report almost everything. The HW diag is fine...
Maybe worth running some H/W diagnostics.
Cheers
Alex
On 27/05/15 15:40, Daniel Helgenberger wrote:
Hello,
can somebody help me getting to the bottom of this panic? vdsm.log was empty (stopped 10min prior panic) and is rotated away already (forgot to save it...)
Might this be [1]? However, I do not have an active subscription so I cannot acccess the helpdoc.
Thanks!
[1] https://access.redhat.com/solutions/442193
[64457.990217] kvm: zapping shadow pages for mmio generation wraparound [64469.916912] kvm [21434]: vcpu0 unhandled rdmsr: 0x1ad [71035.222120] ixgbe 0000:04:00.0 ens1f0: NIC Link is Down [71035.223699] storageA: port 1(ens1f0.2) entered disabled state [71035.223932] storageB: port 1(ens1f0.3) entered disabled state [71035.224173] server: port 1(ens1f0.10) entered disabled state [71035.224447] workstations: port 1(ens1f0.11) entered disabled state [71035.572619] bnx2 0000:02:00.0 enp2s0f0: NIC Copper Link is Down [71036.225789] ovirtmgmt: port 1(enp2s0f0) entered disabled state [71091.146266] hpwdt: Unexpected close, not stopping watchdog! [71138.579788] ovirtmgmt: port 2(vnet0) entered disabled state [71138.580067] device vnet0 left promiscuous mode [71138.580079] ovirtmgmt: port 2(vnet0) entered disabled state [71146.413292] Kernel panic - not syncing: An NMI occurred. Depending on your system the reason for the NMI is logged in any one of the following resources: 1. Integrated Management Log (IML) 2. OA Syslog 3. OA Forward Progress Log 4. iLO Event Log [71146.739719] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G I -------------- 3.10.0-229.4.2.el7.x86_64 #1 [71146.858234] Hardware name: HP ProLiant DL360 G6, BIOS P64 07/02/2013 [71146.934129] ffffffffa026d2d8 f7d5e6c540f56fc7 ffff8807ffc05de0 ffffffff81604eaa [71147.022535] ffff8807ffc05e60 ffffffff815fe71e 0000000000000008 ffff8807ffc05e70 [71147.111065] ffff8807ffc05e10 f7d5e6c540f56fc7 ffffffff8101bad9 ffffc9000a806072 [71147.199595] Call Trace: [71147.228779] <NMI> [<ffffffff81604eaa>] dump_stack+0x19/0x1b [71147.297491] [<ffffffff815fe71e>] panic+0xd8/0x1e7 [71147.354746] [<ffffffff8101bad9>] ? sched_clock+0x9/0x10 [71147.418233] [<ffffffffa026c8ed>] hpwdt_pretimeout+0xdd/0xe0 [hpwdt] [71147.494192] [<ffffffff8160d6d9>] nmi_handle.isra.0+0x69/0xb0 [71147.562873] [<ffffffff8160d8dd>] do_nmi+0x1bd/0x340 [71147.622201] [<ffffffff8160cb31>] end_repeat_nmi+0x1e/0x2e [71147.687768] [<ffffffff8133d4d4>] ? intel_idle+0xe4/0x170 [71147.752291] [<ffffffff8133d4d4>] ? intel_idle+0xe4/0x170 [71147.816818] [<ffffffff8133d4d4>] ? intel_idle+0xe4/0x170 [71147.881343] <<EOE>> [<ffffffff814aaa20>] cpuidle_enter_state+0x40/0xc0 [71147.961483] [<ffffffff814aab65>] cpuidle_idle_call+0xc5/0x200 [71148.031208] [<ffffffff8101d21e>] arch_cpu_idle+0xe/0x30 [71148.094694] [<ffffffff810c6985>] cpu_startup_entry+0xf5/0x290 [71148.164420] [<ffffffff815f33e7>] rest_init+0x77/0x80 [71148.224791] [<ffffffff81a45057>] start_kernel+0x429/0x44a [71148.290354] [<ffffffff81a44a37>] ? repair_env_string+0x5c/0x5c [71148.361120] [<ffffffff81a44120>] ? early_idt_handlers+0x120/0x120 [71148.434999] [<ffffffff81a445ee>] x86_64_start_reservations+0x2a/0x2c [71148.511997] [<ffffffff81a44742>] x86_64_start_kernel+0x152/0x175
-- Daniel Helgenberger m box bewegtbild GmbH P: +49/30/2408781-22 F: +49/30/2408781-10 ACKERSTR. 19 D-10115 BERLIN www.m-box.de www.monkeymen.tv Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
Do you have chance to access iLO web administration page of the server and check Integrated management log (IML) And/Or ILO event log And verify if anything has ben registered there?
Hello Gianluca, On 27.05.2015 17:42, Gianluca Cecchi wrote:
Do you have chance to access iLO web administration page of the server and check Integrated management log (IML) And/Or ILO event log And verify if anything has ben registered there?
no, strangely. I only see the server reset in iLO logs; IML log is clean.
-- Daniel Helgenberger m box bewegtbild GmbH P: +49/30/2408781-22 F: +49/30/2408781-10 ACKERSTR. 19 D-10115 BERLIN www.m-box.de www.monkeymen.tv Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
participants (3)
-
Alex Crow -
Daniel Helgenberger -
Gianluca Cecchi