[ovirt-users] [OVIRT-3.5-TEST-DAY-3] : fence kdump integration

[Eli Mesika]

I was testing fence kdump integration.

Due to the following issues I was not able to test over JSON RPC and all tests were done with XML RPC

1) Manually applied : http://gerrit.ovirt.org/#/c/32843 (this was not included yet 
   in 3.5 and cause an exception in VDSM when fenceNode is called)

2) JSON : does not return status correctly : Test Succeeded, null
   Therefor , can not stop/start/restart host 
   Related to engine patch : http://gerrit.ovirt.org/#/c/32855/
   not back-ported yet to 3.5 (blocker IMO)

--------------------------------------------------------------
ALL tests below should be redone after resolving JSON issues !
--------------------------------------------------------------

-------------------
General comment
-------------------

All fence kdump flows are related to a scenario in which host became non responsive.
In the case of manual PM action (stop/restart) fence kdump is not taking place.
I think this should be fully documented in order to prevent misunderstanding when the kdump flag
is checked and host is rebooted/stopped manual while kdumping' which will result in losing the dump file.


*********************************
Tests using XML RPC 
*********************************

------------------------
Installation tests :
------------------------

1) Adding a host with Detect kdump flow set to on and without crashkernel command line parameter
   Result: host installation is OK, but warning message is displayed in Events tab and Audit log 

   TEST PASSED

2) Adding a host with Detect kdump flow set to on, with crashkernel command line parameter, but without      
   required version of kexec-tools package
   Result: host installation is OK, but warning message is displayed in Events tab and Audit log 

   TEST PASSED

3) Adding a host with Detect kdump flow set to on, with crashkernel command line parameter and with required 
   version of kexec-tools package
   Result: host installation is OK, in General tab of host detail view you should see Kdump Status: Enabled 

   TEST PASSED

------------------------
Kdump detection tests:
------------------------

1) Crashdumping a host with kdump detection disabled

    Prerequisites: host was successfully deployed with Detect kdump flow set to off, fence_kdump listener 
    is running
    Result: Host changes its status Up -> Connecting -> Non Responsive -> Reboot -> Non Responsive -> Up, 
    hard fencing is executed 

    TEST PASSED

2) Crashdumping a host with kdump detection enabled

    Prerequisites: host was successfully deployed with Detect kdump flow set to on, fence_kdump listener 
    is running
    Result: Host changes its status Up -> Connecting -> Non Responsive -> Kdumping -> Non Responsive -> Up,
    hard fencing is not executed, there are messages in Events tab Kdump flow detected on host and Kdump 
    flow finished on host  

    TEST PASSED

3) Crashdumping a host with kdump detection enabled but fence_kdump listener down

    Prerequisites: host was successfully deployed with Detect kdump flow set to on, fence_kdump listener 
    is not running
    Result: Host changes its status Up -> Connecting -> Non Responsive -> Reboot -> Non Responsive -> Up, 
    hard fencing is executed, there's message in Events tab Kdump detection for host had started, 
    but fence_kdump listener is not running  

    TEST PASSED


4) Host with kdump detection enabled, fence_kdump listener is running, but network between engine 
   and host is down

    Prerequisites: host was successfully deployed with Detect kdump flow set to on, fence_kdump listener 
    is running, alter firewall rules on engine to drop everything coming from host's IP address
    Result: Host changes its status Up -> Connecting -> Non Responsive -> Reboot -> Non Responsive -> Up,
    hard fencing is executed, there's message in Events tab Kdump flow not detected on host  

    TEST PASSED

5) Crashdumping a host with kdump detection enabled, fence_kdump listener is running, stop fence_kdump
   listener during kdump

    Prerequisites: host was successfully deployed with Detect kdump flow set to on, fence_kdump listener
    is running
    Actions: When host status is changed to Kdumping, stop fence_kdump listener
    Result: Host changes its status Up -> Connecting -> Non Responsive -> Kdumping -> Reboot -> Non Responsive
    -> Up, hard fencing is executed, there are messages in Events tab Kdump flow detected on host and Kdump
    detection for host had started, but fence_kdump listener is not running  

    TEST PASSED           I got this message in event log :
                          Unable to determine if Kdump is in progress on host 'pluto-vdsf', because fence_kdump listener is not running.
                          Is this OK ?


6) Crashdumping a host with kdump detection enabled, fence_kdump listener is running, 
   restart engine during kdump

    Prerequisites: host was successfully deployed with Detect kdump flow set to on, 
    fence_kdump listener is running
    Actions: When host status is changed to Kdumping, restart engine
    Result: Host changes its status Up -> Connecting -> Non Responsive -> Kdumping, hard fencing is not
    executed, there are messages in Events tab Kdump flow detected on host, after engine restart host 
    stays in Kdumping status for the period of DisableFenceAtStartupInSec seconds, after that there 
    are messages in Events tab Kdump flow detected on host and Kdump flow finished on host and 
    changes status Kdumping -> Non Responsive -> Up  

    TEST PASSED           I got only this message in event log :
                          Kdump flow is in progress on host 'pluto-vdsf'.
                          Is this OK?












   







Thanks
Eli Mesika