- Users - oVirt List Archives

[Users] oVirt 3.4 test day: iscsi multipath
by Francesco Romani 27 Jan '14

27 Jan '14

Hi everyone, During the last oVirt 3.4 test day I played a bit with ISCSI multipath: http://www.ovirt.org/Feature/iSCSI-Multipath#Configure_iSCSI_Multipathing It took me longer than expected to properly set up things but, once done, everything looked fine at the config level (both seen from engine and the actual OS configuration) and worked well on my tests, including physical disconnections mid-I/O. Well done! :) -- Francesco Romani RedHat Engineering Virtualization R & D IRC: fromani

1 0

[Users] multiple storage domains
by Michal Skrivanek 27 Jan '14

27 Jan '14

Hey, I was trying to play with multiple storage domain during the 3.4 test day and even though it took me ages to set storage up correctly, the feature itself seems to be working just fine. Much easier to select what kind of storage I want to use…just Shared vs Local storage pool. VM could use volumes from different domains and I didn't have to worry about the right type (other than local vs shared, obviously:) Thanks, michal

1 0

[Users] CANCELED: Feature Review: Gluster Volume Snapshots
by Blue Jeans Network 26 Jan '14

26 Jan '14

------=_Part_794_1616979341.1390801575818 Content-Type: multipart/related; boundary="----=_Part_795_166231402.1390801575818" ------=_Part_795_166231402.1390801575818 Content-Type: multipart/alternative; boundary="----=_Part_796_2122432314.1390801575818" ------=_Part_796_2122432314.1390801575818 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Shubhendu Ram Tripathi (shtripat(a)redhat.com) has cancelled the following meeting: Meeting Title: Feature Review: Gluster Volume Snapshots Meeting Time: Monday January 27, 2014 6:30 p.m. IST / 1 hr ************************************************ Message/Description: Cancelled ************************************************ Blue Jeans Network 2012 ------=_Part_796_2122432314.1390801575818 Content-Type: text/html;charset="utf-8" Content-Transfer-Encoding: 7bit <table width="100%"> <tr bgcolor="#d8d8d8"> <td align="center"> <table width="600px" bgcolor="#FFFFFF" style="margin-top:20px;padding:0px 50px 10px 50px;margin-bottom:20px;text-align:left;"> <tr> <td style="padding-right:10px;padding-bottom:20px;"> <a href="http://bluejeans.com"> <img src="https://static.bluejeans.com/z1/static/images/bjn/BlueJeans_Logo.png" alt="Blue Jeans" style="border:none;"> </a> </td> </tr> <tr> <td style="font-size: 20px;color: #333333; padding-bottom:30px;"> <a href="mailto:shtripat@redhat.com">Shubhendu Ram Tripathi</a> has cancelled the following meeting: </td> </tr> <tr> <td style="padding-top:10px; font-size: 16px;"> Meeting Title: Feature Review: Gluster Volume Snapshots </td> </tr> <tr> <td style="padding-top:10px; padding-bottom: 10px; font-size: 16px;"> Meeting Time: Monday January 27, 2014 • 6:30 p.m. IST / 1 hr </td> </tr> <tr> <td> <hr size="1" noshade="noshade" style="line-height:1px;color:#CCCCCC;"> </td> </tr> <tr> <td style="style="padding-top:10px;""> Message/Description: Cancelled </td> </tr> <tr> <td style="line-height:1px;color:#CCCCCC;"> <hr size="1" noshade="noshade" style="line-height:1px;color:#CCCCCC;border-color:#CCCCCC;"> </td> </tr> <tr> <td style="font-size:11px;text-align:left"> ©Blue Jeans Network 2013 </td> </tr> </table> </td> </tr> </table> ------=_Part_796_2122432314.1390801575818 Content-Type: text/calendar; method=REQUEST; charset="utf-8" Content-Transfer-Encoding: 7bit BEGIN:VCALENDAR VERSION:2.0 METHOD:CANCEL PRODID:-//PYVOBJECT//NONSGML Version 1//EN BEGIN:VTIMEZONE TZID:Asia/Kolkata TZURL:http://tzurl.org/zoneinfo-outlook/Asia/Kolkata X-LIC-LOCATION:Asia/Kolkata BEGIN:STANDARD TZOFFSETFROM:+0530 TZOFFSETTO:+0530 TZNAME:IST DTSTART:19700101T000000 END:STANDARD END:VTIMEZONE BEGIN:VEVENT UID:0H44B_sN0PzTAnM2@H6D3JmW4G7v71tX DTSTART;TZID=Asia/Kolkata:20140127T183000 DTEND;TZID=Asia/Kolkata:20140127T193000 ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=DECLINED:mailto:u sers(a)ovirt.org CREATED:20140121T113648Z DESCRIPTION:\nShubhendu Ram Tripathi (shtripat(a)redhat.com) has cancelled t his meeting.\n\n************************************************\n\nMessag e/Description: Cancelled\n\n\n******************************************** ****\n\nBlue Jeans Network 2012\n DTSTAMP:20140127T054615Z LAST-MODIFIED:20140127T054615Z LOCATION: ORGANIZER:mailto:shtripat@redhat.com SEQUENCE:1 STATUS:CANCELLED SUMMARY:Feature Review: Gluster Volume Snapshots TRANSP:OPAQUE END:VEVENT END:VCALENDAR ------=_Part_796_2122432314.1390801575818-- ------=_Part_795_166231402.1390801575818-- ------=_Part_794_1616979341.1390801575818 Content-Type: application/ics; name=invite.ics Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=invite.ics BEGIN:VCALENDAR VERSION:2.0 METHOD:CANCEL PRODID:-//PYVOBJECT//NONSGML Version 1//EN BEGIN:VTIMEZONE TZID:Asia/Kolkata TZURL:http://tzurl.org/zoneinfo-outlook/Asia/Kolkata X-LIC-LOCATION:Asia/Kolkata BEGIN:STANDARD TZOFFSETFROM:+0530 TZOFFSETTO:+0530 TZNAME:IST DTSTART:19700101T000000 END:STANDARD END:VTIMEZONE BEGIN:VEVENT UID:0H44B_sN0PzTAnM2@H6D3JmW4G7v71tX DTSTART;TZID=Asia/Kolkata:20140127T183000 DTEND;TZID=Asia/Kolkata:20140127T193000 ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=DECLINED:mailto:u sers(a)ovirt.org CREATED:20140121T113648Z DESCRIPTION:\nShubhendu Ram Tripathi (shtripat(a)redhat.com) has cancelled t his meeting.\n\n************************************************\n\nMessag e/Description: Cancelled\n\n\n******************************************** ****\n\nBlue Jeans Network 2012\n DTSTAMP:20140127T054615Z LAST-MODIFIED:20140127T054615Z LOCATION: ORGANIZER:mailto:shtripat@redhat.com SEQUENCE:1 STATUS:CANCELLED SUMMARY:Feature Review: Gluster Volume Snapshots TRANSP:OPAQUE END:VEVENT END:VCALENDAR ------=_Part_794_1616979341.1390801575818--

1 0

[Users] Cancelled: Feature Review: Gluster Volume Snapshot
by Shubhendu Tripathi 26 Jan '14

26 Jan '14

------=_Part_8859059_641577150.1390801417195 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit The following meeting has been cancelled: Subject: Feature Review: Gluster Volume Snapshot Organizer: "Shubhendu Tripathi" <shtripat(a)redhat.com> Location: "Vijayanagara - BLR" <vijayanagara-blr(a)redhat.com> Resources: "Vijayanagara - BLR" <vijayanagara-blr(a)redhat.com> (Vijayanagara - BLR) Time: Monday, January 27, 2014, 6:30:00 PM - 7:30:00 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Invitees: arch(a)ovirt.org; users(a)ovirt.org; rnachimu(a)redhat.com; don(a)techcetera.com; otavio.ferranti(a)eldorado.org.br; lucas.lima(a)suinfra.ufrgs.br; pagupta(a)redhat.com; ehildesh(a)redhat.com; srevivo(a)redhat.com; Karli.Sjoberg(a)slu.se; pportant(a)redhat.com ... *~*~*~*~*~*~*~*~*~* Hi, The meeting is cancelled as the feature is being dropped from the current release because of other high priority features being taken up. Extremely sorry for the last moment cancellation. Will schedule the feature review later once we take up this feature. Thanks and Regards, Shubhendu Tripathi ------=_Part_8859059_641577150.1390801417195 Content-Type: text/calendar; charset=utf-8; method=CANCEL; name=meeting.ics Content-Transfer-Encoding: quoted-printable BEGIN:VCALENDAR PRODID:Zimbra-Calendar-Provider VERSION:2.0 METHOD:CANCEL BEGIN:VTIMEZONE TZID:Asia/Kolkata BEGIN:STANDARD DTSTART:16010101T000000 TZOFFSETTO:+0530 TZOFFSETFROM:+0530 TZNAME:IST END:STANDARD END:VTIMEZONE BEGIN:VEVENT UID:3440764a-ef38-4491-a023-f369dcfe9a18 SUMMARY:Cancelled: Feature Review: Gluster Volume Snapshot COMMENT:The following meeting has been cancelled: LOCATION:"Vijayanagara - BLR" <vijayanagara-blr(a)redhat.com> ATTENDEE;ROLE=3DREQ-PARTICIPANT;PARTSTAT=3DNEEDS-ACTION;RSVP=3DTRUE:mailto:= arch@ov irt.org ATTENDEE;ROLE=3DREQ-PARTICIPANT;PARTSTAT=3DNEEDS-ACTION;RSVP=3DTRUE:mailto:= users@o virt.org ATTENDEE;CN=3DVijayanagara - BLR;CUTYPE=3DRESOURCE;ROLE=3DNON-PARTICIPANT;P= ARTSTAT =3DACCEPTED;RSVP=3DTRUE:mailto:vijayanagara-blr@redhat.com ATTENDEE;CN=3DRamesh Nachimuthu;PARTSTAT=3DACCEPTED:mailto:rnachimu@redhat.= com ATTENDEE;CN=3DDon Harris;PARTSTAT=3DTENTATIVE:mailto:don@techcetera.com ATTENDEE;CN=3DOtavio Luiz Ferranti;PARTSTAT=3DDECLINED:mailto:otavio.ferran= ti@el dorado.org.br ATTENDEE;CN=3DLucas Lima;PARTSTAT=3DACCEPTED:mailto:lucas.lima@suinfra.ufrg= s.br ATTENDEE;CN=3DPankaj Gupta;PARTSTAT=3DACCEPTED:mailto:pagupta@redhat.com ATTENDEE;CN=3DEldan Hildesheim;PARTSTAT=3DTENTATIVE:mailto:ehildesh@redhat.= com ATTENDEE;CN=3DShai Revivo;PARTSTAT=3DTENTATIVE:mailto:srevivo@redhat.com ATTENDEE;CN=3D"Karli Sj=C3=B6berg";PARTSTAT=3DDECLINED:mailto:Karli.Sjoberg= @slu.se ATTENDEE;CN=3DPeter Portante;PARTSTAT=3DDECLINED:mailto:pportant@redhat.com ATTENDEE;CUTYPE=3DINDIVIDUAL;PARTSTAT=3DDECLINED:mailto:dbishop@controlscan= .com ATTENDEE;CN=3DRic Wheeler;PARTSTAT=3DTENTATIVE:mailto:rwheeler@redhat.com ATTENDEE;CN=3DPavithra;PARTSTAT=3DACCEPTED:mailto:psriniva@redhat.com ATTENDEE;CN=3DSatya Vempati;PARTSTAT=3DACCEPTED:mailto:svempati@tintri.com ATTENDEE;CN=3DEldad Marciano;PARTSTAT=3DACCEPTED:mailto:emarcian@redhat.com ORGANIZER;CN=3DShubhendu Tripathi:mailto:shtripat@redhat.com DTSTART;TZID=3D"Asia/Kolkata":20140127T183000 DTEND;TZID=3D"Asia/Kolkata":20140127T193000 STATUS:CANCELLED CLASS:PUBLIC X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY TRANSP:OPAQUE LAST-MODIFIED:20140127T054337Z DTSTAMP:20140127T054337Z SEQUENCE:1 DESCRIPTION:The following meeting has been cancelled:\n\nSubject: Feature R= e view: Gluster Volume Snapshot \nOrganizer: "Shubhendu Tripathi" <shtripat@= re dhat.com> \n\nLocation: "Vijayanagara - BLR" <vijayanagara-blr(a)redhat.com>= \ nResources: "Vijayanagara - BLR" <vijayanagara-blr(a)redhat.com> (Vijayanaga= ra - BLR) \nTime: Monday\, January 27\, 2014\, 6:30:00 PM - 7:30:00 PM GMT += 05 :30 Chennai\, Kolkata\, Mumbai\, New Delhi\n \nInvitees: arch(a)ovirt.org\; = us ers(a)ovirt.org\; rnachimu(a)redhat.com\; don(a)techcetera.com\; otavio.ferranti= @e ldorado.org.br\; lucas.lima(a)suinfra.ufrgs.br\; pagupta(a)redhat.com\; ehilde= sh @redhat.com\; srevivo(a)redhat.com\; Karli.Sjoberg(a)slu.se\; pportant(a)redhat.= co m ... \n\n\n*~*~*~*~*~*~*~*~*~*\n\nHi\,\n\nThe meeting is cancelled as the= f eature is being dropped from the current release because of other high pri= or ity features being taken up.\nExtremely sorry for the last moment cancella= ti on.\n\nWill schedule the feature review later once we take up this feature= .\ n\nThanks and Regards\,\nShubhendu Tripathi END:VEVENT END:VCALENDAR ------=_Part_8859059_641577150.1390801417195--

1 0

[Users] sanlock can't read empty 'ids' file
by Ted Miller 26 Jan '14

26 Jan '14

--------------000306010201050105090700 Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit After some indirect path to getting here, I believe that the cause of my problem is with sanlock. (See thread on 'Data Center Status on the GUI flips between "Non Responsive" and "Contending" ' for how I got here). My sanlock.log file is full of: 2014-01-26 21:36:19-0500 36474 [2905]: s4576 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0 2014-01-26 21:36:19-0500 36474 [26767]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512 2014-01-26 21:36:19-0500 36474 [26767]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids 2014-01-26 21:36:20-0500 36475 [2905]: s4576 add_lockspace fail result -90 2014-01-26 21:36:26-0500 36481 [2906]: s4577 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0 2014-01-26 21:36:26-0500 36481 [26809]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512 2014-01-26 21:36:26-0500 36481 [26809]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids 2014-01-26 21:36:27-0500 36482 [2906]: s4577 add_lockspace fail result -90 The only other thing is after rebooting, when it says: 2014-01-26 11:28:53-0500 29 [2897]: sanlock daemon started 2.8 host 09423427-2f70-4f68-9696-e41a93937ef0.office2a.l 2014-01-26 11:29:22-0500 57 [2905]: s1 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0 2014-01-26 11:29:22-0500 57 [3569]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c001000 result 0:0 match len 512 2014-01-26 11:29:22-0500 57 [3569]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids 2014-01-26 11:29:23-0500 58 [2905]: s1 add_lockspace fail result -90 2014-01-26 11:29:32-0500 67 [2905]: s2 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0 2014-01-26 11:29:32-0500 67 [3731]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512 2014-01-26 11:29:32-0500 67 [3731]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids 2014-01-26 11:29:33-0500 68 [2905]: s2 add_lockspace fail result -90 2014-01-26 11:29:42-0500 77 [2905]: s3 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0 2014-01-26 11:29:42-0500 77 [3806]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512 2014-01-26 11:29:42-0500 77 [3806]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids 2014-01-26 11:29:43-0500 78 [2905]: s3 add_lockspace fail result -90 Background: Setup is 2 hosts on Centos 6.5 with gluster storage, engine on separate Centos 6.5, all up to date. Originally set up running: Data Center: default Cluster: default Master storage domain: VM log had entries like: 2013-12-18 23:20:39-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -103:0 match res 2013-12-18 23:20:39-0500 4510 [3318]: s1 delta_renew read rv -103 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids 2013-12-18 23:20:39-0500 4510 [3318]: s1 renewal error -103 delta_length 9 last_success 4481 2013-12-18 23:20:40-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res 2013-12-18 23:20:40-0500 4510 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids 2013-12-18 23:20:40-0500 4510 [3318]: s1 renewal error -107 delta_length 0 last_success 4481 2013-12-18 23:20:40-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res 2013-12-18 23:20:40-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids 2013-12-18 23:20:40-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481 2013-12-18 23:20:41-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res 2013-12-18 23:20:41-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids 2013-12-18 23:20:41-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481 storage 'VM' had a Split-Brain. In trying to fix split-brain, file dom_md/ids was erased. I never revived the Data Center 'default' after that. log is full of: 2014-01-16 17:23:21-0500 62902 [29262]: open error -13 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids 2014-01-16 17:23:21-0500 62902 [29262]: s3 open_disk /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids error -13 2014-01-16 17:23:22-0500 62903 [3368]: s3 add_lockspace fail result -19 2014-01-16 17:23:31-0500 62912 [3368]: s4 lockspace 9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88:1:/rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids:0 Jan. 16: Created new: Data Center: mill; Cluster: one; Master storage domain: 'VM2' All was working. Logs looked like this: 2013-12-18 23:20:39-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -103:0 match res 2013-12-18 23:20:39-0500 4510 [3318]: s1 delta_renew read rv -103 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids 2013-12-18 23:20:39-0500 4510 [3318]: s1 renewal error -103 delta_length 9 last_success 4481 2013-12-18 23:20:40-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res 2013-12-18 23:20:40-0500 4510 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids 2013-12-18 23:20:40-0500 4510 [3318]: s1 renewal error -107 delta_length 0 last_success 4481 2013-12-18 23:20:40-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res 2013-12-18 23:20:40-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids 2013-12-18 23:20:40-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481 2013-12-18 23:20:41-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res 2013-12-18 23:20:41-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids 2013-12-18 23:20:41-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481 For a while, very little was logged at all, and this were working fine. Then, Jan 22 logs went from: 2014-01-22 00:44:56-0500 521397 [30933]: leader3 m 0 v 0 ss 0 nh 0 mh 0 oi 0 og 0 lv 0 2014-01-22 00:44:56-0500 521397 [30933]: leader4 sn rn ts 0 cs 0 2014-01-22 00:44:56-0500 521397 [30933]: s123 delta_renew verify_leader error -223 2014-01-22 00:44:56-0500 521397 [30933]: s123 renewal error -223 delta_length 0 last_success 521369 2014-01-22 00:44:57-0500 521398 [3362]: s123 kill 32599 sig 15 count 9 2014-01-22 00:44:57-0500 521398 [30933]: verify_leader 2 wrong magic 0 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids 2014-01-22 00:44:57-0500 521398 [30933]: leader1 delta_renew error -223 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c host_id 2 2014-01-22 00:44:57-0500 521398 [30933]: leader2 path /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids offset 0 2014-01-22 00:44:57-0500 521398 [30933]: leader3 m 0 v 0 ss 0 nh 0 mh 0 oi 0 og 0 lv 0 to: 2014-01-22 00:45:21-0500 521422 [17199]: 0322a407 aio collect 0 0x7f87ac0008c0:0x7f87ac0008d0:0x7f87ac101000 result 0:0 match len 512 2014-01-22 00:45:21-0500 521422 [17199]: read_sectors delta_leader offset 512 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids 2014-01-22 00:45:22-0500 521423 [3368]: s125 add_lockspace fail result -90 2014-01-22 00:45:25-0500 521426 [3368]: s126 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:2:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0 2014-01-22 00:45:25-0500 521426 [17243]: 0322a407 aio collect 0 0x7f87ac0008c0:0x7f87ac0008d0:0x7f87ac101000 result 0:0 match len 512 2014-01-22 00:45:25-0500 521426 [17243]: read_sectors delta_leader offset 512 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids 2014-01-22 00:45:26-0500 521427 [3368]: s126 add_lockspace fail result -90 2014-01-22 00:45:36-0500 521437 [3368]: s127 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:2:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0 and the Data Center has been down ever since. Best I can figure is that something is supposed to be in dom_md/ids, but that file is empty: ls /rhev/data-center/mnt/glusterSD/10.41.65.2\:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ -l total 1029 -rw-rw---- 1 vdsm kvm 0 Jan 22 00:44 ids -rw-rw---- 1 vdsm kvm 0 Jan 16 18:50 inbox -rw-rw---- 1 vdsm kvm 2097152 Jan 21 18:20 leases -rw-r--r-- 1 vdsm kvm 491 Jan 21 18:20 metadata -rw-rw---- 1 vdsm kvm 0 Jan 16 18:50 outbox Any hints as to how to put whatever is needed into 'ids', or reinitialize the sanlock system--or a better diagnosis and solution--gladly accepted. Ted Miller Elkhart, IN, USA --------------000306010201050105090700 Content-Type: text/html; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> </head> <body text="#000000" bgcolor="#FFFFFF"> After some indirect path to getting here, I believe that the cause of my problem is with sanlock.  (See thread on 'Data Center Status on the GUI flips between "Non Responsive" and "Contending" '  for how I got here). My sanlock.log file is full of: <pre>2014-01-26 21:36:19-0500 36474 [2905]: s4576 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0</pre> <pre>2014-01-26 21:36:19-0500 36474 [26767]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512</pre> <pre>2014-01-26 21:36:19-0500 36474 [26767]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre> <pre>2014-01-26 21:36:20-0500 36475 [2905]: s4576 add_lockspace fail result -90</pre> <pre>2014-01-26 21:36:26-0500 36481 [2906]: s4577 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0</pre> <pre>2014-01-26 21:36:26-0500 36481 [26809]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512</pre> <pre>2014-01-26 21:36:26-0500 36481 [26809]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre> <pre>2014-01-26 21:36:27-0500 36482 [2906]: s4577 add_lockspace fail result -90</pre> The only other thing is after rebooting, when it says: <pre>2014-01-26 11:28:53-0500 29 [2897]: sanlock daemon started 2.8 host 09423427-2f70-4f68-9696-e41a93937ef0.office2a.l</pre> <pre>2014-01-26 11:29:22-0500 57 [2905]: s1 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0</pre> <pre>2014-01-26 11:29:22-0500 57 [3569]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c001000 result 0:0 match len 512</pre> <pre>2014-01-26 11:29:22-0500 57 [3569]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre> <pre>2014-01-26 11:29:23-0500 58 [2905]: s1 add_lockspace fail result -90</pre> <pre>2014-01-26 11:29:32-0500 67 [2905]: s2 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0</pre> <pre>2014-01-26 11:29:32-0500 67 [3731]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512</pre> <pre>2014-01-26 11:29:32-0500 67 [3731]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre> <pre>2014-01-26 11:29:33-0500 68 [2905]: s2 add_lockspace fail result -90</pre> <pre>2014-01-26 11:29:42-0500 77 [2905]: s3 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0</pre> <pre>2014-01-26 11:29:42-0500 77 [3806]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512</pre> <pre>2014-01-26 11:29:42-0500 77 [3806]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre> <pre>2014-01-26 11:29:43-0500 78 [2905]: s3 add_lockspace fail result -90</pre> Background: Setup is 2 hosts on Centos 6.5 with gluster storage, engine on separate Centos 6.5, all up to date. Originally set up running: <blockquote>Data Center: default Cluster: default Master storage domain: VM </blockquote> log had entries like: <blockquote> <pre>2013-12-18 23:20:39-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -103:0 match res</pre> <pre>2013-12-18 23:20:39-0500 4510 [3318]: s1 delta_renew read rv -103 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre> <pre>2013-12-18 23:20:39-0500 4510 [3318]: s1 renewal error -103 delta_length 9 last_success 4481</pre> <pre>2013-12-18 23:20:40-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res</pre> <pre>2013-12-18 23:20:40-0500 4510 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre> <pre>2013-12-18 23:20:40-0500 4510 [3318]: s1 renewal error -107 delta_length 0 last_success 4481</pre> <pre>2013-12-18 23:20:40-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res</pre> <pre>2013-12-18 23:20:40-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre> <pre>2013-12-18 23:20:40-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481</pre> <pre>2013-12-18 23:20:41-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res</pre> <pre>2013-12-18 23:20:41-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre> <pre>2013-12-18 23:20:41-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481</pre> </blockquote> storage 'VM' had a Split-Brain.  In trying to fix split-brain, file dom_md/ids was erased.  I never revived the Data Center 'default' after that. <blockquote>log is full of: <pre>2014-01-16 17:23:21-0500 62902 [29262]: open error -13 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre> <pre>2014-01-16 17:23:21-0500 62902 [29262]: s3 open_disk /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids error -13</pre> <pre>2014-01-16 17:23:22-0500 62903 [3368]: s3 add_lockspace fail result -19</pre> <pre>2014-01-16 17:23:31-0500 62912 [3368]: s4 lockspace 9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88:1:/rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids:0</pre> </blockquote> Jan. 16: Created new:  Data Center: mill; Cluster: one;  Master storage domain: 'VM2'  All was working. <blockquote>Logs looked like this: <pre>2013-12-18 23:20:39-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -103:0 match res</pre> <pre>2013-12-18 23:20:39-0500 4510 [3318]: s1 delta_renew read rv -103 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre> <pre>2013-12-18 23:20:39-0500 4510 [3318]: s1 renewal error -103 delta_length 9 last_success 4481</pre> <pre>2013-12-18 23:20:40-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res</pre> <pre>2013-12-18 23:20:40-0500 4510 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre> <pre>2013-12-18 23:20:40-0500 4510 [3318]: s1 renewal error -107 delta_length 0 last_success 4481</pre> <pre>2013-12-18 23:20:40-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res</pre> <pre>2013-12-18 23:20:40-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre> <pre>2013-12-18 23:20:40-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481</pre> <pre>2013-12-18 23:20:41-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res</pre> <pre>2013-12-18 23:20:41-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre> <pre>2013-12-18 23:20:41-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481</pre> </blockquote> For a while, very little was logged at all, and this were working fine.  Then, Jan 22 logs went from: <blockquote> <pre>2014-01-22 00:44:56-0500 521397 [30933]: leader3 m 0 v 0 ss 0 nh 0 mh 0 oi 0 og 0 lv 0</pre> <pre>2014-01-22 00:44:56-0500 521397 [30933]: leader4 sn  rn  ts 0 cs 0</pre> <pre>2014-01-22 00:44:56-0500 521397 [30933]: s123 delta_renew verify_leader error -223</pre> <pre>2014-01-22 00:44:56-0500 521397 [30933]: s123 renewal error -223 delta_length 0 last_success 521369</pre> <pre>2014-01-22 00:44:57-0500 521398 [3362]: s123 kill 32599 sig 15 count 9</pre> <pre>2014-01-22 00:44:57-0500 521398 [30933]: verify_leader 2 wrong magic 0 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre> <pre>2014-01-22 00:44:57-0500 521398 [30933]: leader1 delta_renew error -223 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c host_id 2</pre> <pre>2014-01-22 00:44:57-0500 521398 [30933]: leader2 path /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids offset 0</pre> <pre>2014-01-22 00:44:57-0500 521398 [30933]: leader3 m 0 v 0 ss 0 nh 0 mh 0 oi 0 og 0 lv 0</pre> </blockquote> to: <blockquote> <pre>2014-01-22 00:45:21-0500 521422 [17199]: 0322a407 aio collect 0 0x7f87ac0008c0:0x7f87ac0008d0:0x7f87ac101000 result 0:0 match len 512</pre> <pre>2014-01-22 00:45:21-0500 521422 [17199]: read_sectors delta_leader offset 512 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre> <pre>2014-01-22 00:45:22-0500 521423 [3368]: s125 add_lockspace fail result -90</pre> <pre>2014-01-22 00:45:25-0500 521426 [3368]: s126 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:2:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0</pre> <pre>2014-01-22 00:45:25-0500 521426 [17243]: 0322a407 aio collect 0 0x7f87ac0008c0:0x7f87ac0008d0:0x7f87ac101000 result 0:0 match len 512</pre> <pre>2014-01-22 00:45:25-0500 521426 [17243]: read_sectors delta_leader offset 512 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre> <pre>2014-01-22 00:45:26-0500 521427 [3368]: s126 add_lockspace fail result -90</pre> <pre>2014-01-22 00:45:36-0500 521437 [3368]: s127 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:2:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0</pre> </blockquote> and the Data Center has been down ever since. Best I can figure is that something is supposed to be in dom_md/ids, but that file is empty: <pre>ls /rhev/data-center/mnt/glusterSD/10.41.65.2\:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ -l</pre> <pre>total 1029</pre> <pre>-rw-rw---- 1 vdsm kvm       0 Jan 22 00:44 ids</pre> <pre>-rw-rw---- 1 vdsm kvm       0 Jan 16 18:50 inbox</pre> <pre>-rw-rw---- 1 vdsm kvm 2097152 Jan 21 18:20 leases</pre> <pre>-rw-r--r-- 1 vdsm kvm     491 Jan 21 18:20 metadata</pre> <pre>-rw-rw---- 1 vdsm kvm       0 Jan 16 18:50 outbox </pre> Any hints as to how to put whatever is needed into 'ids', or reinitialize the sanlock system--or a better diagnosis and solution--gladly accepted. Ted Miller Elkhart, IN, USA </body> </html> --------------000306010201050105090700--

1 0

[Users] oVirt 3.4.0 issues with storage
by Jon Archer 26 Jan '14

26 Jan '14

Hi all, I upgraded to 3.4 beta this evening and seem to have developed a problem with the storage domains. I seem unable to bring them online getting the error in the console "Failed to activate Storage Domain". Also alongside this error is a warning, "Invalid status on Data Center local_datacenter. Setting status to Non Responsive" Anyone seen this? Or any ideas? Thanks Jon

4 11

[Users] mavericks as a guest vm?
by Mark 26 Jan '14

26 Jan '14

I have older Xserve hardware that is 64 bit but the bios is 32 bit. Fusion runs mavericks with no problem. The vm runs in 64 bit mode. I'd like to run mavericks as vm under ovirt on this hardware. Google searches I've done keep pointing to vmware. Any pointers?

3 2

[Users] Configure VNC-Password timeout
by Sven Kieske 26 Jan '14

26 Jan '14

Hi, the default timeout for the vnc passwords is 2 Minutes. I'd like to reconfigure this. Is there an engine-config value or is this hardcoded? How can this be changed? -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen

4 5

[Users] issue with conversion of ESXi 5 centos VM to fedora19 ovirt host
by Madhav V Diwan 26 Jan '14

26 Jan '14

Hello everybody! I seem to be having a problem similar to the posting from " importing VM from ESXI " posted by emitor(a)gmail.com on 25.09.2013 21:10 UTC " I'm not being able to import a VM from ESXI:" In my case i have a bit more of a trace on the permission issue but not much more > virt-v2v: Transferring storage volume razDC_razDC: 107374182400 bytes > libguestfs: trace: set_verbose true > libguestfs: trace: set_verbose = 0 > libguestfs: create: flags = 0, handle = 0x4725420, program = perl > libguestfs: trace: set_attach_method "appliance" > libguestfs: trace: set_backend "appliance" > libguestfs: trace: set_backend = 0 > libguestfs: trace: set_attach_method = 0 > libguestfs: trace: add_drive "/tmp/a7SyYhkkeX/8eff2927-3bff-4b15-bdd0-8c4e0f40652d/v2v.z_Lv0fTf/81388ffb-6aed-4ae4-bed8-6b1999e78907/2cc85d68-d343-4961-9a28-58e4c695d78f" "format:raw" "iface:ide" "name:sda" > libguestfs: trace: add_drive = -1 (error) > /tmp/a7SyYhkkeX/8eff2927-3bff-4b15-bdd0-8c4e0f40652d/v2v.z_Lv0fTf/81388ffb-6aed-4ae4-bed8-6b1999e78907/2cc85d68-d343-4961-9a28-58e4c695d78f: Permission denied at /usr/lib64/perl5/vendor_perl/Sys/Guestfs.pm line 670. > libguestfs: trace: close > libguestfs: closing guestfs handle 0x4725420 (state 0) Has this issue been solved ? if so how as i do not see the resolution on the archives. Else would you all help me resolve it please? It is very annoying to have a hundred gig disk copied over to your NFS export and then deleted because qemu or libguestfs has a permissions issue in opening it. this is an issue with Ovirt Version 3.3.2-1.fc19 virt-v2v version is virt-v2v --version 0.9.0 and libguestfs version and qemu version are : libguestfs-1.22.7-4.fc19.x86_64 QEMU emulator version 1.4.2

9 44

[Users] Data Center stuck between "Non Responsive" and "Contending"
by Ted Miller 26 Jan '14

26 Jan '14

--------------090008040302090307060003 Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit My Data Center is down, and won't come back up. Data Center Status on the GUI flips between "Non Responsive" and "Contending" Also noted: Host sometimes seen flipping between "Low" and "Contending" in SPM column. Storage VM2 "Data (Master)" is in "Cross Data-Center Status" = Unknown VM2 is "up" under "Volumes" tab Created another volume for VM storage. It shows up in "volumes" tab, but when I try to add "New Domain" in storage tab, says that "There are No Data Centers to which the Storage Domain can be attached" Setup: 2 hosts w/ glusterfs storage 1 engine all 3 computers Centos 6.5, just updated ovirt-engine 3.3.0.1-1.el6 ovirt-engine-lib 3.3.2-1.el6 ovirt-host-deploy.noarch 1.1.3-1.el6 glusterfs.x86_64 3.4.2-1.el6 This loop seems to repeat in the ovirt-engine log (grep of log showing only DefaultQuartzScheduler_Worker-79 thread: 2014-01-26 14:44:58,416 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-79) Irs placed on server 9a591103-83be-4ca9-b207-06929223b541 failed. Proceed Failover 2014-01-26 14:44:58,511 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-79) hostFromVds::selectedVds - office4a, spmStatus Free, storage pool mill 2014-01-26 14:44:58,550 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-79) SpmStatus on vds 127ed939-34af-41a8-87a0-e2f6174b1877: Free 2014-01-26 14:44:58,571 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-79) starting spm on vds office4a, storage pool mill, prevId 2, LVER 15 2014-01-26 14:44:58,579 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-79) START, SpmStartVDSCommand(HostName = office4a, HostId = 127ed939-34af-41a8-87a0-e2f6174b1877, storagePoolId = 536a864d-83aa-473a-a675-e38aafdd9071, prevId=2, prevLVER=15, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 74c38eb7 2014-01-26 14:44:58,617 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-79) spmStart polling started: taskId = e8986753-fc80-4b11-a11d-6d3470b1728c 2014-01-26 14:45:00,662 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetTaskStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) Failed in HSMGetTaskStatusVDS method 2014-01-26 14:45:00,664 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetTaskStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) Error code AcquireHostIdFailure and error message VDSGenericException: VDSErrorException: Failed to HSMGetTaskStatusVDS, error = Cannot acquire host id 2014-01-26 14:45:00,665 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-79) spmStart polling ended: taskId = e8986753-fc80-4b11-a11d-6d3470b1728c task status = finished 2014-01-26 14:45:00,666 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-79) Start SPM Task failed - result: cleanSuccess, message: VDSGenericException: VDSErrorException: Failed to HSMGetTaskStatusVDS, error = Cannot acquire host id 2014-01-26 14:45:00,695 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-79) spmStart polling ended, spm status: Free 2014-01-26 14:45:00,702 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-79) START, HSMClearTaskVDSCommand(HostName = office4a, HostId = 127ed939-34af-41a8-87a0-e2f6174b1877, taskId=e8986753-fc80-4b11-a11d-6d3470b1728c), log id: 336ec5a6 2014-01-26 14:45:00,722 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-79) FINISH, HSMClearTaskVDSCommand, log id: 336ec5a6 2014-01-26 14:45:00,724 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-79) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@13652652, log id: 74c38eb7 2014-01-26 14:45:00,733 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-79) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 536a864d-83aa-473a-a675-e38aafdd9071 Type: StoragePool 2014-01-26 14:45:00,778 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-79) IrsBroker::Failed::GetStoragePoolInfoVDS due to: IrsSpmStartFailedException: IRSGenericException: IRSErrorException: SpmStart failed Ted Miller Elkhart, IN, USA --------------090008040302090307060003 Content-Type: text/html; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> </head> <body text="#000000" bgcolor="#FFFFFF"> My Data Center is down, and won't come back up. Data Center Status on the GUI flips between "Non Responsive" and "Contending" Also noted: Host sometimes seen flipping between "Low" and "Contending" in SPM column. Storage VM2 "Data (Master)" is in "Cross Data-Center Status" = Unknown VM2 is "up" under "Volumes" tab Created another volume for VM storage.  It shows up in "volumes" tab, but when I try to add "New Domain" in storage tab, says that "There are No Data Centers to which the Storage Domain can be attached" Setup: 2 hosts w/ glusterfs storage 1 engine all 3 computers Centos 6.5, just updated ovirt-engine                       3.3.0.1-1.el6 ovirt-engine-lib                 3.3.2-1.el6 ovirt-host-deploy.noarch  1.1.3-1.el6 glusterfs.x86_64               3.4.2-1.el6 This loop seems to repeat in the ovirt-engine log (grep of log showing only DefaultQuartzScheduler_Worker-79 thread: 2014-01-26 14:44:58,416 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-79) Irs placed on server 9a591103-83be-4ca9-b207-06929223b541 failed. Proceed Failover 2014-01-26 14:44:58,511 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-79) hostFromVds::selectedVds - office4a, spmStatus Free, storage pool mill 2014-01-26 14:44:58,550 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-79) SpmStatus on vds 127ed939-34af-41a8-87a0-e2f6174b1877: Free 2014-01-26 14:44:58,571 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-79) starting spm on vds office4a, storage pool mill, prevId 2, LVER 15 2014-01-26 14:44:58,579 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-79) START, SpmStartVDSCommand(HostName = office4a, HostId = 127ed939-34af-41a8-87a0-e2f6174b1877, storagePoolId = 536a864d-83aa-473a-a675-e38aafdd9071, prevId=2, prevLVER=15, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 74c38eb7 2014-01-26 14:44:58,617 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-79) spmStart polling started: taskId = e8986753-fc80-4b11-a11d-6d3470b1728c 2014-01-26 14:45:00,662 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetTaskStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) Failed in HSMGetTaskStatusVDS method 2014-01-26 14:45:00,664 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetTaskStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) Error code AcquireHostIdFailure and error message VDSGenericException: VDSErrorException: Failed to HSMGetTaskStatusVDS, error = Cannot acquire host id 2014-01-26 14:45:00,665 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-79) spmStart polling ended: taskId = e8986753-fc80-4b11-a11d-6d3470b1728c task status = finished 2014-01-26 14:45:00,666 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-79) Start SPM Task failed - result: cleanSuccess, message: VDSGenericException: VDSErrorException: Failed to HSMGetTaskStatusVDS, error = Cannot acquire host id 2014-01-26 14:45:00,695 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-79) spmStart polling ended, spm status: Free 2014-01-26 14:45:00,702 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-79) START, HSMClearTaskVDSCommand(HostName = office4a, HostId = 127ed939-34af-41a8-87a0-e2f6174b1877, taskId=e8986753-fc80-4b11-a11d-6d3470b1728c), log id: 336ec5a6 2014-01-26 14:45:00,722 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-79) FINISH, HSMClearTaskVDSCommand, log id: 336ec5a6 2014-01-26 14:45:00,724 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-79) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@13652652, log id: 74c38eb7 2014-01-26 14:45:00,733 INFO  [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-79) Running command: SetStoragePoolStatusCommand internal: true. Entities affected :  ID: 536a864d-83aa-473a-a675-e38aafdd9071 Type: StoragePool 2014-01-26 14:45:00,778 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-79) IrsBroker::Failed::GetStoragePoolInfoVDS due to: IrsSpmStartFailedException: IRSGenericException: IRSErrorException: SpmStart failed Ted Miller Elkhart, IN, USA </body> </html> --------------090008040302090307060003--

2 1