[Users] Power saving cluster policy (3.4 test day)
by Lior Vernia
Hello,
Just wanted to drop a line here about this feature I tested last week,
which I thought was cool. I only tested it with two hosts and a single
VM, using one of the hosts as a reserve, but it was quite satisfying to
see the hosts being powered off and on as the VM was being run and stopped.
This could be interesting to look at for those of you running a
moderately-sized deployment, if often the load on your hosts is low.
Yours, Lior.
10 years, 3 months
[Users] sanlock leases on VM disks
by José Luis Sanz Boixader
I have an oVirt testing setup with 3 hosts running for a few weeks:
CentOS 6.4, oVirt 3.3.1, VDSM 4.13.0, iSCSI based storage domain.
I have just realized that sanlock has no leases on VM disks, so nothing
prevents vdsm/libvirt from starting a VM on two different hosts,
corrupting disk data. I know that something has to go wrong on oVirt
engine to do it, but I've manually forced some errors ("Setting Host
state to Non-Operational", "VM xxxx is not responding") for a "Highly
available" VM and oVirt engine started that VM on another host. oVirt
engine was not aware, but the VM was running on two hosts.
I think this is a job for libvirt/sanlock/wdmd, but libvirt is not
receiving "lease" tags for disks when creating domains. I think it should.
What's left in my config? What am I doing wrong?
Thanks
10 years, 3 months
[Users] multiple storage domains
by Michal Skrivanek
Hey,
I was trying to play with multiple storage domain during the 3.4 test day and even though it took me ages to set storage up correctly, the feature itself seems to be working just fine. Much easier to select what kind of storage I want to use…just Shared vs Local storage pool. VM could use volumes from different domains and I didn't have to worry about the right type (other than local vs shared, obviously:)
Thanks,
michal
10 years, 3 months
[Users] CANCELED: Feature Review: Gluster Volume Snapshots
by Blue Jeans Network
------=_Part_794_1616979341.1390801575818
Content-Type: multipart/related;
boundary="----=_Part_795_166231402.1390801575818"
------=_Part_795_166231402.1390801575818
Content-Type: multipart/alternative;
boundary="----=_Part_796_2122432314.1390801575818"
------=_Part_796_2122432314.1390801575818
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Shubhendu Ram Tripathi (shtripat(a)redhat.com) has cancelled the following meeting:
Meeting Title: Feature Review: Gluster Volume Snapshots
Meeting Time: Monday January 27, 2014 6:30 p.m. IST / 1 hr
************************************************
Message/Description: Cancelled
************************************************
Blue Jeans Network 2012
------=_Part_796_2122432314.1390801575818
Content-Type: text/html;charset="utf-8"
Content-Transfer-Encoding: 7bit
<table width="100%">
<tr bgcolor="#d8d8d8">
<td align="center">
<table width="600px" bgcolor="#FFFFFF" style="margin-top:20px;padding:0px 50px 10px 50px;margin-bottom:20px;text-align:left;">
<tr>
<td style="padding-right:10px;padding-bottom:20px;">
<a href="http://bluejeans.com">
<img src="https://static.bluejeans.com/z1/static/images/bjn/BlueJeans_Logo.png" alt="Blue Jeans" style="border:none;">
</a>
</td>
</tr>
<tr>
<td style="font-size: 20px;color: #333333; padding-bottom:30px;">
<a href="mailto:shtripat@redhat.com">Shubhendu Ram Tripathi</a> has cancelled the following meeting:
</td>
</tr>
<tr>
<td style="padding-top:10px; font-size: 16px;">
<span style="color:#666666; padding-right: 20px;">Meeting Title: </span>Feature Review: Gluster Volume Snapshots
</td>
</tr>
<tr>
<td style="padding-top:10px; padding-bottom: 10px; font-size: 16px;">
<span style="color:#666666; padding-right: 15px;">Meeting Time:</span>
Monday January 27, 2014 • 6:30 p.m. IST / 1 hr
</td>
</tr>
<tr>
<td>
<hr size="1" noshade="noshade" style="line-height:1px;color:#CCCCCC;">
</td>
</tr>
<tr>
<td style="style="padding-top:10px;"">
<b><span style="color:#333333">Message/Description:</span></b><br>
Cancelled
</td>
</tr>
<tr>
<td style="line-height:1px;color:#CCCCCC;">
<hr size="1" noshade="noshade" style="line-height:1px;color:#CCCCCC;border-color:#CCCCCC;">
</td>
</tr>
<tr>
<td style="font-size:11px;text-align:left">
<p align="center">©Blue Jeans Network 2013</p>
</td>
</tr>
</table>
</td>
</tr>
</table>
------=_Part_796_2122432314.1390801575818
Content-Type: text/calendar; method=REQUEST; charset="utf-8"
Content-Transfer-Encoding: 7bit
BEGIN:VCALENDAR
VERSION:2.0
METHOD:CANCEL
PRODID:-//PYVOBJECT//NONSGML Version 1//EN
BEGIN:VTIMEZONE
TZID:Asia/Kolkata
TZURL:http://tzurl.org/zoneinfo-outlook/Asia/Kolkata
X-LIC-LOCATION:Asia/Kolkata
BEGIN:STANDARD
TZOFFSETFROM:+0530
TZOFFSETTO:+0530
TZNAME:IST
DTSTART:19700101T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
UID:0H44B_sN0PzTAnM2@H6D3JmW4G7v71tX
DTSTART;TZID=Asia/Kolkata:20140127T183000
DTEND;TZID=Asia/Kolkata:20140127T193000
ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=DECLINED:mailto:u
sers(a)ovirt.org
CREATED:20140121T113648Z
DESCRIPTION:\nShubhendu Ram Tripathi (shtripat(a)redhat.com) has cancelled t
his meeting.\n\n************************************************\n\nMessag
e/Description: Cancelled\n\n\n********************************************
****\n\nBlue Jeans Network 2012\n
DTSTAMP:20140127T054615Z
LAST-MODIFIED:20140127T054615Z
LOCATION:
ORGANIZER:mailto:shtripat@redhat.com
SEQUENCE:1
STATUS:CANCELLED
SUMMARY:Feature Review: Gluster Volume Snapshots
TRANSP:OPAQUE
END:VEVENT
END:VCALENDAR
------=_Part_796_2122432314.1390801575818--
------=_Part_795_166231402.1390801575818--
------=_Part_794_1616979341.1390801575818
Content-Type: application/ics; name=invite.ics
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename=invite.ics
BEGIN:VCALENDAR
VERSION:2.0
METHOD:CANCEL
PRODID:-//PYVOBJECT//NONSGML Version 1//EN
BEGIN:VTIMEZONE
TZID:Asia/Kolkata
TZURL:http://tzurl.org/zoneinfo-outlook/Asia/Kolkata
X-LIC-LOCATION:Asia/Kolkata
BEGIN:STANDARD
TZOFFSETFROM:+0530
TZOFFSETTO:+0530
TZNAME:IST
DTSTART:19700101T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
UID:0H44B_sN0PzTAnM2@H6D3JmW4G7v71tX
DTSTART;TZID=Asia/Kolkata:20140127T183000
DTEND;TZID=Asia/Kolkata:20140127T193000
ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=DECLINED:mailto:u
sers(a)ovirt.org
CREATED:20140121T113648Z
DESCRIPTION:\nShubhendu Ram Tripathi (shtripat(a)redhat.com) has cancelled t
his meeting.\n\n************************************************\n\nMessag
e/Description: Cancelled\n\n\n********************************************
****\n\nBlue Jeans Network 2012\n
DTSTAMP:20140127T054615Z
LAST-MODIFIED:20140127T054615Z
LOCATION:
ORGANIZER:mailto:shtripat@redhat.com
SEQUENCE:1
STATUS:CANCELLED
SUMMARY:Feature Review: Gluster Volume Snapshots
TRANSP:OPAQUE
END:VEVENT
END:VCALENDAR
------=_Part_794_1616979341.1390801575818--
10 years, 3 months
[Users] Cancelled: Feature Review: Gluster Volume Snapshot
by Shubhendu Tripathi
------=_Part_8859059_641577150.1390801417195
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
The following meeting has been cancelled:
Subject: Feature Review: Gluster Volume Snapshot
Organizer: "Shubhendu Tripathi" <shtripat(a)redhat.com>
Location: "Vijayanagara - BLR" <vijayanagara-blr(a)redhat.com>
Resources: "Vijayanagara - BLR" <vijayanagara-blr(a)redhat.com> (Vijayanagara - BLR)
Time: Monday, January 27, 2014, 6:30:00 PM - 7:30:00 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi
Invitees: arch(a)ovirt.org; users(a)ovirt.org; rnachimu(a)redhat.com; don(a)techcetera.com; otavio.ferranti(a)eldorado.org.br; lucas.lima(a)suinfra.ufrgs.br; pagupta(a)redhat.com; ehildesh(a)redhat.com; srevivo(a)redhat.com; Karli.Sjoberg(a)slu.se; pportant(a)redhat.com ...
*~*~*~*~*~*~*~*~*~*
Hi,
The meeting is cancelled as the feature is being dropped from the current release because of other high priority features being taken up.
Extremely sorry for the last moment cancellation.
Will schedule the feature review later once we take up this feature.
Thanks and Regards,
Shubhendu Tripathi
------=_Part_8859059_641577150.1390801417195
Content-Type: text/calendar; charset=utf-8; method=CANCEL; name=meeting.ics
Content-Transfer-Encoding: quoted-printable
BEGIN:VCALENDAR
PRODID:Zimbra-Calendar-Provider
VERSION:2.0
METHOD:CANCEL
BEGIN:VTIMEZONE
TZID:Asia/Kolkata
BEGIN:STANDARD
DTSTART:16010101T000000
TZOFFSETTO:+0530
TZOFFSETFROM:+0530
TZNAME:IST
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
UID:3440764a-ef38-4491-a023-f369dcfe9a18
SUMMARY:Cancelled: Feature Review: Gluster Volume Snapshot
COMMENT:The following meeting has been cancelled:
LOCATION:"Vijayanagara - BLR" <vijayanagara-blr(a)redhat.com>
ATTENDEE;ROLE=3DREQ-PARTICIPANT;PARTSTAT=3DNEEDS-ACTION;RSVP=3DTRUE:mailto:=
arch@ov
irt.org
ATTENDEE;ROLE=3DREQ-PARTICIPANT;PARTSTAT=3DNEEDS-ACTION;RSVP=3DTRUE:mailto:=
users@o
virt.org
ATTENDEE;CN=3DVijayanagara - BLR;CUTYPE=3DRESOURCE;ROLE=3DNON-PARTICIPANT;P=
ARTSTAT
=3DACCEPTED;RSVP=3DTRUE:mailto:vijayanagara-blr@redhat.com
ATTENDEE;CN=3DRamesh Nachimuthu;PARTSTAT=3DACCEPTED:mailto:rnachimu@redhat.=
com
ATTENDEE;CN=3DDon Harris;PARTSTAT=3DTENTATIVE:mailto:don@techcetera.com
ATTENDEE;CN=3DOtavio Luiz Ferranti;PARTSTAT=3DDECLINED:mailto:otavio.ferran=
ti@el
dorado.org.br
ATTENDEE;CN=3DLucas Lima;PARTSTAT=3DACCEPTED:mailto:lucas.lima@suinfra.ufrg=
s.br
ATTENDEE;CN=3DPankaj Gupta;PARTSTAT=3DACCEPTED:mailto:pagupta@redhat.com
ATTENDEE;CN=3DEldan Hildesheim;PARTSTAT=3DTENTATIVE:mailto:ehildesh@redhat.=
com
ATTENDEE;CN=3DShai Revivo;PARTSTAT=3DTENTATIVE:mailto:srevivo@redhat.com
ATTENDEE;CN=3D"Karli Sj=C3=B6berg";PARTSTAT=3DDECLINED:mailto:Karli.Sjoberg=
@slu.se
ATTENDEE;CN=3DPeter Portante;PARTSTAT=3DDECLINED:mailto:pportant@redhat.com
ATTENDEE;CUTYPE=3DINDIVIDUAL;PARTSTAT=3DDECLINED:mailto:dbishop@controlscan=
.com
ATTENDEE;CN=3DRic Wheeler;PARTSTAT=3DTENTATIVE:mailto:rwheeler@redhat.com
ATTENDEE;CN=3DPavithra;PARTSTAT=3DACCEPTED:mailto:psriniva@redhat.com
ATTENDEE;CN=3DSatya Vempati;PARTSTAT=3DACCEPTED:mailto:svempati@tintri.com
ATTENDEE;CN=3DEldad Marciano;PARTSTAT=3DACCEPTED:mailto:emarcian@redhat.com
ORGANIZER;CN=3DShubhendu Tripathi:mailto:shtripat@redhat.com
DTSTART;TZID=3D"Asia/Kolkata":20140127T183000
DTEND;TZID=3D"Asia/Kolkata":20140127T193000
STATUS:CANCELLED
CLASS:PUBLIC
X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY
TRANSP:OPAQUE
LAST-MODIFIED:20140127T054337Z
DTSTAMP:20140127T054337Z
SEQUENCE:1
DESCRIPTION:The following meeting has been cancelled:\n\nSubject: Feature R=
e
view: Gluster Volume Snapshot \nOrganizer: "Shubhendu Tripathi" <shtripat@=
re
dhat.com> \n\nLocation: "Vijayanagara - BLR" <vijayanagara-blr(a)redhat.com>=
\
nResources: "Vijayanagara - BLR" <vijayanagara-blr(a)redhat.com> (Vijayanaga=
ra
- BLR) \nTime: Monday\, January 27\, 2014\, 6:30:00 PM - 7:30:00 PM GMT +=
05
:30 Chennai\, Kolkata\, Mumbai\, New Delhi\n \nInvitees: arch(a)ovirt.org\; =
us
ers(a)ovirt.org\; rnachimu(a)redhat.com\; don(a)techcetera.com\; otavio.ferranti=
@e
ldorado.org.br\; lucas.lima(a)suinfra.ufrgs.br\; pagupta(a)redhat.com\; ehilde=
sh
@redhat.com\; srevivo(a)redhat.com\; Karli.Sjoberg(a)slu.se\; pportant(a)redhat.=
co
m ... \n\n\n*~*~*~*~*~*~*~*~*~*\n\nHi\,\n\nThe meeting is cancelled as the=
f
eature is being dropped from the current release because of other high pri=
or
ity features being taken up.\nExtremely sorry for the last moment cancella=
ti
on.\n\nWill schedule the feature review later once we take up this feature=
.\
n\nThanks and Regards\,\nShubhendu Tripathi
END:VEVENT
END:VCALENDAR
------=_Part_8859059_641577150.1390801417195--
10 years, 3 months
[Users] sanlock can't read empty 'ids' file
by Ted Miller
--------------000306010201050105090700
Content-Type: text/plain; charset="ISO-8859-1"; format=flowed
Content-Transfer-Encoding: 7bit
After some indirect path to getting here, I believe that the cause of my
problem is with sanlock. (See thread on 'Data Center Status on the GUI flips
between "Non Responsive" and
"Contending" ' for how I got here).
My sanlock.log file is full of:
2014-01-26 21:36:19-0500 36474 [2905]: s4576 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0
2014-01-26 21:36:19-0500 36474 [26767]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512
2014-01-26 21:36:19-0500 36474 [26767]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids
2014-01-26 21:36:20-0500 36475 [2905]: s4576 add_lockspace fail result -90
2014-01-26 21:36:26-0500 36481 [2906]: s4577 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0
2014-01-26 21:36:26-0500 36481 [26809]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512
2014-01-26 21:36:26-0500 36481 [26809]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids
2014-01-26 21:36:27-0500 36482 [2906]: s4577 add_lockspace fail result -90
The only other thing is after rebooting, when it says:
2014-01-26 11:28:53-0500 29 [2897]: sanlock daemon started 2.8 host 09423427-2f70-4f68-9696-e41a93937ef0.office2a.l
2014-01-26 11:29:22-0500 57 [2905]: s1 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0
2014-01-26 11:29:22-0500 57 [3569]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c001000 result 0:0 match len 512
2014-01-26 11:29:22-0500 57 [3569]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids
2014-01-26 11:29:23-0500 58 [2905]: s1 add_lockspace fail result -90
2014-01-26 11:29:32-0500 67 [2905]: s2 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0
2014-01-26 11:29:32-0500 67 [3731]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512
2014-01-26 11:29:32-0500 67 [3731]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids
2014-01-26 11:29:33-0500 68 [2905]: s2 add_lockspace fail result -90
2014-01-26 11:29:42-0500 77 [2905]: s3 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0
2014-01-26 11:29:42-0500 77 [3806]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512
2014-01-26 11:29:42-0500 77 [3806]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids
2014-01-26 11:29:43-0500 78 [2905]: s3 add_lockspace fail result -90
Background:
Setup is 2 hosts on Centos 6.5 with gluster storage, engine on separate
Centos 6.5, all up to date.
Originally set up running:
Data Center: default
Cluster: default
Master storage domain: VM
log had entries like:
2013-12-18 23:20:39-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -103:0 match res
2013-12-18 23:20:39-0500 4510 [3318]: s1 delta_renew read rv -103 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids
2013-12-18 23:20:39-0500 4510 [3318]: s1 renewal error -103 delta_length 9 last_success 4481
2013-12-18 23:20:40-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res
2013-12-18 23:20:40-0500 4510 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids
2013-12-18 23:20:40-0500 4510 [3318]: s1 renewal error -107 delta_length 0 last_success 4481
2013-12-18 23:20:40-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res
2013-12-18 23:20:40-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids
2013-12-18 23:20:40-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481
2013-12-18 23:20:41-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res
2013-12-18 23:20:41-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids
2013-12-18 23:20:41-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481
storage 'VM' had a Split-Brain. In trying to fix split-brain, file
dom_md/ids was erased. I never revived the Data Center 'default' after that.
log is full of:
2014-01-16 17:23:21-0500 62902 [29262]: open error -13 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids
2014-01-16 17:23:21-0500 62902 [29262]: s3 open_disk /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids error -13
2014-01-16 17:23:22-0500 62903 [3368]: s3 add_lockspace fail result -19
2014-01-16 17:23:31-0500 62912 [3368]: s4 lockspace 9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88:1:/rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids:0
Jan. 16: Created new: Data Center: mill; Cluster: one; Master storage
domain: 'VM2' All was working.
Logs looked like this:
2013-12-18 23:20:39-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -103:0 match res
2013-12-18 23:20:39-0500 4510 [3318]: s1 delta_renew read rv -103 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids
2013-12-18 23:20:39-0500 4510 [3318]: s1 renewal error -103 delta_length 9 last_success 4481
2013-12-18 23:20:40-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res
2013-12-18 23:20:40-0500 4510 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids
2013-12-18 23:20:40-0500 4510 [3318]: s1 renewal error -107 delta_length 0 last_success 4481
2013-12-18 23:20:40-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res
2013-12-18 23:20:40-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids
2013-12-18 23:20:40-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481
2013-12-18 23:20:41-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res
2013-12-18 23:20:41-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids
2013-12-18 23:20:41-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481
For a while, very little was logged at all, and this were working fine. Then,
Jan 22 logs went from:
2014-01-22 00:44:56-0500 521397 [30933]: leader3 m 0 v 0 ss 0 nh 0 mh 0 oi 0 og 0 lv 0
2014-01-22 00:44:56-0500 521397 [30933]: leader4 sn rn ts 0 cs 0
2014-01-22 00:44:56-0500 521397 [30933]: s123 delta_renew verify_leader error -223
2014-01-22 00:44:56-0500 521397 [30933]: s123 renewal error -223 delta_length 0 last_success 521369
2014-01-22 00:44:57-0500 521398 [3362]: s123 kill 32599 sig 15 count 9
2014-01-22 00:44:57-0500 521398 [30933]: verify_leader 2 wrong magic 0 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids
2014-01-22 00:44:57-0500 521398 [30933]: leader1 delta_renew error -223 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c host_id 2
2014-01-22 00:44:57-0500 521398 [30933]: leader2 path /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids offset 0
2014-01-22 00:44:57-0500 521398 [30933]: leader3 m 0 v 0 ss 0 nh 0 mh 0 oi 0 og 0 lv 0
to:
2014-01-22 00:45:21-0500 521422 [17199]: 0322a407 aio collect 0 0x7f87ac0008c0:0x7f87ac0008d0:0x7f87ac101000 result 0:0 match len 512
2014-01-22 00:45:21-0500 521422 [17199]: read_sectors delta_leader offset 512 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids
2014-01-22 00:45:22-0500 521423 [3368]: s125 add_lockspace fail result -90
2014-01-22 00:45:25-0500 521426 [3368]: s126 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:2:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0
2014-01-22 00:45:25-0500 521426 [17243]: 0322a407 aio collect 0 0x7f87ac0008c0:0x7f87ac0008d0:0x7f87ac101000 result 0:0 match len 512
2014-01-22 00:45:25-0500 521426 [17243]: read_sectors delta_leader offset 512 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids
2014-01-22 00:45:26-0500 521427 [3368]: s126 add_lockspace fail result -90
2014-01-22 00:45:36-0500 521437 [3368]: s127 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:2:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0
and the Data Center has been down ever since.
Best I can figure is that something is supposed to be in dom_md/ids, but that
file is empty:
ls /rhev/data-center/mnt/glusterSD/10.41.65.2\:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ -l
total 1029
-rw-rw---- 1 vdsm kvm 0 Jan 22 00:44 ids
-rw-rw---- 1 vdsm kvm 0 Jan 16 18:50 inbox
-rw-rw---- 1 vdsm kvm 2097152 Jan 21 18:20 leases
-rw-r--r-- 1 vdsm kvm 491 Jan 21 18:20 metadata
-rw-rw---- 1 vdsm kvm 0 Jan 16 18:50 outbox
Any hints as to how to put whatever is needed into 'ids', or reinitialize the
sanlock system--or a better diagnosis and solution--gladly accepted.
Ted Miller
Elkhart, IN, USA
--------------000306010201050105090700
Content-Type: text/html; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
</head>
<body text="#000000" bgcolor="#FFFFFF">
After some indirect path to getting here, I believe that the cause
of my problem is with sanlock. (See thread on 'Data Center Status
on the GUI flips between "Non Responsive" and
<br>
"Contending" '
for how I got here).<br>
<br>
My sanlock.log file is full of:<br>
<br>
<pre>2014-01-26 21:36:19-0500 36474 [2905]: s4576 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0</pre>
<pre>2014-01-26 21:36:19-0500 36474 [26767]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512</pre>
<pre>2014-01-26 21:36:19-0500 36474 [26767]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre>
<pre>2014-01-26 21:36:20-0500 36475 [2905]: s4576 add_lockspace fail result -90</pre>
<pre>2014-01-26 21:36:26-0500 36481 [2906]: s4577 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0</pre>
<pre>2014-01-26 21:36:26-0500 36481 [26809]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512</pre>
<pre>2014-01-26 21:36:26-0500 36481 [26809]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre>
<pre>2014-01-26 21:36:27-0500 36482 [2906]: s4577 add_lockspace fail result -90</pre>
<br>
The only other thing is after rebooting, when it says:<br>
<br>
<pre>2014-01-26 11:28:53-0500 29 [2897]: sanlock daemon started 2.8 host 09423427-2f70-4f68-9696-e41a93937ef0.office2a.l</pre>
<pre>2014-01-26 11:29:22-0500 57 [2905]: s1 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0</pre>
<pre>2014-01-26 11:29:22-0500 57 [3569]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c001000 result 0:0 match len 512</pre>
<pre>2014-01-26 11:29:22-0500 57 [3569]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre>
<pre>2014-01-26 11:29:23-0500 58 [2905]: s1 add_lockspace fail result -90</pre>
<pre>2014-01-26 11:29:32-0500 67 [2905]: s2 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0</pre>
<pre>2014-01-26 11:29:32-0500 67 [3731]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512</pre>
<pre>2014-01-26 11:29:32-0500 67 [3731]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre>
<pre>2014-01-26 11:29:33-0500 68 [2905]: s2 add_lockspace fail result -90</pre>
<pre>2014-01-26 11:29:42-0500 77 [2905]: s3 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:1:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0</pre>
<pre>2014-01-26 11:29:42-0500 77 [3806]: 0322a407 aio collect 0 0x7f196c0008c0:0x7f196c0008d0:0x7f196c101000 result 0:0 match len 512</pre>
<pre>2014-01-26 11:29:42-0500 77 [3806]: read_sectors delta_leader offset 0 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre>
<pre>2014-01-26 11:29:43-0500 78 [2905]: s3 add_lockspace fail result -90</pre>
<br>
Background:<br>
Setup is 2 hosts on Centos 6.5 with gluster storage, engine on
separate Centos 6.5, all up to date.<br>
Originally set up running:<br>
<blockquote>Data Center: default<br>
Cluster: default<br>
Master storage domain: VM<br>
</blockquote>
log had entries like:<br>
<blockquote>
<pre>2013-12-18 23:20:39-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -103:0 match res</pre>
<pre>2013-12-18 23:20:39-0500 4510 [3318]: s1 delta_renew read rv -103 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre>
<pre>2013-12-18 23:20:39-0500 4510 [3318]: s1 renewal error -103 delta_length 9 last_success 4481</pre>
<pre>2013-12-18 23:20:40-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res</pre>
<pre>2013-12-18 23:20:40-0500 4510 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre>
<pre>2013-12-18 23:20:40-0500 4510 [3318]: s1 renewal error -107 delta_length 0 last_success 4481</pre>
<pre>2013-12-18 23:20:40-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res</pre>
<pre>2013-12-18 23:20:40-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre>
<pre>2013-12-18 23:20:40-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481</pre>
<pre>2013-12-18 23:20:41-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res</pre>
<pre>2013-12-18 23:20:41-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre>
<pre>2013-12-18 23:20:41-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481</pre>
<br>
</blockquote>
storage 'VM' had a Split-Brain. In trying to fix split-brain, file
dom_md/ids was erased. I never revived the Data Center 'default'
after that.<br>
<blockquote>log is full of: <br>
<pre>2014-01-16 17:23:21-0500 62902 [29262]: open error -13 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre>
<pre>2014-01-16 17:23:21-0500 62902 [29262]: s3 open_disk /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids error -13</pre>
<pre>2014-01-16 17:23:22-0500 62903 [3368]: s3 add_lockspace fail result -19</pre>
<pre>2014-01-16 17:23:31-0500 62912 [3368]: s4 lockspace 9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88:1:/rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids:0</pre>
<br>
</blockquote>
Jan. 16: Created new: Data Center: mill; Cluster: one; Master
storage domain: 'VM2' All was working.<br>
<blockquote>Logs looked like this:<br>
<pre>2013-12-18 23:20:39-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -103:0 match res</pre>
<pre>2013-12-18 23:20:39-0500 4510 [3318]: s1 delta_renew read rv -103 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre>
<pre>2013-12-18 23:20:39-0500 4510 [3318]: s1 renewal error -103 delta_length 9 last_success 4481</pre>
<pre>2013-12-18 23:20:40-0500 4510 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res</pre>
<pre>2013-12-18 23:20:40-0500 4510 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre>
<pre>2013-12-18 23:20:40-0500 4510 [3318]: s1 renewal error -107 delta_length 0 last_success 4481</pre>
<pre>2013-12-18 23:20:40-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res</pre>
<pre>2013-12-18 23:20:40-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre>
<pre>2013-12-18 23:20:40-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481</pre>
<pre>2013-12-18 23:20:41-0500 4511 [3318]: 9b6e6bf1 aio collect 0 0x7fd3b40008c0:0x7fd3b40008d0:0x7fd3c59ae000 result -107:0 match res</pre>
<pre>2013-12-18 23:20:41-0500 4511 [3318]: s1 delta_renew read rv -107 offset 0 /rhev/data-center/mnt/10.41.65.4:_VM/9b6e6bf1-29a1-4abf-8e85-bf9c4a329c88/dom_md/ids</pre>
<pre>2013-12-18 23:20:41-0500 4511 [3318]: s1 renewal error -107 delta_length 0 last_success 4481</pre>
</blockquote>
For a while, very little was logged at all, and this were working
fine. Then, <br>
<br>
Jan 22 logs went from:<br>
<br>
<blockquote>
<pre>2014-01-22 00:44:56-0500 521397 [30933]: leader3 m 0 v 0 ss 0 nh 0 mh 0 oi 0 og 0 lv 0</pre>
<pre>2014-01-22 00:44:56-0500 521397 [30933]: leader4 sn rn ts 0 cs 0</pre>
<pre>2014-01-22 00:44:56-0500 521397 [30933]: s123 delta_renew verify_leader error -223</pre>
<pre>2014-01-22 00:44:56-0500 521397 [30933]: s123 renewal error -223 delta_length 0 last_success 521369</pre>
<pre>2014-01-22 00:44:57-0500 521398 [3362]: s123 kill 32599 sig 15 count 9</pre>
<pre>2014-01-22 00:44:57-0500 521398 [30933]: verify_leader 2 wrong magic 0 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre>
<pre>2014-01-22 00:44:57-0500 521398 [30933]: leader1 delta_renew error -223 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c host_id 2</pre>
<pre>2014-01-22 00:44:57-0500 521398 [30933]: leader2 path /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids offset 0</pre>
<pre>2014-01-22 00:44:57-0500 521398 [30933]: leader3 m 0 v 0 ss 0 nh 0 mh 0 oi 0 og 0 lv 0</pre>
</blockquote>
<br>
to:<br>
<br>
<blockquote>
<pre>2014-01-22 00:45:21-0500 521422 [17199]: 0322a407 aio collect 0 0x7f87ac0008c0:0x7f87ac0008d0:0x7f87ac101000 result 0:0 match len 512</pre>
<pre>2014-01-22 00:45:21-0500 521422 [17199]: read_sectors delta_leader offset 512 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre>
<pre>2014-01-22 00:45:22-0500 521423 [3368]: s125 add_lockspace fail result -90</pre>
<pre>2014-01-22 00:45:25-0500 521426 [3368]: s126 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:2:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0</pre>
<pre>2014-01-22 00:45:25-0500 521426 [17243]: 0322a407 aio collect 0 0x7f87ac0008c0:0x7f87ac0008d0:0x7f87ac101000 result 0:0 match len 512</pre>
<pre>2014-01-22 00:45:25-0500 521426 [17243]: read_sectors delta_leader offset 512 rv -90 /rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids</pre>
<pre>2014-01-22 00:45:26-0500 521427 [3368]: s126 add_lockspace fail result -90</pre>
<pre>2014-01-22 00:45:36-0500 521437 [3368]: s127 lockspace 0322a407-2b16-40dc-ac67-13d387c6eb4c:2:/rhev/data-center/mnt/glusterSD/10.41.65.2:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ids:0</pre>
</blockquote>
<br>
and the Data Center has been down ever since.<br>
<br>
Best I can figure is that something is supposed to be in dom_md/ids,
but that file is empty:<br>
<br>
<pre>ls /rhev/data-center/mnt/glusterSD/10.41.65.2\:VM2/0322a407-2b16-40dc-ac67-13d387c6eb4c/dom_md/ -l</pre>
<pre>total 1029</pre>
<pre>-rw-rw---- 1 vdsm kvm 0 Jan 22 00:44 ids</pre>
<pre>-rw-rw---- 1 vdsm kvm 0 Jan 16 18:50 inbox</pre>
<pre>-rw-rw---- 1 vdsm kvm 2097152 Jan 21 18:20 leases</pre>
<pre>-rw-r--r-- 1 vdsm kvm 491 Jan 21 18:20 metadata</pre>
<pre>-rw-rw---- 1 vdsm kvm 0 Jan 16 18:50 outbox
</pre>
Any hints as to how to put whatever is needed into 'ids', or
reinitialize the sanlock system--or a better diagnosis and
solution--gladly accepted.<br>
<br>
Ted Miller<br>
Elkhart, IN, USA<br>
<br>
</body>
</html>
--------------000306010201050105090700--
10 years, 3 months
[Users] oVirt 3.4.0 issues with storage
by Jon Archer
Hi all,
I upgraded to 3.4 beta this evening and seem to have developed a problem
with the storage domains. I seem unable to bring them online getting the
error in the console "Failed to activate Storage Domain".
Also alongside this error is a warning, "Invalid status on Data Center
local_datacenter. Setting status to Non Responsive"
Anyone seen this? Or any ideas?
Thanks
Jon
10 years, 3 months
[Users] mavericks as a guest vm?
by Mark
I have older Xserve hardware that is 64 bit but the bios is 32 bit. Fusion
runs mavericks with no problem. The vm runs in 64 bit mode.
I'd like to run mavericks as vm under ovirt on this hardware.
Google searches I've done keep pointing to vmware.
Any pointers?
10 years, 3 months
[Users] Configure VNC-Password timeout
by Sven Kieske
Hi,
the default timeout for the vnc passwords
is 2 Minutes. I'd like to reconfigure
this.
Is there an engine-config value or is
this hardcoded? How can this be changed?
--
Mit freundlichen Grüßen / Regards
Sven Kieske
Systemadministrator
Mittwald CM Service GmbH & Co. KG
Königsberger Straße 6
32339 Espelkamp
T: +49-5772-293-100
F: +49-5772-293-333
https://www.mittwald.de
Geschäftsführer: Robert Meyer
St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen
Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
10 years, 3 months
