3.6.0 nighty hosted-engine deploy quits silently on second host
by Alex Crow
Hi.
I've manged to get a 3.6.0-pre self-hosted engine running on one Centos
7.1 host (glusterfs-backed) but on the second host it the deploy process
just quits with no error before I even get to configure the storage:
[root@olympia ~]# hosted-engine --deploy
[ INFO ] Stage: Initializing
[ INFO ] Generating a temporary VNC password.
[ INFO ] Stage: Environment setup
Continuing will configure this host for serving as hypervisor
and create a VM where you have to install oVirt Engine afterwards.
Are you sure you want to continue? (Yes, No)[Yes]: yes
Configuration files: []
Log file:
/var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20150912174714-4jr1ru.log
Version: otopi-1.4.0_master
(otopi-1.4.0-0.0.master.20150910103318.gitdd73099.el7)
[ INFO ] Hardware supports virtualization
[ INFO ] Stage: Environment packages setup
[ INFO ] Stage: Programs detection
[ INFO ] Stage: Environment setup
[root@olympia ~]#
My setup log ends like this:
2015-09-12 17:15:12 DEBUG otopi.context context.dumpEnvironment:510 ENV
COMMAND/reboot=str:'/sbin/reboot'
2015-09-12 17:15:12 DEBUG otopi.context context.dumpEnvironment:510 ENV
COMMAND/remote-viewer=str:'/bin/remote-viewer'
2015-09-12 17:15:12 DEBUG otopi.context context.dumpEnvironment:510 ENV
COMMAND/service=str:'/sbin/service'
2015-09-12 17:15:12 DEBUG otopi.context context.dumpEnvironment:510 ENV
COMMAND/sshd=str:'/sbin/sshd'
2015-09-12 17:15:12 DEBUG otopi.context context.dumpEnvironment:510 ENV
COMMAND/sudo=str:'/bin/sudo'
2015-09-12 17:15:12 DEBUG otopi.context context.dumpEnvironment:510 ENV
COMMAND/systemctl=str:'/bin/systemctl'
2015-09-12 17:15:12 DEBUG otopi.context context.dumpEnvironment:510 ENV
COMMAND/truncate=str:'/bin/truncate'
2015-09-12 17:15:12 DEBUG otopi.context context.dumpEnvironment:510 ENV
COMMAND/umount=str:'/bin/umount'
2015-09-12 17:15:12 DEBUG otopi.context context.dumpEnvironment:510 ENV
COMMAND/vdsm-tool=str:'/bin/vdsm-tool'
2015-09-12 17:15:12 DEBUG otopi.context context.dumpEnvironment:514
ENVIRONMENT DUMP - END
2015-09-12 17:15:12 DEBUG otopi.context context._executeMethod:142 Stage
programs METHOD otopi.plugins.otopi.services.systemd.Plugin._prog
rams
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.executeRaw:828 execute: ('/bin/systemctl', 'show-environment'), exec
utable='None', cwd='None', env=None
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.executeRaw:878 execute-result: ('/bin/systemctl', 'show-environment'
), rc=0
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.execute:936 execute-output: ('/bin/systemctl',
'show-environment') stdout:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
LANG=en_GB.UTF-8
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.execute:941 execute-output: ('/bin/systemctl',
'show-environment') stderr:
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
systemd._programs:64 registering systemd provider
2015-09-12 17:15:12 DEBUG otopi.context context._executeMethod:142 Stage
programs METHOD otopi.plugins.otopi.services.rhel.Plugin._programs
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.rhel
plugin.executeRaw:828 execute: ('/bin/systemctl', 'show-environment'),
executable='None', cwd='None', env=None
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.rhel
plugin.executeRaw:878 execute-result: ('/bin/systemctl',
'show-environment'), rc=0
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.rhel
plugin.execute:936 execute-output: ('/bin/systemctl',
'show-environment') stdout:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
LANG=en_GB.UTF-8
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.rhel
plugin.execute:941 execute-output: ('/bin/systemctl',
'show-environment') stderr:
2015-09-12 17:15:12 DEBUG otopi.context context._executeMethod:142 Stage
programs METHOD otopi.plugins.otopi.services.openrc.Plugin._programs
2015-09-12 17:15:12 DEBUG otopi.context context._executeMethod:142 Stage
programs METHOD
otopi.plugins.ovirt_hosted_engine_setup.ha.ha_services.Plugin._programs
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
systemd.status:105 check service ovirt-ha-agent status
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.executeRaw:828 execute: ('/bin/systemctl', 'status',
'ovirt-ha-agent.service'), executable='None', cwd='None', env=None
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.executeRaw:878 execute-result: ('/bin/systemctl', 'status',
'ovirt-ha-agent.service'), rc=3
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.execute:936 execute-output: ('/bin/systemctl', 'status',
'ovirt-ha-agent.service') stdout:
ovirt-ha-agent.service - oVirt Hosted Engine High Availability
Monitoring Agent
Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service;
disabled)
Active: inactive (dead)
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.execute:941 execute-output: ('/bin/systemctl', 'status',
'ovirt-ha-agent.service') stderr:
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
systemd.status:105 check service ovirt-ha-broker status
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.executeRaw:828 execute: ('/bin/systemctl', 'status',
'ovirt-ha-broker.service'), executable='None', cwd='None', env=None
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.executeRaw:878 execute-result: ('/bin/systemctl', 'status',
'ovirt-ha-broker.service'), rc=3
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.execute:936 execute-output: ('/bin/systemctl', 'status',
'ovirt-ha-broker.service') stdout:
ovirt-ha-broker.service - oVirt Hosted Engine High Availability
Communications Broker
Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service;
disabled)
Active: inactive (dead)
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.execute:941 execute-output: ('/bin/systemctl', 'status',
'ovirt-ha-broker.service') stderr:
2015-09-12 17:15:12 INFO otopi.context context.runSequence:427 Stage:
Environment setup
2015-09-12 17:15:12 DEBUG otopi.context context.runSequence:431 STAGE
late_setup
2015-09-12 17:15:12 DEBUG otopi.context context._executeMethod:142 Stage
late_setup METHOD
otopi.plugins.ovirt_hosted_engine_setup.vdsmd.vdsmconf.Plugin._late_setup
2015-09-12 17:15:12 DEBUG otopi.context context._executeMethod:142 Stage
late_setup METHOD
otopi.plugins.ovirt_hosted_engine_setup.system.vdsmenv.Plugin._late_setup
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
systemd.status:105 check service vdsmd status
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.executeRaw:828 execute: ('/bin/systemctl', 'status',
'vdsmd.service'), executable='None', cwd='None', env=None
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.executeRaw:878 execute-result: ('/bin/systemctl', 'status',
'vdsmd.service'), rc=0
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.execute:936 execute-output: ('/bin/systemctl', 'status',
'vdsmd.service') stdout:
vdsmd.service - Virtual Desktop Server Manager
Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled)
Active: active (running) since Sat 2015-09-12 17:13:53 BST; 1min 19s ago
Process: 3140 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh
--pre-start (code=exited, status=0/SUCCESS)
Main PID: 3234 (vdsm)
CGroup: /system.slice/vdsmd.service
└─3234 /usr/bin/python /usr/share/vdsm/vdsm
Sep 12 17:13:55 olympia python[3234]: DIGEST-MD5 client step 1
Sep 12 17:13:55 olympia python[3234]: DIGEST-MD5 ask_user_info()
Sep 12 17:13:55 olympia python[3234]: DIGEST-MD5 client step 1
Sep 12 17:13:55 olympia python[3234]: DIGEST-MD5 ask_user_info()
Sep 12 17:13:55 olympia python[3234]: DIGEST-MD5 make_client_response()
Sep 12 17:13:55 olympia python[3234]: DIGEST-MD5 client step 2
Sep 12 17:13:55 olympia python[3234]: DIGEST-MD5 parse_server_challenge()
Sep 12 17:13:55 olympia python[3234]: DIGEST-MD5 ask_user_info()
Sep 12 17:13:55 olympia python[3234]: DIGEST-MD5 make_client_response()
Sep 12 17:13:55 olympia python[3234]: DIGEST-MD5 client step 3
2015-09-12 17:15:12 DEBUG otopi.plugins.otopi.services.systemd
plugin.execute:941 execute-output: ('/bin/systemctl', 'status',
'vdsmd.service') stderr:
2015-09-12 17:15:12 DEBUG
otopi.plugins.ovirt_hosted_engine_setup.system.vdsmenv
vdsmenv._connect:62 {'status': {'message': 'Done', 'code': 0}, 'info':
{'systemProductName': 'X9DR3-F', 'systemSerialNumber': '0123456789',
'systemFamily': 'To be filled by O.E.M.', 'systemVersion': '0123456789',
'systemUUID': '00000000-0000-0000-0000-002590E5DAD8',
'systemManufacturer': 'Supermicro'}}
I will also submit this to bugzilla, but any idea for a quick fix?
Cheers
Alex
.
9 years, 2 months
multi-able nics on hypervisor
by Kyle Bassett
I have a few hypervisors that have 2 pics, up till now only one was configured with dhcp.
I enabled the 2nd nic and set it to dhcp also - as soon as I do that it make the other nic “unconfigured” - even when I set it to static it marks the other “unconfigured”.
Can I not have 2 interfaces configured at the same time - I am doing this from the selection menu on the hypervisor.
Ideally I want one for management and the other for virtual machines.
Thanks
Kyle
9 years, 2 months
RESTAPI: 'action type cpu profile empty' if more then one profile defined in the cluster
by Daniel Helgenberger
Hello,
I still seem to have an issue in ovirt with a [BZ1160846].
In my case, foreman is unable to add hosts if I have multiple CPU
Profiles defined in the cluster:
Failed to create a compute oVirt (oVirt) instance $hostname: action type
cpu profile empty
BZ says this was resolved in RHEV 3.5.1; but running 3.5.3 I still have
this issue (if this BZ is even applicable)
Thanks!
Versions:
oVirt 3.5.3 el7
Foreman 1.8.2
Thanks!
[BZ1160846] https://bugzilla.redhat.com/show_bug.cgi?id=1160846
--
Daniel Helgenberger
m box bewegtbild GmbH
P: +49/30/2408781-22
F: +49/30/2408781-10
ACKERSTR. 19
D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner
Handeslregister: Amtsgericht Charlottenburg / HRB 112767
9 years, 2 months
vdsm high mem usage
by Michael Kleinpaste
Hi everybody.
So I ran into that high mem usage thing. The problem I have with patching
is that this is a live system so I can't do it mid day. Can anybody tell
me if it is possible to just restart the vdsm service or does the host have
to be in "maintenance mode" before restarting it? It is using gluster
storage, if that makes a difference as well.
Thanks,
--
*Michael Kleinpaste*
Senior Systems Administrator
SharperLending, LLC.
www.SharperLending.com
Michael.Kleinpaste(a)SharperLending.com
(509) 324-1230 Fax: (509) 324-1234
9 years, 2 months
[ANN] oVirt 3.6.0 Fifth Beta Release is now available for testing
by Sandro Bonazzola
The oVirt Project is pleased to announce the availability
of the Fifth Beta release of oVirt 3.6 for testing, as of September 10th,
2015.
oVirt is an open source alternative to VMware vSphere, and provides an
excellent KVM management interface for multi-node virtualization.
This release is available now for Fedora 22,
Red Hat Enterprise Linux 6.7, CentOS Linux 6.7 (or similar) and
Red Hat Enterprise Linux 7.1, CentOS Linux 7.1 (or similar).
This release supports Hypervisor Hosts running
Red Hat Enterprise Linux 7.1, CentOS Linux 7.1 (or similar),
Fedora 21 and Fedora 22.
Highly experimental support for Debian 8.1 Jessie has been added too.
This release of oVirt 3.6.0 includes numerous bug fixes.
See the release notes [1] for an initial list of the new features and bugs
fixed.
Please refer to release notes [1] for Installation / Upgrade instructions.
New oVirt Node ISO and oVirt Live ISO will be available soon as well[2].
Please note that mirrors[3] may need usually one day before being
synchronized.
Please refer to the release notes for known issues in this release.
[1] http://www.ovirt.org/OVirt_3.6_Release_Notes
[2] http://plain.resources.ovirt.org/pub/ovirt-3.6-pre/iso/
[3] http://www.ovirt.org/Repository_mirrors#Current_mirrors
--
Sandro Bonazzola
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
9 years, 2 months
datacenter and cluster UUID retrieving .....
by Jean-Pierre Ribeauville
--_000_1051EFB4D3A1704680C38CCAAC5836D279B8DC1DWPTXMAIL1ptxaxw_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Hi,
I'm not sure it's the right place to post following question :
By using a RHEV Cluster , how may I , programmatically within a node of the=
cluster , retrieve relevant Datacenter and Cluster UUID ?
Thanks for help.
Regards,
J.P. Ribeauville
P: +33.(0).1.47.17.27.87
Puteaux 3 Etage 5 Bureau 4
jpribeauville(a)axway.com<mailto:jpribeauville@axway.com>
http://www.axway.com<http://www.axway.com/>
P Pensez =E0 l'environnement avant d'imprimer.
--_000_1051EFB4D3A1704680C38CCAAC5836D279B8DC1DWPTXMAIL1ptxaxw_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-=
1">
<meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Webdings;
panose-1:5 3 1 2 1 5 9 6 7 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"FR" link=3D"blue" vlink=3D"purple">
<div class=3D"WordSection1">
<p class=3D"MsoNormal">Hi,<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US">I’m not sure it’s t=
he right place to post following question :<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US">By using a RHEV Cluster , how m=
ay I , programmatically within a node of the cluster , retrieve relevant Da=
tacenter and Cluster UUID ?<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US">Thanks for help.<o:p></o:p></sp=
an></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US">Regards,<o:p></o:p></span></p>
<table class=3D"MsoNormalTable" border=3D"0" cellspacing=3D"0" cellpadding=
=3D"0">
<tbody>
<tr>
<td valign=3D"top" style=3D"padding:0cm 4.5pt 0cm 0cm">
<p class=3D"MsoNormal"><b><span lang=3D"EN-US" style=3D"font-size:9.0pt;fon=
t-family:"Arial","sans-serif";color:#888888;mso-fareast=
-language:FR"><o:p> </o:p></span></b></p>
<p class=3D"MsoNormal"><b><span lang=3D"EN-US" style=3D"font-size:8.5pt;fon=
t-family:"Arial","sans-serif";color:#888888;mso-fareast=
-language:FR">J.P. Ribeauville<o:p></o:p></span></b></p>
<p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"font-size:8.5pt;font-f=
amily:"Arial","sans-serif";color:#737373;mso-fareast-la=
nguage:FR"><o:p> </o:p></span></p>
</td>
</tr>
<tr>
<td style=3D"padding:2.25pt 4.5pt 4.5pt 0cm">
<p class=3D"MsoNormal"><span style=3D"font-size:8.5pt;font-family:"Ari=
al","sans-serif";color:red;mso-fareast-language:FR">P:
</span><span style=3D"font-size:8.5pt;font-family:"Arial","s=
ans-serif";color:#737373;mso-fareast-language:FR">+33.(0).1.47.17.=
27.87</span><span style=3D"font-size:8.5pt;font-family:"Arial",&q=
uot;sans-serif";color:#737373;mso-fareast-language:FR"><o:p></o:p></sp=
an></p>
<p class=3D"MsoNormal"><span style=3D"font-size:8.5pt;font-family:"Ari=
al","sans-serif";color:#737373;mso-fareast-language:FR"><o:p=
> </o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:8.5pt;font-family:"Ari=
al","sans-serif";color:#737373;mso-fareast-language:FR"><o:p=
> </o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:8.5pt;font-family:"Ari=
al","sans-serif";color:#737373;mso-fareast-language:FR">Pute=
aux 3 Etage 5 Bureau 4<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:8.5pt;font-family:"Ari=
al","sans-serif";color:#737373;mso-fareast-language:FR"><o:p=
> </o:p></span></p>
<p class=3D"MsoNormal"><u><span lang=3D"EN-US" style=3D"font-size:9.0pt;col=
or:red;mso-fareast-language:FR"><a href=3D"mailto:jpribeauville@axway.com">=
<span lang=3D"FR" style=3D"color:blue">jpribeauville(a)axway.com</span></a></=
span></u><u><span style=3D"font-size:9.0pt;color:red;mso-fareast-language:F=
R"><br>
</span></u><span style=3D"font-size:9.0pt;color:red;mso-fareast-language:FR=
"><a href=3D"http://www.axway.com/"><span style=3D"color:red">http://www.ax=
way.com</span></a><o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:8.5pt;font-family:"Ari=
al","sans-serif";color:#737373;mso-fareast-language:FR"><o:p=
> </o:p></span></p>
</td>
</tr>
</tbody>
</table>
<p class=3D"MsoNormal"><span style=3D"display:none;mso-fareast-language:FR"=
><o:p> </o:p></span></p>
<table class=3D"MsoNormalTable" border=3D"0" cellspacing=3D"0" cellpadding=
=3D"0">
<tbody>
<tr>
<td style=3D"padding:4.5pt 4.5pt 4.5pt 0cm">
<p class=3D"MsoNormal"><span style=3D"font-size:18.0pt;font-family:Webdings=
;color:#888888;mso-fareast-language:FR">P</span><span style=3D"font-size:8.=
0pt;font-family:"Arial","sans-serif";color:#888888;mso-=
fareast-language:FR"> Pensez =E0 l’environnement avant d’i=
mprimer.</span><span style=3D"font-size:8.0pt;font-family:"Arial"=
,"sans-serif";color:#888888;mso-fareast-language:FR"><o:p></o:p><=
/span></p>
</td>
</tr>
</tbody>
</table>
<p class=3D"MsoNormal"><span style=3D"mso-fareast-language:FR"><o:p> <=
/o:p></span></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>
--_000_1051EFB4D3A1704680C38CCAAC5836D279B8DC1DWPTXMAIL1ptxaxw_--
9 years, 2 months
strange iscsi issue
by Demeter Tibor
------=_Part_1904412_191957680.1441657611396
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi All,
I have to create a test environment for testing purposes, because we need to testing our new 10gbe infrastructure.
One server that have a 10gbe nic - this is the vdsm host and ovirt portal.
One server that have a 10gbe nic - this is the storage.
Its connected to each other throught a dlink 10gbe switch.
Everything good and nice, the server can connect to storage, I can make and run VMs, but the storage performance from inside VM seems to be 1Gb/sec only.
I did try the iperf command for testing connections beetwen servers, and it was 9.40 GB/sec. I have try to use hdparm -tT /dev/mapper/iscsidevice and also it was 400-450 MB/sec. I've got same result on storage server.
So:
- hdparm test on local storage ~ 400 mb/sec
- hdparm test on ovirt node server through attached iscsi device ~ 400 Mb/sec
- hdparm test from inside vm on local virtual disk - 93-102 Mb /sec
The question is : Why?
ps. I Have only one ovirtmgmt device, so there are no other networks. The router is only 1gbe/sec, but i've tested and the traffic does not going through this.
Thanks in advance,
Regards,
Tibor
------=_Part_1904412_191957680.1441657611396
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable
<html><body><div style=3D"font-family: arial, helvetica, sans-serif; font-s=
ize: 12pt; color: #000000"><div>Hi All,</div><div><br data-mce-bogus=3D"1">=
</div><div>I have to create a test environment for testing purposes, becaus=
e we need to testing our new 10gbe infrastructure.</div><div>One server tha=
t have a 10gbe nic - this is the vdsm host and ovirt portal.</div><div>One =
server that have a 10gbe nic - this is the storage.</div><div><br data-mce-=
bogus=3D"1"></div><div>Its connected to each other throught a dlink 10gbe s=
witch.</div><div><br></div><div>Everything good and nice, the server can co=
nnect to storage, I can make and run VMs, but the storage performance from =
inside VM seems to be 1Gb/sec only. </div><div>I did try the iperf com=
mand for testing connections beetwen servers, and it was 9.40 GB/sec. I hav=
e try to use hdparm -tT /dev/mapper/iscsidevice and also it was 400-450 MB/=
sec. I've got same result on storage server.</div><div><br data-mce-bogus=
=3D"1"></div><div>So:</div><div><br data-mce-bogus=3D"1"></div><div>- hdpar=
m test on local storage ~ 400 mb/sec</div><div>- hdparm test on ovirt node =
server through attached iscsi device ~ 400 Mb/sec</div><div>- hdparm test f=
rom inside vm on local virtual disk - 93-102 Mb /sec</div><div><br data-mce=
-bogus=3D"1"></div><div>The question is : Why?</div><div><br data-mce-bogus=
=3D"1"></div><div>ps. I Have only one ovirtmgmt device, so there are no oth=
er networks. The router is only 1gbe/sec, but i've tested and the traffic d=
oes not going through this.</div><div><br data-mce-bogus=3D"1"></div>=
<div>Thanks in advance,</div><div><br data-mce-bogus=3D"1"></div><div>Regar=
ds, </div><div>Tibor<br></div><div data-marker=3D"__SIG_PRE__"><p></p>=
</div></div></body></html>
------=_Part_1904412_191957680.1441657611396--
9 years, 2 months
Question about upgrading ovirt 3.5.3 to 3.5.4
by Jason Keltz
Hi.
I have a system consisting of an engine + several hosts running 3.5.3,
and I want to upgrade everything to 3.5.4. According to the release
notes, all I should do is:
> # yum update "ovirt-engine-setup*"
> # engine-setup
I did this with engine, and it seemed to upgrade okay.
I'm puzzled whether this applies to the hosts as well? The release
notes aren't clear to me in that respect.
Thanks for any assistance!
Jason.
9 years, 2 months
urgent issue
by Chris Liebman
In attempting to put an ovirt cluster in production I'm running into some
off errors with gluster it looks like. Its 12 hosts each with one brick in
distributed-replicate. (actually 2 bricks but they are separate volumes)
[root@ovirt-node268 glusterfs]# rpm -qa | grep vdsm
vdsm-jsonrpc-4.16.20-0.el6.noarch
vdsm-gluster-4.16.20-0.el6.noarch
vdsm-xmlrpc-4.16.20-0.el6.noarch
vdsm-yajsonrpc-4.16.20-0.el6.noarch
vdsm-4.16.20-0.el6.x86_64
vdsm-python-zombiereaper-4.16.20-0.el6.noarch
vdsm-python-4.16.20-0.el6.noarch
vdsm-cli-4.16.20-0.el6.noarch
Everything was fine last week, however, today various clients in the
gluster cluster seem get "client quorum not met" periodically - when they
get this they take one of the bricks offline - this causes VM's to be
attempted to move - sometimes 20 at a time. That takes a long time :-(.
I've tried disabling automatic migration and teh VM's get paused when this
happens - resuming gets nothing at that point as the volumes mount on the
server hosting the VM is not connected:
from rhev-data-center-mnt-glusterSD-ovirt-node268.la.taboolasyndication.com:
_LADC-TBX-V02.log:
[2015-09-08 21:18:42.920771] W [MSGID: 108001]
[afr-common.c:4043:afr_notify] 2-LADC-TBX-V02-replicate-2: Client-quorum is not
met
[2015-09-08 21:18:42.931751] I [fuse-bridge.c:4900:fuse_thread_proc]
0-fuse: unmounting
/rhev/data-center/mnt/glusterSD/ovirt-node268.la.taboolasyndication.com:
_LADC-TBX-V02
[2015-09-08 21:18:42.931836] W [glusterfsd.c:1219:cleanup_and_exit]
(-->/lib64/libpthread.so.0(+0x7a51) [0x7f1bebc84a51]
-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xcd) [0x405e4d]
-->/usr/sbin/glusterfs(cleanup_and_exit+0x
65) [0x4059b5] ) 0-: received signum (15), shutting down
[2015-09-08 21:18:42.931858] I [fuse-bridge.c:5595:fini] 0-fuse: Unmounting
'/rhev/data-center/mnt/glusterSD/ovirt-node268.la.taboolasyndication.com:
_LADC-TBX-V02'.
And the mount is broken at that point:
[root@ovirt-node267 ~]# df
*df:
`/rhev/data-center/mnt/glusterSD/ovirt-node268.la.taboolasyndication.com:_LADC-TBX-V02':
Transport endpoint is not connected*
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda3 51475068 1968452 46885176 5% /
tmpfs 132210244 0 132210244 0% /dev/shm
/dev/sda2 487652 32409 429643 8% /boot
/dev/sda1 204580 260 204320 1% /boot/efi
/dev/sda5 1849960960 156714056 1599267616 9% /data1
/dev/sdb1 1902274676 18714468 1786923588 2% /data2
ovirt-node268.la.taboolasyndication.com:/LADC-TBX-V01
9249804800 727008640 8052899712 9%
/rhev/data-center/mnt/glusterSD/ovirt-node268.la.taboolasyndication.com:
_LADC-TBX-V01
ovirt-node251.la.taboolasyndication.com:/LADC-TBX-V03
1849960960 73728 1755907968 1%
/rhev/data-center/mnt/glusterSD/ovirt-node251.la.taboolasyndication.com:
_LADC-TBX-V03
The fix for that is to put the server in maintenance mode then activate it
again. But all VM's need to be migrated or stopped for that to work.
I'm not seeing any obvious network or disk errors......
Are their configuration options I'm missing?
9 years, 2 months
Host loses all network configuration on update to oVirt 3.5.4
by Patrick Hurrelmann
Hi all,
I just updated my existing oVirt 3.5.3 installation (iSCSI hosted-engine on
CentOS 7.1). The engine update went fine. Updating the hosts succeeds until the
first reboot. After a reboot the host does not come up again. It is missing all
network configuration. All network cfgs in /etc/sysconfig/network-scripts are
missing except ifcfg-lo. The host boots up without working networking. Using
IPMI and config backups, I was able to restore the lost network configs. Once
these are restored and the host is rebooted again all seems to be back to good.
This has now happend to 2 updated hosts (this installation has a total of 4
hosts, so 2 more to debug/try). I'm happy to assist in furter debugging.
Before updating the second host, I gathered some information. All these hosts
have 3 physical nics. One is used for the ovirtmgmt bridge and the other 2 are
used for iSCSI storage vlans.
ifcfgs before update:
/etc/sysconfig/network-scripts/ifcfg-em1
# Generated by VDSM version 4.16.20-0.el7.centos
DEVICE=em1
HWADDR=d0:67:e5:f0:e5:c6
BRIDGE=ovirtmgmt
ONBOOT=yes
NM_CONTROLLED=no
DEVICE=lo
IPADDR=127.0.0.1
NETMASK=255.0.0.0
NETWORK=127.0.0.0
# If you're having problems with gated making 127.0.0.0/8 a martian,
# you can change this to something else (255.255.255.255, for example)
BROADCAST=127.255.255.255
ONBOOT=yes
NAME=loopback
/etc/sysconfig/network-scripts/ifcfg-ovirtmgmt
# Generated by VDSM version 4.16.20-0.el7.centos
DEVICE=ovirtmgmt
TYPE=Bridge
DELAY=0
STP=off
ONBOOT=yes
IPADDR=1.2.3.16
NETMASK=255.255.255.0
GATEWAY=1.2.3.11
BOOTPROTO=none
DEFROUTE=yes
NM_CONTROLLED=no
HOTPLUG=no
/etc/sysconfig/network-scripts/ifcfg-p4p1
# Generated by VDSM version 4.16.20-0.el7.centos
DEVICE=p4p1
HWADDR=68:05:ca:01:bc:0c
ONBOOT=no
IPADDR=4.5.7.102
NETMASK=255.255.255.0
BOOTPROTO=none
MTU=9000
DEFROUTE=no
NM_CONTROLLED=no
/etc/sysconfig/network-scripts/ifcfg-p3p1
# Generated by VDSM version 4.16.20-0.el7.centos
DEVICE=p3p1
HWADDR=68:05:ca:18:86:45
ONBOOT=no
IPADDR=4.5.6.102
NETMASK=255.255.255.0
BOOTPROTO=none
MTU=9000
DEFROUTE=no
NM_CONTROLLED=no
/etc/sysconfig/network-scripts/ifcfg-lo
ip link before update:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: bond0: <BROADCAST,MULTICAST,MASTER> mtu 1500 qdisc noop state DOWN mode DEFAULT
link/ether 46:50:22:7a:f3:9d brd ff:ff:ff:ff:ff:ff
3: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovirtmgmt state UP mode DEFAULT qlen 1000
link/ether d0:67:e5:f0:e5:c6 brd ff:ff:ff:ff:ff:ff
4: p3p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
link/ether 68:05:ca:18:86:45 brd ff:ff:ff:ff:ff:ff
5: p4p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
link/ether 68:05:ca:01:bc:0c brd ff:ff:ff:ff:ff:ff
7: ovirtmgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
link/ether d0:67:e5:f0:e5:c6 brd ff:ff:ff:ff:ff:ff
8: ;vdsmdummy;: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
link/ether ce:0f:16:49:a7:da brd ff:ff:ff:ff:ff:ff
vdsm files before update:
/var/lib/vdsm
/var/lib/vdsm/bonding-defaults.json
/var/lib/vdsm/netconfback
/var/lib/vdsm/netconfback/ifcfg-ovirtmgmt
/var/lib/vdsm/netconfback/ifcfg-em1
/var/lib/vdsm/netconfback/route-ovirtmgmt
/var/lib/vdsm/netconfback/rule-ovirtmgmt
/var/lib/vdsm/netconfback/ifcfg-p4p1
/var/lib/vdsm/netconfback/ifcfg-p3p1
/var/lib/vdsm/persistence
/var/lib/vdsm/persistence/netconf
/var/lib/vdsm/persistence/netconf.1416666697752319079
/var/lib/vdsm/persistence/netconf.1416666697752319079/nets
/var/lib/vdsm/persistence/netconf.1416666697752319079/nets/san1
/var/lib/vdsm/persistence/netconf.1416666697752319079/nets/san2
/var/lib/vdsm/persistence/netconf.1416666697752319079/nets/ovirtmgmt
/var/lib/vdsm/upgrade
/var/lib/vdsm/upgrade/upgrade-unified-persistence
/var/lib/vdsm/transient
File in /var/lib/vdsm/netconfback each only contained a comment:
# original file did not exist
/var/lib/vdsm/persistence/netconf.1416666697752319079/nets/ovirtmgmt
{"nic": "em1", "netmask": "255.255.255.0", "bootproto": "none", "ipaddr": "1.2.3.16", "gateway": "1.2.3.11"}
/var/lib/vdsm/persistence/netconf.1416666697752319079/nets/san1
{"nic": "p3p1", "netmask": "255.255.255.0", "ipaddr": "4.5.6.102", "bridged": "false", "mtu": "9000"}
/var/lib/vdsm/persistence/netconf.1416666697752319079/nets/san2
{"nic": "p4p1", "netmask": "255.255.255.0", "ipaddr": "4.5.7.102", "bridged": "false", "mtu": "9000"}
After update and reboot, no ifcfg scripts are left. Only interface lo is up.
Syslog doess not seem to contain anything suspicious before refore reboot.
Log excerpts from bootup:
Sep 3 17:27:23 vhm-prd-02 network: Bringing up loopback interface: [ OK ]
Sep 3 17:27:23 vhm-prd-02 systemd-ovirt-ha-agent: Starting ovirt-ha-agent: [ OK ]
Sep 3 17:27:23 vhm-prd-02 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent.
Sep 3 17:27:23 vhm-prd-02 kernel: IPv6: ADDRCONF(NETDEV_UP): em1: link is not ready
Sep 3 17:27:23 vhm-prd-02 kernel: device em1 entered promiscuous mode
Sep 3 17:27:23 vhm-prd-02 network: Bringing up interface em1: [ OK ]
Sep 3 17:27:23 vhm-prd-02 kernel: IPv6: ADDRCONF(NETDEV_UP): ovirtmgmt: link is not ready
Sep 3 17:27:25 vhm-prd-02 avahi-daemon[778]: Joining mDNS multicast group on interface ovirtmgmt.IPv4 with address 1.2.3.16.
Sep 3 17:27:25 vhm-prd-02 avahi-daemon[778]: New relevant interface ovirtmgmt.IPv4 for mDNS.
Sep 3 17:27:25 vhm-prd-02 avahi-daemon[778]: Registering new address record for 1.2.3.16 on ovirtmgmt.IPv4.
Sep 3 17:27:26 vhm-prd-02 kernel: tg3 0000:03:00.0 em1: Link is up at 1000 Mbps, full duplex
Sep 3 17:27:26 vhm-prd-02 kernel: tg3 0000:03:00.0 em1: Flow control is off for TX and off for RX
Sep 3 17:27:26 vhm-prd-02 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): em1: link becomes ready
Sep 3 17:27:26 vhm-prd-02 kernel: ovirtmgmt: port 1(em1) entered forwarding state
Sep 3 17:27:26 vhm-prd-02 kernel: ovirtmgmt: port 1(em1) entered forwarding state
Sep 3 17:27:26 vhm-prd-02 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): ovirtmgmt: link becomes ready
Sep 3 17:27:26 vhm-prd-02 network: Bringing up interface ovirtmgmt: [ OK ]
Sep 3 17:27:26 vhm-prd-02 systemd: Started LSB: Bring up/down networking.
Sep 3 17:27:26 vhm-prd-02 systemd: Starting Network.
Sep 3 17:27:26 vhm-prd-02 systemd: Reached target Network.
So ovirtmgmt and em1 were restore and initialized just fine (p3p1 and p4p1
should have been started, too, but engine configured them as ONBOOT=no).
Further in messages (full log is attached):
Sep 3 17:27:26 vhm-prd-02 systemd: Starting Virtual Desktop Server Manager network restoration...
Sep 3 17:27:26 vhm-prd-02 systemd: Started OSAD daemon.
Sep 3 17:27:27 vhm-prd-02 systemd: Started Terminate Plymouth Boot Screen.
Sep 3 17:27:27 vhm-prd-02 systemd: Started Wait for Plymouth Boot Screen to Quit.
Sep 3 17:27:27 vhm-prd-02 systemd: Starting Serial Getty on ttyS1...
Sep 3 17:27:27 vhm-prd-02 systemd: Started Serial Getty on ttyS1.
Sep 3 17:27:27 vhm-prd-02 systemd: Starting Getty on tty1...
Sep 3 17:27:27 vhm-prd-02 systemd: Started Getty on tty1.
Sep 3 17:27:27 vhm-prd-02 systemd: Starting Login Prompts.
Sep 3 17:27:27 vhm-prd-02 systemd: Reached target Login Prompts.
Sep 3 17:27:27 vhm-prd-02 iscsid: iSCSI daemon with pid=1300 started!
Sep 3 17:27:27 vhm-prd-02 avahi-daemon[778]: Registering new address record for fe80::d267:e5ff:fef0:e5c6 on ovirtmgmt.*.
Sep 3 17:27:27 vhm-prd-02 kdumpctl: kexec: loaded kdump kernel
Sep 3 17:27:27 vhm-prd-02 kdumpctl: Starting kdump: [OK]
Sep 3 17:27:27 vhm-prd-02 systemd: Started Crash recovery kernel arming.
Sep 3 17:27:27 vhm-prd-02 avahi-daemon[778]: Registering new address record for fe80::d267:e5ff:fef0:e5c6 on em1.*.
Sep 3 17:27:27 vhm-prd-02 avahi-daemon[778]: Withdrawing address record for 1.2.3.16 on ovirtmgmt.
Sep 3 17:27:27 vhm-prd-02 avahi-daemon[778]: Leaving mDNS multicast group on interface ovirtmgmt.IPv4 with address 1.2.3.16.
Sep 3 17:27:27 vhm-prd-02 avahi-daemon[778]: Interface ovirtmgmt.IPv4 no longer relevant for mDNS.
Sep 3 17:27:27 vhm-prd-02 kernel: ovirtmgmt: port 1(em1) entered disabled state
Sep 3 17:27:27 vhm-prd-02 avahi-daemon[778]: Withdrawing address record for fe80::d267:e5ff:fef0:e5c6 on ovirtmgmt.
Sep 3 17:27:28 vhm-prd-02 avahi-daemon[778]: Withdrawing address record for fe80::d267:e5ff:fef0:e5c6 on em1.
Sep 3 17:27:28 vhm-prd-02 kernel: device em1 left promiscuous mode
Sep 3 17:27:28 vhm-prd-02 kernel: ovirtmgmt: port 1(em1) entered disabled state
Sep 3 17:27:28 vhm-prd-02 avahi-daemon[778]: Withdrawing workstation service for ovirtmgmt.
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: Traceback (most recent call last):
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: File "/usr/share/vdsm/vdsm-restore-net-config", line 345, in <module>
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: restore(args)
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: File "/usr/share/vdsm/vdsm-restore-net-config", line 314, in restore
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: unified_restoration()
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: File "/usr/share/vdsm/vdsm-restore-net-config", line 93, in unified_restoration
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: setupNetworks(nets, bonds, connectivityCheck=False, _inRollback=True)
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: File "/usr/share/vdsm/network/api.py", line 642, in setupNetworks
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: implicitBonding=False, _netinfo=_netinfo)
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: File "/usr/share/vdsm/network/api.py", line 213, in wrapped
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: ret = func(**attrs)
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: File "/usr/share/vdsm/network/api.py", line 429, in delNetwork
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: netEnt.remove()
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: File "/usr/share/vdsm/network/models.py", line 100, in remove
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: self.configurator.removeNic(self)
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: File "/usr/share/vdsm/network/configurators/ifcfg.py", line 215, in removeNic
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: self.configApplier.removeNic(nic.name)
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: File "/usr/share/vdsm/network/configurators/ifcfg.py", line 657, in removeNic
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: with open(cf) as nicFile:
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: IOError: [Errno 2] No such file or directory: u'/etc/sysconfig/network-scripts/ifcfg-p4p1'
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: Traceback (most recent call last):
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: File "/usr/bin/vdsm-tool", line 219, in main
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: return tool_command[cmd]["command"](*args)
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py", line 40, in restore_command
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: exec_restore(cmd)
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py", line 53, in exec_restore
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: raise EnvironmentError('Failed to restore the persisted networks')
Sep 3 17:27:28 vhm-prd-02 vdsm-tool: EnvironmentError: Failed to restore the persisted networks
Sep 3 17:27:28 vhm-prd-02 systemd: vdsm-network.service: main process exited, code=exited, status=1/FAILURE
Sep 3 17:27:28 vhm-prd-02 systemd: Failed to start Virtual Desktop Server Manager network restoration.
Sep 3 17:27:28 vhm-prd-02 systemd: Dependency failed for Virtual Desktop Server Manager.
Sep 3 17:27:28 vhm-prd-02 systemd:
Sep 3 17:27:28 vhm-prd-02 systemd: Unit vdsm-network.service entered failed state.
Sep 3 17:27:33 vhm-prd-02 systemd: Started Postfix Mail Transport Agent.
Sep 3 17:27:33 vhm-prd-02 systemd: Starting Multi-User System.
Sep 3 17:27:33 vhm-prd-02 systemd: Reached target Multi-User System.
Sep 3 17:27:33 vhm-prd-02 systemd: Starting Update UTMP about System Runlevel Changes...
Sep 3 17:27:33 vhm-prd-02 systemd: Starting Stop Read-Ahead Data Collection 10s After Completed Startup.
Sep 3 17:27:33 vhm-prd-02 systemd: Started Stop Read-Ahead Data Collection 10s After Completed Startup.
Sep 3 17:27:33 vhm-prd-02 systemd: Started Update UTMP about System Runlevel Changes.
Sep 3 17:27:33 vhm-prd-02 systemd: Startup finished in 2.964s (kernel) + 2.507s (initrd) + 15.996s (userspace) = 21.468s.
So, as I have two more hosts, that need updating, I'm happy to assist in
bisecting and debugging this update issue. Suggestions and help are very
welcome.
Thank you very much and with best regards
Patrick
--
Lobster SCM GmbH, Hindenburgstraße 15, D-82343 Pöcking
HRB 178831, Amtsgericht München
Geschäftsführer: Dr. Martin Fischer, Rolf Henrich
9 years, 2 months