- Users - oVirt List Archives

Vagrant Plugin
by Jeremy Tourville 29 Mar '19

29 Mar '19

I am having some trouble getting the Ovirt Vagrant plugin working. I was able to get Vagrant installed and could even run the example scenario listed in the blog. https://www.ovirt.org/blog/2017/02/using-oVirt-vagrant.html My real issue is getting a vm generated by the SecGen project https://github.com/SecGen/SecGen to come up. If I use the VirtualBox provider everything works as expected and I can launch the vm with vagrant up. If I try to run using Ovirt provider it fails. I had originally posted this over in Google groups / Vagrant forums and it was suggested to take it to Ovirt. Hopefully, somebody here has some insights. The process fails quickly with the following output. Can anyone give some suggestions on how to fix the issue? I have also included a copy of my vagrantfile below. Thanks in advance for your assistance! ***Output*** Bringing machine 'escalation' up with 'ovirt4' provider... ==> escalation: Creating VM with the following settings... ==> escalation: -- Name: SecGen-default-scenario-escalation ==> escalation: -- Cluster: default ==> escalation: -- Template: debian_stretch_server_291118 ==> escalation: -- Console Type: spice ==> escalation: -- Memory: ==> escalation: ---- Memory: 512 MB ==> escalation: ---- Maximum: 512 MB ==> escalation: ---- Guaranteed: 512 MB ==> escalation: -- Cpu: ==> escalation: ---- Cores: 1 ==> escalation: ---- Sockets: 1 ==> escalation: ---- Threads: 1 ==> escalation: -- Cloud-Init: false ==> escalation: An error occured. Recovering.. ==> escalation: VM is not created. Please run `vagrant up` first. /home/secgenadmin/.vagrant.d/gems/2.4.4/gems/ovirt-engine-sdk-4.0.12/lib/ovirtsdk4/service.rb:52:in `raise_error': Fault reason is "Operation Failed". Fault detail is "Entity not found: Cluster: name=default". HTTP response code is 404. (OvirtSDK4::Error) from /home/secgenadmin/.vagrant.d/gems/2.4.4/gems/ovirt-engine-sdk-4.0.12/lib/ovirtsdk4/service.rb:67:in `check_fault' from /home/secgenadmin/.vagrant.d/gems/2.4.4/gems/ovirt-engine-sdk-4.0.12/lib/ovirtsdk4/services.rb:35570:in `add' from /home/secgenadmin/.vagrant.d/gems/2.4.4/gems/vagrant-ovirt4-1.2.2/lib/vagrant-ovirt4/action/create_vm.rb:67:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/warden.rb:50:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/builtin/before_trigger.rb:23:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/warden.rb:50:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/builtin/after_trigger.rb:26:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/warden.rb:50:in `call' from /home/secgenadmin/.vagrant.d/gems/2.4.4/gems/vagrant-ovirt4-1.2.2/lib/vagrant-ovirt4/action/set_name_of_domain.rb:17:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/warden.rb:50:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/builtin/before_trigger.rb:23:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/warden.rb:50:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/warden.rb:121:in `block in finalize_action' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/warden.rb:50:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/builder.rb:116:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/runner.rb:102:in `block in run' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/util/busy.rb:19:in `busy' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/runner.rb:102:in `run' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/builtin/call.rb:53:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/warden.rb:50:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/builtin/before_trigger.rb:23:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/warden.rb:50:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/builtin/after_trigger.rb:26:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/warden.rb:50:in `call' from /home/secgenadmin/.vagrant.d/gems/2.4.4/gems/vagrant-ovirt4-1.2.2/lib/vagrant-ovirt4/action/connect_ovirt.rb:31:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/warden.rb:50:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/builtin/before_trigger.rb:23:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/warden.rb:50:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/builtin/after_trigger.rb:26:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/warden.rb:50:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/builtin/config_validate.rb:25:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/warden.rb:50:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/builtin/before_trigger.rb:23:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/warden.rb:50:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/builder.rb:116:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/runner.rb:102:in `block in run' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/util/busy.rb:19:in `busy' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/action/runner.rb:102:in `run' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/machine.rb:238:in `action_raw' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/machine.rb:209:in `block in action' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/environment.rb:615:in `lock' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/machine.rb:195:in `call' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/machine.rb:195:in `action' from /opt/vagrant/embedded/gems/2.2.4/gems/vagrant-2.2.4/lib/vagrant/batch_action.rb:82:in `block (2 levels) in run' Non-zero exit status... Not going to destroy escalation, since it does not exist Failed. Not retrying. Please refer to the error above. **Vagrantfile*** VAGRANTFILE_API_VERSION = "2" Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| config.vm.define "escalation" do |escalation| escalation.vm.provider :ovirt4 do |ovirt| ovirt.username = 'admin@internal' ovirt.password = 'my_password' ovirt.url = 'https://engine.cyber-range.lan/ovirt-engine/api' ovirt.cluster = 'default' ovirt.template = 'debian_stretch_server_291118' ovirt.memory_size = '512 MB' ovirt.console = 'spice' ovirt.insecure = true ovirt.debug = true end escalation.vm.provision 'shell', inline: "echo 'datasource_list: [ None ] '> /etc/cloud/cloud.cfg.d/90_dpkg.cfg" escalation.vm.hostname = 'SecGen-default-scenario-escalation' escalation.vm.box = 'ovirt4' escalation.vm.box_url = 'https://github.com/myoung34/vagrant-ovirt4/blob/master/example_box/dummy.bo…' escalation.vm.provision "puppet" do | modules_utilities_unix_update_unix_update | modules_utilities_unix_update_unix_update.module_path = "puppet/escalation/modules" modules_utilities_unix_update_unix_update.environment_path = "environments/" modules_utilities_unix_update_unix_update.environment = "production" modules_utilities_unix_update_unix_update.synced_folder_type = "rsync" modules_utilities_unix_update_unix_update.manifests_path = "puppet/escalation/modules/unix_update" modules_utilities_unix_update_unix_update.manifest_file = "unix_update.pp" end escalation.vm.provision "puppet" do | modules_vulnerabilities_unix_misc_distcc_exec | modules_vulnerabilities_unix_misc_distcc_exec.facter = { "base64_inputs_file" => '94bad9a7b0c9aa1462eb1d4d8277a712', } modules_vulnerabilities_unix_misc_distcc_exec.module_path = "puppet/escalation/modules" modules_vulnerabilities_unix_misc_distcc_exec.environment_path = "environments/" modules_vulnerabilities_unix_misc_distcc_exec.environment = "production" modules_vulnerabilities_unix_misc_distcc_exec.synced_folder_type = "rsync" modules_vulnerabilities_unix_misc_distcc_exec.manifests_path = "puppet/escalation/modules/distcc_exec" modules_vulnerabilities_unix_misc_distcc_exec.manifest_file = "distcc_exec.pp" end escalation.vm.provision "puppet" do | modules_vulnerabilities_unix_local_chkrootkit | modules_vulnerabilities_unix_local_chkrootkit.facter = { "base64_inputs_file" => '170197cbf03410f71d7d777e9d468382', } modules_vulnerabilities_unix_local_chkrootkit.module_path = "puppet/escalation/modules" modules_vulnerabilities_unix_local_chkrootkit.environment_path = "environments/" modules_vulnerabilities_unix_local_chkrootkit.environment = "production" modules_vulnerabilities_unix_local_chkrootkit.synced_folder_type = "rsync" modules_vulnerabilities_unix_local_chkrootkit.manifests_path = "puppet/escalation/modules/chkrootkit" modules_vulnerabilities_unix_local_chkrootkit.manifest_file = "chkrootkit.pp" end escalation.vm.provision "puppet" do | modules_services_unix_nfs_nfs_share | modules_services_unix_nfs_nfs_share.facter = { "base64_inputs_file" => '98ddf2a16a97f46fd4856a509df9b75a', } modules_services_unix_nfs_nfs_share.module_path = "puppet/escalation/modules" modules_services_unix_nfs_nfs_share.environment_path = "environments/" modules_services_unix_nfs_nfs_share.environment = "production" modules_services_unix_nfs_nfs_share.synced_folder_type = "rsync" modules_services_unix_nfs_nfs_share.manifests_path = "puppet/escalation/modules/nfs_share" modules_services_unix_nfs_nfs_share.manifest_file = "nfs_share.pp" end escalation.vm.network :private_network, type: "dhcp", :ovirt__network_name => 'ovirtmgmt' end end

1 0

oVirt 4.2.8 - guest Windows 10 - oVirt tools - no mouse pointer
by Timmi 29 Mar '19

29 Mar '19

Hi guys, I have a problem with some of my Windows 10 VMs. There is no mouse pointer in case I'm connecting to the console through the virt-viewer. I have also other Windows 10 VMs which don't show the same issue. The mouse pointer is working as soon as I uninstall the oVirt tools. Any ideas? Best regards Timmi

1 0

ovirt "discard after delete" storage access request
by Simon Coter 29 Mar '19

29 Mar '19

Hi experts, how can I find a storage that supports “Discard” option for our oVirt storage domains ? I mean, I’ve tried more storage models and I always get: Discard is not supported by underlying storage. This is an example I’ve did on a very old storage (HP MSA): Thanks in advance for your help. Simon

2 1

cpu hotplug cannot be applied for windows 10
by zodaoko＠gmail.com 29 Mar '19

29 Mar '19

Hi there, I am running ovirt engine 4.2.8. I did cpu hotplug for windows 10 guest os, ovirt (from 1*2*1 to 2*2*1) successfully, and the output of virsh vcpuinfo also shows now there are total 4 vcpus: # virsh --readonly vcpuinfo 13 VCPU: 0 CPU: 51 State: running CPU time: 3911.8s CPU Affinity: yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy VCPU: 1 CPU: 15 State: running CPU time: 907.7s CPU Affinity: yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy VCPU: 2 CPU: 26 State: running CPU time: 1086.2s CPU Affinity: yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy VCPU: 3 CPU: 2 State: running CPU time: 1085.7s CPU Affinity: yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy Issue: But when I log into windows 10, from the task manager, there are only two vcpus listed. Then I opened device manager, and found that there are 4 processors listed. refer to https://bugzilla.redhat.com/show_bug.cgi?id=1377155, I have updated os to the latest version 1809. refer to the ovirt doc, cpu hot plug is explicitly supported for windows 10 (both x86 and x86_64) , but from my testing result, the cpu hotplug was not applied. So is this a windows 10 bug? or is there any workaround? Thank you! Thanks, -Zhen

1 2

HE Install cannot login to VM
by Callum Smith 29 Mar '19

29 Mar '19

Dear all, Giving up on running ansible playbook remotely we're hitting earlier issues running hosted-engine --deploy 2019-03-28 15:42:33,975+0000 INFO otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:109 TASK [ovirt.hosted_engine_setup : Wait for the local VM] 2019-03-28 15:45:40,739+0000 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:103 {u'msg': u"timed out waiting for ping module test success: Using a SSH password instead of a key is not possible because Host Key checking is enabled and sshpass does not support this. Please add this host's fingerprint to your known_hosts file to manage this host.", u'elapsed': 186, u'changed': False, u'_ansible_no_log': False, u'_ansible_delegated_vars': {u'ansible_delegated_host': u'he.virt.in.bmrc.ox.ac.uk', u'ansibl e_host': u'he.virt.in.bmrc.ox.ac.uk'}} 2019-03-28 15:45:40,840+0000 ERROR otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:107 fatal: [localhost -> he.virt.in.bmrc.ox.ac.uk]: FAILED! => {"changed": false, "elapsed": 186, "msg": "timed out waiting for ping module test success: Using a SSH password instead of a key is not possible because Host Key checking is enabled and sshpass does not support this. Please add this host's fingerprint to your known_hosts file to manage this host."} At a bit of a loss, especially since the old no-ansible install is not available anymore? Regards, Callum -- Callum Smith Research Computing Core Wellcome Trust Centre for Human Genetics University of Oxford e. callum(a)well.ox.ac.uk<mailto:callum@well.ox.ac.uk>

2 2

CLUSTER_CANNOT_DISABLE_GLUSTER_WHEN_CLUSTER_CONTAINS_VOLUMES
by Arsène Gschwind 29 Mar '19

29 Mar '19

Hi, I updated ou oVirt cluster to 4.3.2 and when i try to update the cluster version to 4.3 i get the following error: CLUSTER_CANNOT_DISABLE_GLUSTER_WHEN_CLUSTER_CONTAINS_VOLUMES So far I remember I could update the cluster version to 4.2 without having to stop everything. I've searched around about this error but couldn't find anything. The engine log says: 2019-03-20 14:33:28,125+01 INFO [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-59) [c93c7f4f-b9a3-4e10-82bf-f8bbda46cc87] Lock Acquired to object 'EngineLock:{exclusiveLocks='[25682477-0bd2-4303-a5fd-0ae9adfd276c=TEMPLATE]', sharedLocks='[119fad69-b4a2-441f-9056-354cd1b8a7aa=VM, 1f696eca-c0a0-48ff-8aa9-b977345b5618=VM, 95ee485d-7992-45b8-b1be-256223b5a89f=VM, 432ed647-c150-425e-aac7-3cb5410f4bc8=VM, 7c2646b1-8a4c-4618-b3c9-dfe563f29e00=VM, 629e40c0-e83b-47e0-82bc-df42ec310ca4=VM, 2e0741bd-33e2-4a8e-9624-a9b7bb70b664=VM, 136d0ca0-478c-48d9-9af4-530f98ac30fd=VM, dafaa8d2-c60c-44c8-b9a3-2b5f80f5aee3=VM, 2112902c-ed42-4fbb-a187-167a5f5a446c=VM, 5330b948-f0cd-4b2f-b722-28918f59c5ca=VM, 3f06be8c-8af9-45e2-91bc-9946315192bf=VM, 8cf338bf-8c94-4db4-b271-a85dbc5d6996=VM, c0be6ae6-3d25-4a99-b93d-81b4ecd7c9d7=VM, 8f4acfc6-3bb1-4863-bf97-d7924641b394=VM]'}' 2019-03-20 14:33:28,214+01 WARN [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-59) [c93c7f4f-b9a3-4e10-82bf-f8bbda46cc87] Validation of action 'UpdateCluster' failed for user xxxx@yyy. Reasons: VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,CLUSTER_CANNOT_DISABLE_GLUSTER_WHEN_CLUSTER_CONTAINS_VOLUMES 2019-03-20 14:33:28,215+01 INFO [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-59) [c93c7f4f-b9a3-4e10-82bf-f8bbda46cc87] Lock freed to object 'EngineLock:{exclusiveLocks='[25682477-0bd2-4303-a5fd-0ae9adfd276c=TEMPLATE]', sharedLocks='[119fad69-b4a2-441f-9056-354cd1b8a7aa=VM, 1f696eca-c0a0-48ff-8aa9-b977345b5618=VM, 95ee485d-7992-45b8-b1be-256223b5a89f=VM, 432ed647-c150-425e-aac7-3cb5410f4bc8=VM, 7c2646b1-8a4c-4618-b3c9-dfe563f29e00=VM, 629e40c0-e83b-47e0-82bc-df42ec310ca4=VM, 2e0741bd-33e2-4a8e-9624-a9b7bb70b664=VM, 136d0ca0-478c-48d9-9af4-530f98ac30fd=VM, dafaa8d2-c60c-44c8-b9a3-2b5f80f5aee3=VM, 2112902c-ed42-4fbb-a187-167a5f5a446c=VM, 5330b948-f0cd-4b2f-b722-28918f59c5ca=VM, 3f06be8c-8af9-45e2-91bc-9946315192bf=VM, 8cf338bf-8c94-4db4-b271-a85dbc5d6996=VM, c0be6ae6-3d25-4a99-b93d-81b4ecd7c9d7=VM, 8f4acfc6-3bb1-4863-bf97-d7924641b394=VM]'}' Any Idea for the reason of this error? Thanks

3 13

Migrate HE beetwen hosts failed.
by kiv＠intercom.pro 29 Mar '19

29 Mar '19

Hi all. I have oVirt 4.3.1 and 3 node hosts. All VM migrate beetwen all hosts successfully. VM with HE - does not migrate. vdsm.log: libvirtError: operation failed: guest CPU doesn't match specification: missing features: xsave,avx,xsaveopt Nodes: 1. Intel Westmere IBRS SSBD Family 2. Intel Westmere IBRS SSBD Family 3. Intel SandyBridge IBRS SSBD Family <----- HE now here Cluster CPU Type: Intel Westmere Family Info from VM with HE: Guest CPU Type: Intel Westmere Family Does anyone know what needs to be done to make migration work?

3 6

virsh edit vm not saving after reboot
by Dev Ops 29 Mar '19

29 Mar '19

I am trying to get Cisco ISE installed into Ovirt. I was told I needed to do a virsh edit on the vm and change a line in the smbios settings. This is all simple enough. After a reboot or shutdown/restart the settings revert back to what they were before. A dump of the XML in fact shows the original settings that I had changed. Since I can't edit the file while the VM is off I am not sure how to save this config change. My issue is similar to this: https://access.redhat.com/discussions/2720821 I have been reading and digging around with no luck. I just can't seem to make this change static. Thanks!

2 2

[ANN] oVirt 4.3.3 First Release Candidate is now available
by Sandro Bonazzola 28 Mar '19

28 Mar '19

The oVirt Project is pleased to announce the availability of the oVirt 4.3.3 First Release Candidate, as of March 28th, 2019. This update is a release candidate of the third in a series of stabilization updates to the 4.3 series. This is pre-release software. This pre-release should not to be used in production. This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 7.6 or later * CentOS Linux (or similar) 7.6 or later This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 7.6 or later * CentOS Linux (or similar) 7.6 or later * oVirt Node 4.3 (available for x86_64 only) Experimental tech preview for x86_64 and s390x architectures for Fedora 28 is also included. See the release notes [1] for installation / upgrade instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Node is already available[2] Additional Resources: * Read more about the oVirt 4.3.3 release highlights: http://www.ovirt.org/release/4.3.3/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.3.3/ [2] http://resources.ovirt.org/pub/ovirt-4.3-pre/iso/ -- SANDRO BONAZZOLA MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://red.ht/sig>

1 0

Re: Gluster VM image Resync Time
by Strahil 28 Mar '19

28 Mar '19

Hi Krutika, I have noticed some performance penalties (10%-15%) when using sharing in v3.12 . What is the situation now with 5.5 ? Best Regards, Strahil NikolovOn Mar 28, 2019 08:56, Krutika Dhananjay <kdhananj(a)redhat.com> wrote: > > Right. So Gluster stores what are called "indices" for each modified file (or shard) > under a special hidden directory of the "good" bricks at $BRICK_PATH/.glusterfs/indices/xattrop. > When the offline brick comes back up, the file corresponding to each index is healed, and then the index deleted > to mark the fact that the file has been healed. > > You can try this and see it for yourself. Just create a 1x3 plain replicate volume, and enable shard on it. > Create a big file (big enough to have multiple shards). Check that the shards are created under $BRICK_PATH/.shard. > Now kill a brick. Modify a small portion of the file. Hit `ls` on $BRICK_PATH/.glusterfs/indices/xattrop of the online bricks. > You'll notice there will be entries named after the gfid (unique identifier in gluster for each file) of the shards. > And only for those shards that the write modified, and not ALL shards of this really big file. > And then when you bring the brick back up using `gluster volume start $VOL force`, the > shards get healed and the directory eventually becomes empty. > > -Krutika > > > On Thu, Mar 28, 2019 at 12:14 PM Indivar Nair <indivar.nair(a)techterra.in> wrote: >> >> Hi Krutika, >> >> So how does the Gluster node know which shards were modified after it went down? >> Do the other Gluster nodes keep track of it? >> >> Regards, >> >> >> Indivar Nair >> >> >> On Thu, Mar 28, 2019 at 9:45 AM Krutika Dhananjay <kdhananj(a)redhat.com> wrote: >>> >>> Each shard is a separate file of size equal to value of "features.shard-block-size". >>> So when a brick/node was down, only those shards belonging to the VM that were modified will be sync'd later when the brick's back up. >>> Does that answer your question? >>> >>> -Krutika >>> >>> On Wed, Mar 27, 2019 at 7:48 PM Sahina Bose <sabose(a)redhat.com> wrote: >>>> >>>> On Wed, Mar 27, 2019 at 7:40 PM Indivar Nair <indivar.nair(a)techterra.in> wrote: >>>> > >>>> > Hi Strahil, >>>> > >>>> > Ok. Looks like sharding should make the resyncs faster. >>>> > >>>> > I searched for more info on it, but couldn't find much. >>>> > I believe it will still have to compare each shard to determine whether there are any changes that need to be replicated. >>>> > Am I right? >>>> >>>> +Krutika Dhananjay >>>> > >>>> > Regards, >>>> > >>>> > Indivar Nair >>>> > >>>> > >>>> > >>>> > On Wed, Mar 27, 2019 at 4:34 PM Strahil <hunter86_bg(a)yahoo.com> wrote: >>>> >> >>>> >> By default ovirt uses 'sharding' which splits the files into logical chunks. This greatly reduces healing time, as VM's disk is not always completely overwritten and only the shards that are different will be healed. >>>> >> >>>> >> Maybe you should change the default shard size. >>>> >> >>>> >> Best Regards, >>>> >> Strahil Nikolov >>>> >> >>>> >> On Mar 27, 2019 08:24, Indivar Nair <indivar.nair(a)techterra.in> wrote: >>>> >> >>>> >> Hi All, >>>> >> >>>> >> We are planning a 2 + 1 arbitrated mirrored Gluster setup. >>>> >> We would have around 50 - 60 VMs, with an average 500GB disk size. >>>> >> >>>> >> Now in case one of the Gluster Nodes go completely out of sync, roughly, how long would it take to resync? (as per your experience) >>>> >> Will it impact the working of VMs in any way? >>>> >> Is there anything to be taken care of, in advance, to prepare for such a situation? >>>> >> >>>> >> Regards, >>>> >> >>>> >> >>>> >> Indivar Nair >>>> >> >>>> > ______________

2 2

Re: ovirt data domain with multi luns, any best practice
by Strahil 28 Mar '19

28 Mar '19

In my opinion it's best to have 2 data domains. Best Regards, Strahil NikolovOn Mar 28, 2019 05:32, adam_xu(a)adagene.com.cn wrote: > > Hello, everyone. I had a question on how to create a data domain in the best way. > for example, I have a FCP device and there's a lot of hdds and ssds in that device. I ceate 2 luns. one consisting of HDDs and the other consisting of SSDs. I want to make some normal vms run in the lun with hdd and some vms (such as DB) run in the lun with ssd. > so, shoud I create one data domain with the 2 luns. or shoud I create 2 difference data domains, and every data domain with a single lun in it? > thanks. > ________________________________ > yours Adam

1 0

ovirt data domain with multi luns, any best practice
by adam_xu＠adagene.com.cn 28 Mar '19

28 Mar '19

Hello, everyone. I had a question on how to create a data domain in the best way. for example, I have a FCP device and there's a lot of hdds and ssds in that device. I ceate 2 luns. one consisting of HDDs and the other consisting of SSDs. I want to make some normal vms run in the lun with hdd and some vms (such as DB) run in the lun with ssd. so, shoud I create one data domain with the 2 luns. or shoud I create 2 difference data domains, and every data domain with a single lun in it? thanks. yours Adam

1 0

User permissions needed to clone template disk
by Wood Peter 27 Mar '19

27 Mar '19

Hi, Users have PowerUserRole permissions on the cluster and the storage objects. Also TemplateCreator role on the Datacenter. When users create VMs from templates there is no option to clone the disk and create independent VM disk. The resource allocation section is not visible at all. What permissions should I give users so they can clone the disk when creating a VM from a template? Using oVirt 4.2.8.2-1.el7 Thank you, -- Peter

2 4

Bad Volume Specification
by Bryan Sockel 27 Mar '19

27 Mar '19

Having an issue with starting one of my VM's. Looks like there was a problem with an auto generated snapshot on the VM. The snapshot file is missing: VDSM-Tool Output: image: 52f31c3a-25ac-4930-8e67-105945dd42b5 - 461e1747-3a59-4abc-ac6a-2013ec7858d3 status: OK, voltype: INTERNAL, format: RAW, legality: LEGAL, type: PREALLOCATED, capacity: 53687091200, truesize: 53687091200 - fe5fb4a4-8b3a-4bad-8f03-9dd179ac8397 status: OK, voltype: INTERNAL, format: COW, legality: LEGAL, type: SPARSE, capacity: 53687091200, truesize: 6576668672 - a01d662f-cf33-4c86-b0ba-8bddc6d25dfd status: ILLEGAL, voltype: LEAF, format: COW, legality: ILLEGAL, type: SPARSE, capacity: 53687091200, truesize: 3221225472 lvs -o+lv_tags |grep 52f31c3a-25ac-4930-8e67-105945dd42b5 Couldn't find device with uuid 5UNXpR-SmSR-KqA2-swpe-kcQi-X7tt-7B5wRz. 461e1747-3a59-4abc-ac6a-2013ec7858d3 8dfaac42-32fc-4e23-b5cb-e5c2c3c56c9d -wi-a----- 50.00g IU_52f31c3a-25ac-4930-8e67-105945dd42b5,MD_43,PU_00000000-0000-0000-0000-000000000000 a01d662f-cf33-4c86-b0ba-8bddc6d25dfd 8dfaac42-32fc-4e23-b5cb-e5c2c3c56c9d -wi------- 3.00g IU_52f31c3a-25ac-4930-8e67-105945dd42b5,MD_46,PU_fe5fb4a4-8b3a-4bad-8f03-9dd179ac8397 fe5fb4a4-8b3a-4bad-8f03-9dd179ac8397 8dfaac42-32fc-4e23-b5cb-e5c2c3c56c9d -wi-a----- 6.12g IU_52f31c3a-25ac-4930-8e67-105945dd42b5,MD_45,PU_461e1747-3a59-4abc-ac6a-2013ec7858d3 qemu-img info /dev/8dfaac42-32fc-4e23-b5cb-e5c2c3c56c9d/a01d662f-cf33-4c86-b0ba-8bddc6d25dfd qemu-img: Could not open '/dev/8dfaac42-32fc-4e23-b5cb-e5c2c3c56c9d/a01d662f-cf33-4c86-b0ba-8bddc6d25dfd': Could not open '/dev/8dfaac42-32fc-4e23-b5cb-e5c2c3c56c9d/a01d662f-cf33-4c86-b0ba-8bddc6d25dfd': No such file or directory What is the best way to remove the snapshot from the vm (PSQL statement to delete the entry from the database)? Thank You, Bryan Sockel

1 0

VM disk corruption with LSM on Gluster
by Sander Hoentjen 27 Mar '19

27 Mar '19

Hello, tl;dr We have disk corruption when doing live storage migration on oVirt 4.2 with gluster 3.12.15. Any idea why? We have a 3-node oVirt cluster that is both compute and gluster-storage. The manager runs on separate hardware. We are running out of space on this volume, so we added another Gluster volume that is bigger, put a storage domain on it and then we migrated VM's to it with LSM. After some time, we noticed that (some of) the migrated VM's had corrupted filesystems. After moving everything back with export-import to the old domain where possible, and recovering from backups where needed we set off to investigate this issue. We are now at the point where we can reproduce this issue within a day. What we have found so far: 1) The corruption occurs at the very end of the replication step, most probably between START and FINISH of diskReplicateFinish, before the START merge step 2) In the corrupted VM, at some place where data should be, this data is replaced by zero's. This can be file-contents or a directory-structure or whatever. 3) The source gluster volume has different settings then the destination (Mostly because the defaults were different at creation time): Setting old(src) new(dst) cluster.op-version 30800 30800 (the same) cluster.max-op-version 31202 31202 (the same) cluster.metadata-self-heal off on cluster.data-self-heal off on cluster.entry-self-heal off on performance.low-prio-threads 16 32 performance.strict-o-direct off on network.ping-timeout 42 30 network.remote-dio enable off transport.address-family - inet performance.stat-prefetch off on features.shard-block-size 512MB 64MB cluster.shd-max-threads 1 8 cluster.shd-wait-qlength 1024 10000 cluster.locking-scheme full granular cluster.granular-entry-heal no enable 4) To test, we migrate some VM's back and forth. The corruption does not occur every time. To this point it only occurs from old to new, but we don't have enough data-points to be sure about that. Anybody an idea what is causing the corruption? Is this the best list to ask, or should I ask on a Gluster list? I am not sure if this is oVirt specific or Gluster specific though. Kind regards, Sander Hoentjen

4 6

Thin clients connect to a virtual machine using spice and require a password
by xilazz＠126.com 27 Mar '19

27 Mar '19

Hello everyone, when I want to use the thin client to connect the virtual machine, I will use spice mode to connect, but I need to specify the port of the host and spice password. Could you please tell me where I can check the spice password of the virtual machine

1 0

ETL DWH errors after restoring to a new server
by Staniforth, Paul 27 Mar '19

27 Mar '19

Hello, I have deployed a new machine to run our engine 4.2.8 using engine-backup to do a full backup/restore to a clean Centos install. We are now having a lot of errors with ovirt-engine-dwhd e.g. java.lang.OutOfMemoryError: Java heap space java.lang.OutOfMemoryError: GC overhead limit exceeded org.postgresql.util.PSQLException: An I/O error occured while sending to the backend. We had a file from 2015 /etc/ovirt-engine/engine.conf.d/10-setup-java.conf ENGINE_HEAP_MIN="5845M" ENGINE_HEAP_MAX="5845M" I changed it to ENGINE_HEAP_MIN=2g ENGINE_HEAP_MAX=4g and restarted the engine and dwhd But am still getting errors. Regards, Paul S. To view the terms under which this email is distributed, please go to:- http://leedsbeckett.ac.uk/disclaimer/email/

1 1

Gluster VM image Resync Time
by Indivar Nair 27 Mar '19

27 Mar '19

Hi All, We are planning a 2 + 1 arbitrated mirrored Gluster setup. We would have around 50 - 60 VMs, with an average 500GB disk size. Now in case one of the Gluster Nodes go completely out of sync, roughly, how long would it take to resync? (as per your experience) Will it impact the working of VMs in any way? Is there anything to be taken care of, in advance, to prepare for such a situation? Regards, Indivar Nair

1 0

oVirt Roles and Permissions
by Wood Peter 27 Mar '19

27 Mar '19

Hi, I'm trying to setup oVirt 4.2.8.2-1.el7 based on a few simple requirements: Users should be able to: 1) reate new VMs 2) create VMs from templates 2a) Ability to clone the template disk to create a VM independent of the template 3) create templates 4) view only the VMs they have permissions to work with i.e. UserVmManager role assign to them This was all possible through the "User Portal" in version 3 (3.6 is what I had). The new "VM Portal" doesn't allow users to create templates. In order to satisfy (3) users must login to the Admin Portal. This is OK but I created a new Admin role with just login permissions and users can see all VMs in the cluster. I assume a filter can be created but in general I don't want them to have that much visibility. I can probably get away without (3) but I still can figure out how to do (2a) using "VM Portal". All VMs created from a template are dependent on that template. Even if the template is optimized for Server. How do I accomplish this? Thank you in advance for any suggestions. -- Peter

1 0

Can't manually migrate VM's (4.3.0)
by Ron Jerome 26 Mar '19

26 Mar '19

I've been running 4.3.0 for a few weeks now and just discovered that I can't manually migrate VM's from the UI. I get an error message saying: "Could not fetch data needed for VM migrate operation" Sounds like https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=1670701 Ron.

5 16

Re: Using NFS on Gluster
by Strahil 26 Mar '19

26 Mar '19

Hi Indivar, You can do that , but is complicated. First, I would recommend you to use either replica 3 or replica 3 arbiter 1 cluster volumes. Second , you have 2 options - to use NFS provided by Cluster Cluster ,but this is deprecated and you will have to migrate from that option once it get's completely removed. Third option is to use CTDB to fail over your floating VIP and to start NFS-Ganesha. The benefit of using NFS-Ganesha is that it uses libgfapi and connects to all cluster nodes simultaneously providing accumulated performance. Best Regards, Strahil Nikolov On Mar 26, 2019 16:42, Indivar Nair <indivar.nair(a)techterra.in> wrote: > > > > > Hi All, > > > > We would like to set up a 2 node mirrored Gluster Storage using the ovirt installation tools, but use it as an NFS storage. > > > > We want to do this for 2 reasons - > > > > 1. We have a high-speed link between both the Gluster Nodes which we would like to use for brick replication rather than ovirt node's 1G link > > 2. We have fast SSD disks on the ovirt nodes which we would like to use for local NFS Caching (FS Caching) on the VM Images. > > > > In short, we would like to manage Gluster stroage using the ovirt web interface, but access the Gluster storage as an NFS Server. > > > > How can we do this? > > > > Thanks. > > > > Regards, > > > > > > Indivar Nair > >

2 1

ovn-provider-network
by Staniforth, Paul 26 Mar '19

26 Mar '19

Hello, we are using oVirt-4.2.8 and I have created a logical network using the ovn-network-provider, I haven't configured it to connect to a physical network. I have 2 VMs running on 2 hosts which can connect to each other this logical network. The only connection between the hosts is over the ovirtmgmt network so presumably the traffic is using this? Thanks, Paul S. To view the terms under which this email is distributed, please go to:- http://leedsbeckett.ac.uk/disclaimer/email/

4 13

Re: Changing Ovirt hosted engine memory
by Strahil 26 Mar '19

26 Mar '19

What is the output of 'free -h' on the engine? Doesn't it show the new memory size? You need to wait some time so the OVF get's updated. Best Regards, Strahil NikolovOn Mar 26, 2019 18:11, Dionysis K <dkstiler(a)gmail.com> wrote: > > Thanks you bust still not any resolution for some reason, it does not update the memory of the Hosted engine, while its on global maintenance, whatever i do either it will stay on the same number for example 5120 MiB either the above message > > The physical node that runs the Hosted Engine VM has it has 24 Gigs or ram installed, and availiable are 22 It reports on the gui > > Please any help would be appreciated > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/UAKOQ5FKHOY4Q…

1 0

Using NFS on Gluster
by Indivar Nair 26 Mar '19

26 Mar '19

Hi All, We would like to set up a 2 node mirrored Gluster Storage using the ovirt installation tools, but use it as an NFS storage. We want to do this for 2 reasons - 1. We have a high-speed link between both the Gluster Nodes which we would like to use for brick replication rather than ovirt node's 1G link 2. We have fast SSD disks on the ovirt nodes which we would like to use for local NFS Caching (FS Caching) on the VM Images. In short, we would like to manage Gluster stroage using the ovirt web interface, but access the Gluster storage as an NFS Server. How can we do this? Thanks. Regards, Indivar Nair

2 4

Re: OVirt Gluster Fail
by Strahil 26 Mar '19

26 Mar '19

Hi Andrea, My guess is that while node2 was in maintenance , node3 brick(s) have died, or there were some pending heals. For backup, you can use anything that works for KVM, but the hard part is to get the configuration of each VM. If the VM is running, you can use 'virsh dumpxml domain' to get the configuration of the running VM, but this won't work for VM that are off. Why firewalld was not stopped - my guess is a rare bug that is hard to reproduce. Best Regards, Strahil Nikolov On Mar 26, 2019 17:10, Andrea Milan <commramius(a)tiscali.it> wrote: > > Hi Sahina, Strahil > > > > thank you for the information, I managed to start the heal and restore both the hosted engine and the Vms. > > > > This is the log’s on all nodes > > > > [2019-03-26 08:30:58.462329] I [MSGID: 104045] [glfs-master.c:91:notify] 0-gfapi: New graph 676c6e6f-6465-3032-2e61-736370642e6c (0) coming up > > [2019-03-26 08:30:58.462364] I [MSGID: 114020] [client.c:2356:notify] 0-asc-client-0: parent translators are ready, attempting connect on transport > > [2019-03-26 08:30:58.464374] I [MSGID: 114020] [client.c:2356:notify] 0-asc-client-1: parent translators are ready, attempting connect on transport > > [2019-03-26 08:30:58.464898] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 0-asc-client-0: changing port to 49438 (from 0) > > [2019-03-26 08:30:58.466148] I [MSGID: 114020] [client.c:2356:notify] 0-asc-client-3: parent translators are ready, attempting connect on transport > > [2019-03-26 08:30:58.468028] E [socket.c:2309:socket_connect_finish] 0-asc-client-0: connection to 192.170.254.3:49438 failed (Nessun instradamento per l'host) > > [2019-03-26 08:30:58.468054] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 0-asc-client-1: changing port to 49441 (from 0) > > [2019-03-26 08:30:58.470040] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 0-asc-client-3: changing port to 49421 (from 0) > > [2019-03-26 08:30:58.471345] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 0-asc-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330) > > [2019-03-26 08:30:58.472642] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 0-asc-client-1: Connected to asc-client-1, attached to remote volume '/bricks/asc/brick'. > > [2019-03-26 08:30:58.472659] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 0-asc-client-1: Server and Client lk-version numbers are not same, reopening the fds > > [2019-03-26 08:30:58.472714] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 0-asc-replicate-0: Subvolume 'asc-client-1' came back up; going online. > > [2019-03-26 08:30:58.472731] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-asc-client-1: Server lk version = 1 > > [2019-03-26 08:30:58.473112] E [socket.c:2309:socket_connect_finish] 0-asc-client-3: connection to 192.170.254.6:49421 failed (Nessun instradamento per l'host) > > [2019-03-26 08:30:58.473152] W [MSGID: 108001] [afr-common.c:4467:afr_notify] 0-asc-replicate-0: Client-quorum is not met > > [2019-03-26 08:30:58.477699] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 0-asc-replicate-0: selecting local read_child asc-client-1 > > [2019-03-26 08:30:58.477804] I [MSGID: 104041] [glfs-resolve.c:885:__glfs_active_subvol] 0-asc: switched to graph 676c6e6f-6465-3032-2e61-736370642e6c (0) > > > > > > I analyzed the single nodes and I realized that the firewalld service has been stopped on all nodes. > > Firewalld re-enabled the heal started automatically, and the “gluster heal volume VOLNAME info” immediately gave correct connections. > > > > the recovery of the single bricks started immediatly. > > When finished that I have correctly detected and start the host-engine. > > > > I wanted to tell you about the sequence that led me to the block: > > > > 1) Node03 in maintenance by hosted-engine. > > 2) Maintenance performed and restarted. > > 3) Repositioned active node03 > > 4) Heal automatic controlled with Ovirt Manager. > > 5) Heal completed correctly. > > 6) Node02 put into maintenance. > > 7) During the shutdown of the Node02 some VMs have gone to Pause, Ovirt Manager has signaled the block of the Node01 and immediately the host-engine has stopped. > > 8) Restarted the Node02, I saw that the gluster had the peer but there was no healing between the nodes. > > I had to close everything, and the situation that presented itself was that of previous emails. > > > > Questions: > > - Why did the Node02 in maintenance block Node01? > > - Why was restarting the system not restarting the firewalld service? Is it also managed by vdsm? > > - What is the correct way to backup virtual machines on an external machine? We use Ovirt4.1 > > - can backup be used outside of Ovirt? Es qemu-kvm standard ... > > Thanks for all. > Best regards > Andrea Milan > > Il 25.03.2019 11:53 Sahina Bose ha scritto: >> >> You will first need to restore connectivity between the gluster peers >> >> for heal to work. So restart glusterd on all hosts as Strahil >> >> mentioned, and check if "gluster peer status" returns the other nodes >> >> as connected. If not, please check the glusterd log to see what's >> >> causing the issue. Share the logs if we need to look at it, along with >> >> the version info >> >> >> >> >> >> On Sun, Mar 24, 2019 at 1:08 AM Strahil <hunter86_bg(a)yahoo.com> wrote: >>> >>> Hi Andrea, The cluster volumes might have sharding enabled and thus files larger than shard size can be recovered only via cluster. You can try to restart gluster on all nodes and force heal: 1. Kill gluster processes: systemctl stop glusterd /usr/share/glusterfs/scripts/stop-all-gluster-processes.sh 2. Start gluster: systemctl start glusterd 3. Force heal: for i in $(gluster volume list); do gluster volume heal $i full ; done sleep 300 for i in $(gluster volume list); do gluster volume heal $i info summary ; done Best Regards, Strahil Nikolov On Mar 23, 2019 13:51, commramius(a)tiscali.it wrote: > > During maintenance of a machine the hosted engine crashed. > At that point there was no more chance of managing anything. > > The VMs have paused, and were no longer manageable. > I restarted the machine, but one point all the bricks were no longer reachable. > > Now I am in a situation where the engine support is no longer loaded. > > The gluster sees the peers connected and the services turned on for the various bricks, but fails to heal the messages that I find for each machine are the following > > # gluster volume heal engine info > Brick 192.170.254.3:/bricks/engine/brick > > . > . > . > > Status: Connected Number of entries: 190 > > Brick 192.170.254.4:/bricks/engine/brick > Status: Il socket di destinazione non è connesso > Number of entries: - > > Brick 192.170.254.6:/bricks/engine/brick > Status: Il socket di destinazione non è connesso > Number of entries: - > > this for all the bricks (some have no heal to do because the machines inside were turned off). > > In practice all the bricks see only localhost as connected. > > How can I restore the machines? > Is there a way to read data from the physical machine and export it so that it can be reused? > Unfortunately we need to access that data. > > Someone can help me. > > Thanks Andrea > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/EOIY7ZU4GOEMR… _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/NMBDYBOY4TZB3… > > > > > Con OpenStar hai Giga, SMS e i minuti che vuoi da 4,99€ al mese, per sempre. Cambi gratis quando e come vuoi e in più hai 6 mesi di INFINTY! http://tisca.li/myopen >

1 0