- Users - oVirt List Archives

2-node with FC Storage
by carsten.recker＠ccf.de 16 Sep '21

16 Sep '21

Hello, i want to run an 2-node oVirt cluster, where both nodes are connected to an FC Storage. What is the best practice to connect LUNs to it, so that you can switch VMs between both nodes? Thanks

3 2

host upgrade problems galore
by Diggy Mc 15 Sep '21

15 Sep '21

Brief background: Simple environment -- self-hosted engine and two oVirt Node hosts all running 4.4.4. Couple NFS-based data storage domains and a few VLAN-based networks. Upgrading the Hosted Engine from 4.4.4 to 4.4.8 appears to have completed successfully and without incident. I followed the published instructions. :) Problems upgrading the oVirt Node hosts !!! Initialized cluster upgrade. It got as far as rebooting the first host. ERRORS. Said "vdsmd... BAD" Found journal log entry suggesting I run a vdsm config command. Did that and rebooted. VDSM error went away and all looked good. Unfortunately VMs started on that host now were clearly unstable. Ugh! Unusable host. Removed that host from the cluster. Reinstalled from scratch using the 4.4.8 ISO. All went well. Added it to the cluster and deployed the HE to it. No Errors. Unfortunately, it is now stuck in a NonOperational state and I cannot determine why. Please help.

2 2

oVirt 4.3 DWH with Grafana
by Vrgotic, Marko 14 Sep '21

14 Sep '21

Dear oVirt, We are currently running oVirt 4.3 and upgrade/migration to 4.4 won’t be possible for few more months. I am looking into guidelines, how to, for setting up Grafana using DataWarehouse as data source. Did anyone already did this, and would be willing to share the steps? Kindly awaiting your reply. ----- kind regards/met vriendelijke groeten Marko Vrgotic Sr. System Engineer @ System Administration ActiveVideo o: +31 (35) 6774131 m: +31 (65) 5734174 e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com> w: www.activevideo.com<http://www.activevideo.com> ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.

8 11

HE and Node upgrade procedures
by Diggy Mc 14 Sep '21

14 Sep '21

Where can I find the documentation on the procedure to upgrade the Hosted Engine as well as the oVirt Nodes? Specifically, both the HE and Nodes are currently at 4.4.4 and I would like to upgrade to 4.4.8.3. Also, if I remember correctly, 4.4.4 was the last version to use CentOS Linux 8 and later versions switched to CentOS Stream 8. Will the upgrade procedure automatically switch over to Stream? Is there anything I need to be aware of or concerned with regarding the switch to Stream? Your help and guidance is much appreciated.

2 2

what kind of managed block can oVirt manage ?
by Tommy Sway 14 Sep '21

14 Sep '21

On the create disk page, there is option to create disk that is from managed block, I want wo know what's kind of it ? Ceph block device ? Thanks!

2 17

virtio-win driver licensing
by simon＠justconnect.ie 14 Sep '21

14 Sep '21

Hi, Can someone clarify the licensing around virtio-win drivers on oVirt 4.4.8 as the following seems to imply a RedHat paid subscrition is required when installing "virtio-win-guest-tools.exe" "2. Program License Grant. Red Hat grants User a non-exclusive, non-transferable license to use the Program during the term of and on systems with fully-paid applicable Red Hat subscription(s) provided User may not: (a) modify, copy, or create any derivative works of the Program; (b) decompile, disassemble or reverse engineer the Program (except to the extent permitted by applicable law); (c) redistribute, encumber, sell, rent, lease, sublicense, or otherwise transfer rights to the Program (except to the extent permitted herein); or (d) remove or alter any trademark, logo, copyright or other proprietary notices, legends, symbols or labels in the Program. Upon expiration of the term set forth above, User will promptly remove, delete or destroy all copies of the Program in its possession." Kind Regards Simon...

5 9

Thank you for restreaming the oVirt Conf to China!
by Janos Bonic 14 Sep '21

14 Sep '21

Hey folks, I'm hoping whoever mentioned in the chat that they are rebroadcasting the conference to China is in this mailing list. I owe you a huge thank you! I tried to figure out how to stream to China, but I failed. If you are interested in helping us set up a direct stream or publish to a platform accessible in China next year, please contact me privately so we can coordinate. Janos

1 0

oVirt Monitoring Alerts via Grafana
by Aviv Litman 14 Sep '21

14 Sep '21

Hi all, Hope you all are doing well. Checkout this new oVirt blog: oVirt Monitoring Alerts via Grafana <https://blogs.ovirt.org/2021/08/ovirt-monitoring-alerts-via-grafana/>. The blog explains how to configure alerts in Grafana for your oVirt environment and provides an example alerts dashboard that you can import, use and edit to your needs. When using alerts significant or critical data changes can be immediately recognized, so don't miss this opportunity to learn how to configure and use this important tool. Feedback, comments and suggestions are more than welcome! -- Aviv Litman BI Associate Software Engineer Red Hat <https://www.redhat.com/> alitman(a)redhat.com <https://www.redhat.com/>

3 3

Firewalld problem
by Jorge Visentini 13 Sep '21

13 Sep '21

Hi all. Anyone with the problem below? *firewalld[29746]: ERROR: COMMAND_FAILED: 'python-nftables' failed: internal:0:0-0: Error: No such file or directory internal:0:0-0: Error: No such file or directory internal:0:0-0: Error: No such file or directory internal:0:0-0: Error: No such file or directory internal:0:0-0: Error: No such file or directory internal:0:0-0: Error: No such file or directory internal:0:0-0: Error: No such file or directory internal:0:0-0: Error: No such file or directory* I'm using ovirt 4.4.8.3. The python-nftables package doesn't exist on the repository, just the python3-nftables package... Thanks!! -- Att, Jorge Visentini +55 55 98432-9868

1 0

about the cloud-init options on the vm portal
by Tommy Sway 13 Sep '21

13 Sep '21

It seems that the options for cloud-init in the VM portal interface are only host name and SSH key, other options that can be seen in the administrator interface are not visible. Is there a solution to this problem? Thanks!

1 0

Cannot Start VM After Pausing due to Storage I/O Error
by bob.franzke＠mdaemon.com 13 Sep '21

13 Sep '21

Noticed I had a VM that was 'paused' due to a 'Storage I/O error. I inherited this system from another admin and have no idea where to start figuring this out. We have a 4-node Ovirt cluster with a 5 Manager node. The VM in question is running on a host vm-host-colo-4. Best I can tell the VMs run on a gluster replicated volume replicated between all 4 nodes, with node 1 running as an arbiter node for the gluster volume. Other VMs are running on this host 4 so not sure what the issue is with this one VM. When I look at the status of the gluster volume for this host, I see the self-heal info for the bricks is listed as 'N/A' for this host. All the other hosts in the cluster list this info as 'OK'. When I cd into the gluster directory on host 4, I don't see the same things as I do on the other hosts. I am not sure this is an issue but its just different. When running various gluster commands gluster seems to respond. See below: [root@vm-host-colo-4 gluster]# gluster volume info all Volume Name: gl-colo-1 Type: Replicate Volume ID: 2c545e19-9468-487e-9e9b-cd3202fc24c4 Status: Started Snapshot Count: 0 Number of Bricks: 1 x (2 + 1) = 3 Transport-type: tcp Bricks: Brick1: 10.20.101.181:/gluster/gl-colo-1/brick1 Brick2: 10.20.101.183:/gluster/gl-colo-1/brick1 Brick3: 10.20.101.185:/gluster/gl-colo-1/brick1 (arbiter) Options Reconfigured: network.ping-timeout: 30 cluster.granular-entry-heal: enable performance.strict-o-direct: on storage.owner-gid: 36 storage.owner-uid: 36 cluster.choose-local: off features.shard: on cluster.shd-wait-qlength: 10000 cluster.shd-max-threads: 8 cluster.locking-scheme: granular cluster.data-self-heal-algorithm: full cluster.server-quorum-type: server cluster.quorum-type: auto cluster.eager-lock: enable network.remote-dio: off performance.low-prio-threads: 32 performance.io-cache: off performance.read-ahead: off performance.quick-read: off auth.allow: * user.cifs: off transport.address-family: inet nfs.disable: on performance.client-io-threads: off Volume Name: gl-vm-host-4 Type: Distribute Volume ID: a2ba6b29-2366-4a7e-bda8-2e0574cf4afa Status: Started Snapshot Count: 0 Number of Bricks: 1 Transport-type: tcp Bricks: Brick1: 10.20.101.187:/gluster/gl-vm-host-colo-4 Options Reconfigured: network.ping-timeout: 30 cluster.granular-entry-heal: enable network.remote-dio: off performance.strict-o-direct: on storage.owner-gid: 36 storage.owner-uid: 36 auth.allow: * user.cifs: disable transport.address-family: inet nfs.disable: on [root@vm-host-colo-4 gluster]# [root@vm-host-colo-4 gluster]# gluster-eventsapi status Webhooks: http://mydesktop.altn.int:80/ovirt-engine/services/glusterevents +-------------------------+-------------+-----------------------+ | NODE | NODE STATUS | GLUSTEREVENTSD STATUS | +-------------------------+-------------+-----------------------+ | vm-host-colo-1.altn.int | UP | OK | | vm-idev-colo-1.altn.int | UP | OK | | vm-host-colo-2.altn.int | UP | OK | | localhost | UP | OK | +-------------------------+-------------+-----------------------+ [root@vm-host-colo-4 gluster]# gluster volume status gl-vm-host-4 Status of volume: gl-vm-host-4 Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick 10.20.101.187:/gluster/gl-vm-host-col o-4 49152 0 Y 33221 Task Status of Volume gl-vm-host-4 ------------------------------------------------------------------------------ There are no active volume tasks I also get a timeout error when doing a 'gluster volume status' on this node. So while some aspects seem fine with the gluster volume, some don't. Should I restart the glusterd daemon or will that mess things up? I am not sure if this is due to something wrong with the gluster volume or with the vm-host's ability to access the data for the VM disk, meaning a true I/O problem. There are two VMs in this state both running on this host and I am not sure how to proceed to get them running again. Should I force this VM to be on a different host by editing the VM or should I try and make it work on the host its on. As mentioned, many other VMs are running on this host so not sure why these two have an issue. Up front apologies here. I am a network engineer and not a VM/Ovirt expert. This was dropped in my lap due to a layoff and could use some help on where to go from here. Thanks in advance for any help.

5 7

Create template from snapshot of vm using MBS disk
by ssarang520＠gmail.com 13 Sep '21

13 Sep '21

Hi, If I create a template with a snapshot of the vm that uses the mbs disk, the template cannot be used to create a new vm. Is this normal or a bug?

2 6

problems testing 4.3.10 to 4.4.8 upgrade SHE
by Gianluca Cecchi 13 Sep '21

13 Sep '21

Hello, I'm testing what in object in a test env with novirt1 and novirt2 as hosts. First reinstalled host is novirt2 For this I downloaded the 4.4.8 iso of the node: https://resources.ovirt.org/pub/ovirt-4.4/iso/ovirt-node-ng-installer/4.4.8… before running the restore command for the first scratched node I pre-installed the appliance rpm on it and I got: ovirt-engine-appliance-4.4-20210818155544.1.el8.x86_64 I selected to pause an d I arrived here with local vm engine completing its setup: INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Add host] [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Include after_add_host tasks files] [ INFO ] You can now connect to https://novirt2.localdomain.local:6900/ovirt-engine/ and check the status of this host and eventually remediate it, please continue only when the host is listed as 'up' [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : include_tasks] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Create temporary lock file] [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Pause execution until /tmp/ansible.4_o6a2wo_he_setup_lock is removed, delete it once ready to proceed] But connecting t the provided https://novirt2.localdomain.local:6900/ovirt-engine/ url I see that only the still 4.3.10 host results up while novirt2 is not resp[onsive vm situation: https://drive.google.com/file/d/1OwHHzK0owU2HWZqvHFaLLbHVvjnBhRRX/view?usp=… storage situation: https://drive.google.com/file/d/1D-rmlpGsKfRRmYx2avBk_EYCG7XWMXNq/view?usp=… hosts situation: https://drive.google.com/file/d/1yrmfYF6hJFzKaG54Xk0Rhe2kY-TIcUvA/view?usp=… In engine.log I see 2021-08-25 09:14:38,548+02 ERROR [org.ovirt.engine.core.vdsbroker.HostDevListByCapsVDSCommand] (EE-ManagedThreadFactory-engine-Thread-4) [5f4541ee] Command 'HostDevListByCapsVDSCommand(HostName = novirt2.localdomain.local, VdsIdAndVdsVDSCommandParametersBase:{hostId='ca9ff6f7-5a7c-4168-9632-998c52f76cfa', vds='Host[novirt2.localdomain.local,ca9ff6f7-5a7c-4168-9632-998c52f76cfa]'})' execution failed: java.net.ConnectException: Connection refused and continuouslly this message... I also tried to restart vdsmd on novit2 but nothing changed. Do I have to restart the HA daemons on novirt2? Any insight? Thanks Gianluca

3 15

Error when trying to change master storage domain
by Matthew Benstead 12 Sep '21

12 Sep '21

Hello, I'm trying to decommission the old master storage domain in ovirt, and replace it with a new one. All of the VMs have been migrated off of the old master, and everything has been running on the new storage domain for a couple months. But when I try to put the old domain into maintenance mode I get an error. Old Master: vm-storage-ssd New Domain: vm-storage-ssd2 The error is: Failed to Reconstruct Master Domain for Data Center EDC2 As well as: Sync Error on Master Domain between Host daccs01 and oVirt Engine. Domain: vm-storage-ssd is marked as Master in oVirt Engine database but not on the Storage side. Please consult with Support on how to fix this issue. 2021-07-28 11:41:34,870-07 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engine-Thread-23) [] Master domain version is not in sync between DB and VDSM. Domain vm-storage-ssd marked as master, but the version in DB: 283 and in VDSM: 280 And: Not stopping SPM on vds daccs01, pool id f72ec125-69a1-4c1b-a5e1-313fcb70b6ff as there are uncleared tasks Task '5fa9edf0-56c3-40e4-9327-47bf7764d28d', status 'finished' After a couple minutes all the domains are marked as active again and things continue, but vm-storage-ssd is still listed as the master domain. Any thoughts? This is on 4.3.10.4-1.el7 on CentOS 7. engine=# SELECT storage_name, storage_pool_id, storage, status FROM storage_pool_with_storage_domain ORDER BY storage_name; storage_name | storage_pool_id | storage | status -----------------------+--------------------------------------+----------------------------------------+-------- compute1-iscsi-ssd | f72ec125-69a1-4c1b-a5e1-313fcb70b6ff | yvUESE-yWUv-VIWL-qX90-aAq7-gK0I-EqppRL | 1 compute7-iscsi-ssd | f72ec125-69a1-4c1b-a5e1-313fcb70b6ff | 8ekHdv-u0RJ-B0FO-LUUK-wDWs-iaxb-sh3W3J | 1 export-domain-storage | f72ec125-69a1-4c1b-a5e1-313fcb70b6ff | d3932528-6844-481a-bfed-542872ace9e5 | 1 iso-storage | f72ec125-69a1-4c1b-a5e1-313fcb70b6ff | f800b7a6-6a0c-4560-8476-2f294412d87d | 1 vm-storage-7200rpm | f72ec125-69a1-4c1b-a5e1-313fcb70b6ff | a0bff472-1348-4302-a5c7-f1177efa45a9 | 1 vm-storage-ssd | f72ec125-69a1-4c1b-a5e1-313fcb70b6ff | 95acd9a4-a6fb-4208-80dd-1c53d6aacad0 | 1 vm-storage-ssd2 | f72ec125-69a1-4c1b-a5e1-313fcb70b6ff | 829d0600-c3f7-4dae-a749-d7f05c6a6ca4 | 1 (7 rows) Thanks, -Matthew --

2 12

Poor gluster performances over 10Gbps network
by Mathieu Valois 10 Sep '21

10 Sep '21

Sorry for double post but I don't know if this mail has been received. Hello everyone, I know this issue was already treated on this mailing list. However none of the proposed solutions is satisfying me. Here is my situation : I've got 3 hyperconverged gluster ovirt nodes, with 6 network interfaces, bounded in bunches of 2 (management, VMs and gluster). The gluster network is on a dedicated bound where the 2 interfaces are directly connected to the 2 other ovirt nodes. Gluster is apparently using it : # gluster volume status vmstore Status of volume: vmstore Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick gluster-ov1:/gluster_bricks /vmstore/vmstore 49152 0 Y 3019 Brick gluster-ov2:/gluster_bricks /vmstore/vmstore 49152 0 Y 3009 Brick gluster-ov3:/gluster_bricks /vmstore/vmstore where 'gluster-ov{1,2,3}' are domain names referencing nodes in the gluster network. This networks has 10Gbps capabilities : # iperf3 -c gluster-ov3 Connecting to host gluster-ov3, port 5201 [ 5] local 10.20.0.50 port 46220 connected to 10.20.0.51 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.16 GBytes 9.92 Gbits/sec 17 900 KBytes [ 5] 1.00-2.00 sec 1.15 GBytes 9.90 Gbits/sec 0 900 KBytes [ 5] 2.00-3.00 sec 1.15 GBytes 9.90 Gbits/sec 4 996 KBytes [ 5] 3.00-4.00 sec 1.15 GBytes 9.90 Gbits/sec 1 996 KBytes [ 5] 4.00-5.00 sec 1.15 GBytes 9.89 Gbits/sec 0 996 KBytes [ 5] 5.00-6.00 sec 1.15 GBytes 9.90 Gbits/sec 0 996 KBytes [ 5] 6.00-7.00 sec 1.15 GBytes 9.90 Gbits/sec 0 996 KBytes [ 5] 7.00-8.00 sec 1.15 GBytes 9.91 Gbits/sec 0 996 KBytes [ 5] 8.00-9.00 sec 1.15 GBytes 9.90 Gbits/sec 0 996 KBytes [ 5] 9.00-10.00 sec 1.15 GBytes 9.90 Gbits/sec 0 996 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 11.5 GBytes 9.90 Gbits/sec 22 sender [ 5] 0.00-10.04 sec 11.5 GBytes 9.86 Gbits/sec receiver iperf Done. However, VMs stored on the vmstore gluster volume has poor write performances, oscillating between 100KBps and 30MBps. I almost always observe a write spike (180Mbps) at the beginning until around 500MB written, then it drastically falls at 10MBps, sometimes even less (100KBps). Hypervisors have 32 threads (2 sockets, 8 cores per socket, 2 threads per core). Here is the volume settings : Volume Name: vmstore Type: Replicate Volume ID: XXX Status: Started Snapshot Count: 0 Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: gluster-ov1:/gluster_bricks/vmstore/vmstore Brick2: gluster-ov2:/gluster_bricks/vmstore/vmstore Brick3: gluster-ov3:/gluster_bricks/vmstore/vmstore Options Reconfigured: performance.io-thread-count: 32 # was 16 by default. cluster.granular-entry-heal: enable storage.owner-gid: 36 storage.owner-uid: 36 cluster.lookup-optimize: off server.keepalive-count: 5 server.keepalive-interval: 2 server.keepalive-time: 10 server.tcp-user-timeout: 20 network.ping-timeout: 30 server.event-threads: 4 client.event-threads: 8 # was 4 by default cluster.choose-local: off features.shard: on cluster.shd-wait-qlength: 10000 cluster.shd-max-threads: 8 cluster.locking-scheme: granular cluster.data-self-heal-algorithm: full cluster.server-quorum-type: server cluster.quorum-type: auto cluster.eager-lock: enable performance.strict-o-direct: on network.remote-dio: off performance.low-prio-threads: 32 performance.io-cache: off performance.read-ahead: off performance.quick-read: off auth.allow: * user.cifs: off storage.fips-mode-rchecksum: on transport.address-family: inet nfs.disable: on performance.client-io-threads: on When I naively write directly on the logical volume, which is mounted on a material RAID5 3-disks array, I have interesting performances: # dd if=/dev/zero of=a bs=4M count=2048 2048+0 records in 2048+0 records out 8589934592 bytes (8.6 GB, 8.0 GiB) copied, 17.2485 s, 498 MB/s #urandom gives around 200MBps Moreover, hypervisors have SSD which have been configured as lvcache, but I'm unsure how to test it efficiently. I can't find where is the problem, as every piece of the chain is apparently doing well ... Thanks anyone for helping me :) -- téïcée <https://www.teicee.com/?pk_campaign=Email> *Mathieu Valois* Bureau Caen: Quartier Kœnig - 153, rue Géraldine MOCK - 14760 Bretteville-sur-Odon Bureau Vitré: Zone de la baratière - 12, route de Domalain - 35500 Vitré 02 72 34 13 20 | www.teicee.com <https://www.teicee.com/?pk_campaign=Email> téïcée sur facebook <https://www.facebook.com/teicee> téïcée sur twitter <https://twitter.com/Teicee_fr> téïcée sur linkedin <https://www.linkedin.com/company/t-c-e> téïcée sur viadeo <https://fr.viadeo.com/fr/company/teicee> Datadocké

5 8

about the certification
by CuiTao 09 Sep '21

09 Sep '21

I wanted to understand how certificates work in OLVM, including the following: What certificates exist in the OLVM system? Second, what are the expiration dates of these certificates? Third, how to view the remaining time of the certificate and regenerate the certificate before expiration? Thanks!

1 0

Redeploying hosted engine from backup
by Artem Tambovskiy 09 Sep '21

09 Sep '21

Hello, Just had an issue with my cluster with a self-hosted engine (hosted-engine is not coming up) and decided to redeploy it as I have a backup. Just tried a hosted-engine --deploy --restore-from-file=engine.backup But the script is asking questions like DC name, Cluster name which I can't recall correctly. What will be the consequence of the wrong answer? Is there any chance to get this info from the hosts or backup file? -- Regards, Artem

2 1

how to remove a failed backup operation
by Gianluca Cecchi 09 Sep '21

09 Sep '21

Hello, I was trying incremental backup with the provided /usr/share/doc/python3-ovirt-engine-sdk4/examples/backup_vm.py and began using the "full" option. But I specified an incorrect dir and during backup I got error due to filesystem full [ 156.7 ] Creating image transfer for disk '33b0f6fb-a855-465d-a628-5fce9b64496a' [ 157.8 ] Image transfer 'ccc386d3-9f9d-4727-832a-56d355d60a95' is ready --- Logging error ---, 105.02 seconds, 147.48 MiB/s Traceback (most recent call last): File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/io.py", line 242, in _run handler.copy(req) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/io.py", line 286, in copy self._src.write_to(self._dst, req.length, self._buf) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/backends/http.py", line 216, in write_to writer.write(view[:n]) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/backends/nbd.py", line 118, in write self._client.write(self._position, buf) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/nbd.py", line 445, in write self._recv_reply(cmd) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/nbd.py", line 980, in _recv_reply if self._recv_reply_chunk(cmd): File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/nbd.py", line 1031, in _recv_reply_chunk self._handle_error_chunk(length, flags) File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/nbd.py", line 1144, in _handle_error_chunk raise ReplyError(code, message) ovirt_imageio._internal.nbd.ReplyError: Writing to file failed: [Error 28] No space left on device Now if I try the same backup command (so with "full" option) and I get ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Cannot backup VM. The VM is during a backup operation.]". HTTP response code is 409. How can I clean the situation? BTW: the parameter to put into ovirt.conf is backup-dir or backup_dir or what? Thanks, Gianluca

4 9

Re: Poor gluster performances over 10Gbps network
by Mathieu Valois 09 Sep '21

09 Sep '21

Default shard size was 64MB. I've tested to create another volume with a 512MB shard size and settings you provided. I observe a performance improvement with `dd`, which does a sequential write (from 1MBps to 50MBps). However, I find VM still slow : as an example, `apt upgrade` is very slow to unpack packages, more than 1 hour for around 400MB (90 packages). Le 08/09/2021 à 11:34, Alex K a écrit : > > > On Wed, Sep 8, 2021 at 12:33 PM Alex K <rightkicktech(a)gmail.com > <mailto:rightkicktech@gmail.com>> wrote: > > > > On Wed, Sep 8, 2021 at 12:15 PM Mathieu Valois <mvalois(a)teicee.com > <mailto:mvalois@teicee.com>> wrote: > > Sorry for double post but I don't know if this mail has been > received. > > Hello everyone, > > I know this issue was already treated on this mailing list. > However none of the proposed solutions is satisfying me. > > Here is my situation : I've got 3 hyperconverged gluster ovirt > nodes, with 6 network interfaces, bounded in bunches of 2 > (management, VMs and gluster). The gluster network is on a > dedicated bound where the 2 interfaces are directly connected > to the 2 other ovirt nodes. Gluster is apparently using it : > > # gluster volume status vmstore > Status of volume: vmstore > Gluster process TCP Port RDMA > Port Online Pid > ------------------------------------------------------------------------------ > Brick gluster-ov1:/gluster_bricks > /vmstore/vmstore 49152 0 Y 3019 > Brick gluster-ov2:/gluster_bricks > /vmstore/vmstore 49152 0 Y 3009 > Brick gluster-ov3:/gluster_bricks > /vmstore/vmstore > > where 'gluster-ov{1,2,3}' are domain names referencing nodes > in the gluster network. This networks has 10Gbps capabilities : > > # iperf3 -c gluster-ov3 > Connecting to host gluster-ov3, port 5201 > [ 5] local 10.20.0.50 port 46220 connected to 10.20.0.51 > port 5201 > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 1.16 GBytes 9.92 Gbits/sec > 17 900 KBytes > [ 5] 1.00-2.00 sec 1.15 GBytes 9.90 Gbits/sec > 0 900 KBytes > [ 5] 2.00-3.00 sec 1.15 GBytes 9.90 Gbits/sec > 4 996 KBytes > [ 5] 3.00-4.00 sec 1.15 GBytes 9.90 Gbits/sec > 1 996 KBytes > [ 5] 4.00-5.00 sec 1.15 GBytes 9.89 Gbits/sec > 0 996 KBytes > [ 5] 5.00-6.00 sec 1.15 GBytes 9.90 Gbits/sec > 0 996 KBytes > [ 5] 6.00-7.00 sec 1.15 GBytes 9.90 Gbits/sec > 0 996 KBytes > [ 5] 7.00-8.00 sec 1.15 GBytes 9.91 Gbits/sec > 0 996 KBytes > [ 5] 8.00-9.00 sec 1.15 GBytes 9.90 Gbits/sec > 0 996 KBytes > [ 5] 9.00-10.00 sec 1.15 GBytes 9.90 Gbits/sec > 0 996 KBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 11.5 GBytes 9.90 Gbits/sec > 22 sender > [ 5] 0.00-10.04 sec 11.5 GBytes 9.86 > Gbits/sec receiver > > iperf Done. > > However, VMs stored on the vmstore gluster volume has poor > write performances, oscillating between 100KBps and 30MBps. I > almost always observe a write spike (180Mbps) at the beginning > until around 500MB written, then it drastically falls at > 10MBps, sometimes even less (100KBps). Hypervisors have 32 > threads (2 sockets, 8 cores per socket, 2 threads per core). > > Here is the volume settings : > > Volume Name: vmstore > Type: Replicate > Volume ID: XXX > Status: Started > Snapshot Count: 0 > Number of Bricks: 1 x 3 = 3 > Transport-type: tcp > Bricks: > Brick1: gluster-ov1:/gluster_bricks/vmstore/vmstore > Brick2: gluster-ov2:/gluster_bricks/vmstore/vmstore > Brick3: gluster-ov3:/gluster_bricks/vmstore/vmstore > Options Reconfigured: > performance.io-thread-count: 32 # was 16 by default. > cluster.granular-entry-heal: enable > storage.owner-gid: 36 > storage.owner-uid: 36 > cluster.lookup-optimize: off > server.keepalive-count: 5 > server.keepalive-interval: 2 > server.keepalive-time: 10 > server.tcp-user-timeout: 20 > network.ping-timeout: 30 > server.event-threads: 4 > client.event-threads: 8 # was 4 by default > cluster.choose-local: off > features.shard: on > cluster.shd-wait-qlength: 10000 > cluster.shd-max-threads: 8 > cluster.locking-scheme: granular > cluster.data-self-heal-algorithm: full > cluster.server-quorum-type: server > cluster.quorum-type: auto > cluster.eager-lock: enable > performance.strict-o-direct: on > network.remote-dio: off > performance.low-prio-threads: 32 > performance.io-cache: off > performance.read-ahead: off > performance.quick-read: off > auth.allow: * > user.cifs: off > storage.fips-mode-rchecksum: on > transport.address-family: inet > nfs.disable: on > performance.client-io-threads: on > > I see you have not enabled sharding for the gluster volume that is > hosting VMs. I usually set a shard size of 512MB. Note though that > changing shard size in a live system will corrupt your data. > You need to wipe and set again. > > I mean you have not changed the default sharde size which is I think 4MB. > > > I've seen also good perf with the following tweaks: > > gluster volume set vms remote-dio enable > > gluster volume set vms performance.write-behind-window-size 8MB > > gluster volume set vms performance.cache-size 1GB > > gluster volume set vms performance.io-thread-count 16 > > gluster volume set vms performance.readdir-ahead on > > gluster volume set vms client.event-threads 8 > > gluster volume set vms server.event-threads 8 > > gluster volume set engine features.shard-block-size 512MB > > When I naively write directly on the logical volume, which is > mounted on a material RAID5 3-disks array, I have interesting > performances: > > # dd if=/dev/zero of=a bs=4M count=2048 > 2048+0 records in > 2048+0 records out > 8589934592 bytes (8.6 GB, 8.0 GiB) copied, 17.2485 s, 498 > MB/s #urandom gives around 200MBps > > Moreover, hypervisors have SSD which have been configured as > lvcache, but I'm unsure how to test it efficiently. > > I can't find where is the problem, as every piece of the chain > is apparently doing well ... > > Thanks anyone for helping me :) > > -- > téïcée <https://www.teicee.com/?pk_campaign=Email> *Mathieu > Valois* > > Bureau Caen: Quartier Kœnig - 153, rue Géraldine MOCK - 14760 > Bretteville-sur-Odon > Bureau Vitré: Zone de la baratière - 12, route de Domalain - > 35500 Vitré > 02 72 34 13 20 | www.teicee.com > <https://www.teicee.com/?pk_campaign=Email> > > téïcée sur facebook <https://www.facebook.com/teicee> téïcée > sur twitter <https://twitter.com/Teicee_fr> téïcée sur > linkedin <https://www.linkedin.com/company/t-c-e> téïcée sur > viadeo <https://fr.viadeo.com/fr/company/teicee> Datadocké > > > _______________________________________________ > Users mailing list -- users(a)ovirt.org <mailto:users@ovirt.org> > To unsubscribe send an email to users-leave(a)ovirt.org > <mailto:users-leave@ovirt.org> > Privacy Statement: https://www.ovirt.org/privacy-policy.html > <https://www.ovirt.org/privacy-policy.html> > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > <https://www.ovirt.org/community/about/community-guidelines/> > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/BTY7TJLMQUJ3Q… > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/BTY7TJLMQUJ3Q…> > -- téïcée <https://www.teicee.com/?pk_campaign=Email> *Mathieu Valois* Bureau Caen: Quartier Kœnig - 153, rue Géraldine MOCK - 14760 Bretteville-sur-Odon Bureau Vitré: Zone de la baratière - 12, route de Domalain - 35500 Vitré 02 72 34 13 20 | www.teicee.com <https://www.teicee.com/?pk_campaign=Email> téïcée sur facebook <https://www.facebook.com/teicee> téïcée sur twitter <https://twitter.com/Teicee_fr> téïcée sur linkedin <https://www.linkedin.com/company/t-c-e> téïcée sur viadeo <https://fr.viadeo.com/fr/company/teicee> Datadocké

1 0

data storage domain iso upload problem
by csabany＠freemail.hu 09 Sep '21

09 Sep '21

Hi, I managed a ovirt 4.4.7 for production systems. Last week i removed the Master storage domain (moved tamplates, vm-s well, unattached, etc), but i forgot to move isos. Now, when i upload a new iso to a data storage domain, the system show it, but it's unbootable: "could not read from cdrom code 0005" thanks csabany

4 6

Re: Poor gluster performances over 10Gbps network
by Strahil Nikolov 09 Sep '21

09 Sep '21

Did you check https://lists.ovirt.org/archives/list/users@ovirt.org/message/S7I3PQVERQZT6… ? Best Regards,Strahil Nikolov On Wed, Sep 8, 2021 at 14:25, Staniforth, Paul<P.Staniforth(a)leedsbeckett.ac.uk> wrote: _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/DGITVP5DCEF33…

1 0

Re: Time Drift Issues
by Marcos Sungaila 08 Sep '21

08 Sep '21

Hi Nur, Yes, oVirt Node comes with an NTP client installed and started. You need to configure the servers it will use as external sources. Yes, it will be preserved in a host upgrade. Regards, Marcos From: Nur Imam Febrianto <nur_imam(a)outlook.com> Sent: quarta-feira, 8 de setembro de 2021 08:09 To: Marcos Sungaila <marcos.sungaila(a)oracle.com>; oVirt Users <users(a)ovirt.org> Subject: [External] : RE: Time Drift Issues Hi Marcos, Want to clarify one thing. If I'm using and oVirt Node based host (not EL Linux Based), does it already configured with ntp client or can I just configure any ntp client on the host ? If I just configure ntp client on the host either chronyc or ntpd, will it persist after upgrading the node ? Thanks before. Regards, Nur Imam Febrianto Sent from Mail<https://urldefense.com/v3/__https:/go.microsoft.com/fwlink/?LinkId=550986__…> for Windows From: Marcos Sungaila<mailto:marcos.sungaila@oracle.com> Sent: Tuesday, 07 September 2021 20:54 To: Nur Imam Febrianto<mailto:nur_imam@outlook.com>; oVirt Users<mailto:users@ovirt.org> Subject: RE: Time Drift Issues Hi Nur, ntpd has a 300sec limit for synchronization. If you want a more flexible NTP client, use chronyd. If you prefer to use ntpd, you should have ntpdate as a boot client and ntpd as a runtime client. When the server boots, the ntpdate client will sync time no matter the difference. As a boot client, it runs only once and exits. Then the ntpd service starts, and keep your server clock in sync with an external source. On distributions like CentOS and similar, ntpdate reads the external time sources configuration from ntpd.conf. You should need booth packages and services installed and enabled: I prefer chronyd since it does not need any extra service or procedure to keep clock synchronization. Regards, Marcos From: Nur Imam Febrianto <nur_imam(a)outlook.com<mailto:nur_imam@outlook.com>> Sent: terça-feira, 7 de setembro de 2021 01:08 To: oVirt Users <users(a)ovirt.org<mailto:users@ovirt.org>> Subject: [External] : [ovirt-users] Time Drift Issues Hi All, Recently I got an warning in our cluster about time drift : Host xxxxx has time-drift of 672 seconds while maximum configured value is 300 seconds. What should I do to address this issue ? Should I reconfigure / configure ntp client on all host ? Thanks before. Regards, Nur Imam Febrianto Sent from Mail<https://urldefense.com/v3/__https:/apac01.safelinks.protection.outlook.com/…> for Windows

1 0

Re: [External] : Re: Poor gluster performances over 10Gbps network
by Marcos Sungaila 08 Sep '21

08 Sep '21

Hi Mathieu, Using dd for disk performance tests is not recommended. Besides needing oflag and iflag options, dd will perform a sequential write what is not a real scenario, To evaluate your servers, use fio or bonnie++ for performance tests. These applications will play a more realistic scenario to validate your environment. Regards, Marcos From: Staniforth, Paul <P.Staniforth(a)leedsbeckett.ac.uk> Sent: quarta-feira, 8 de setembro de 2021 08:17 To: Mathieu Valois <mvalois(a)teicee.com>; users <users(a)ovirt.org> Subject: [External] : [ovirt-users] Re: Poor gluster performances over 10Gbps network Hi Mathieu, with a linux VM using dd without the oflag=sync will mean it's using disk bufffers, hence the faster thoughput at the beginning until the buffers are full. Regards, Paul S. ________________________________ From: Mathieu Valois <mvalois(a)teicee.com<mailto:mvalois@teicee.com>> Sent: 08 September 2021 11:56 To: Staniforth, Paul <P.Staniforth(a)leedsbeckett.ac.uk<mailto:P.Staniforth@leedsbeckett.ac.uk>>; users <users(a)ovirt.org<mailto:users@ovirt.org>> Subject: Re: [ovirt-users] Poor gluster performances over 10Gbps network Caution External Mail: Do not click any links or open any attachments unless you trust the sender and know that the content is safe. Hi Paul, thank you for your answer. Indeed I did a `dd` inside a VM to measure the Gluster disk performance. I've also tried `dd` on an hypervisor writing into one of the replicated gluster bricks, which gives good performances (similar to logical volume ones). Le 08/09/2021 à 12:51, Staniforth, Paul a écrit : Hi Mathieu, How are you measuring the Gluster disk performance? also using dd you should use the oflag=dsync to avoid buffer caching. Regards, Paul S ________________________________ From: Mathieu Valois <mvalois(a)teicee.com><mailto:mvalois@teicee.com> Sent: 08 September 2021 10:12 To: users <users(a)ovirt.org><mailto:users@ovirt.org> Subject: [ovirt-users] Poor gluster performances over 10Gbps network Caution External Mail: Do not click any links or open any attachments unless you trust the sender and know that the content is safe. Sorry for double post but I don't know if this mail has been received. Hello everyone, I know this issue was already treated on this mailing list. However none of the proposed solutions is satisfying me. Here is my situation : I've got 3 hyperconverged gluster ovirt nodes, with 6 network interfaces, bounded in bunches of 2 (management, VMs and gluster). The gluster network is on a dedicated bound where the 2 interfaces are directly connected to the 2 other ovirt nodes. Gluster is apparently using it : # gluster volume status vmstore Status of volume: vmstore Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick gluster-ov1:/gluster_bricks /vmstore/vmstore 49152 0 Y 3019 Brick gluster-ov2:/gluster_bricks /vmstore/vmstore 49152 0 Y 3009 Brick gluster-ov3:/gluster_bricks /vmstore/vmstore where 'gluster-ov{1,2,3}' are domain names referencing nodes in the gluster network. This networks has 10Gbps capabilities : # iperf3 -c gluster-ov3 Connecting to host gluster-ov3, port 5201 [ 5] local 10.20.0.50 port 46220 connected to 10.20.0.51 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.16 GBytes 9.92 Gbits/sec 17 900 KBytes [ 5] 1.00-2.00 sec 1.15 GBytes 9.90 Gbits/sec 0 900 KBytes [ 5] 2.00-3.00 sec 1.15 GBytes 9.90 Gbits/sec 4 996 KBytes [ 5] 3.00-4.00 sec 1.15 GBytes 9.90 Gbits/sec 1 996 KBytes [ 5] 4.00-5.00 sec 1.15 GBytes 9.89 Gbits/sec 0 996 KBytes [ 5] 5.00-6.00 sec 1.15 GBytes 9.90 Gbits/sec 0 996 KBytes [ 5] 6.00-7.00 sec 1.15 GBytes 9.90 Gbits/sec 0 996 KBytes [ 5] 7.00-8.00 sec 1.15 GBytes 9.91 Gbits/sec 0 996 KBytes [ 5] 8.00-9.00 sec 1.15 GBytes 9.90 Gbits/sec 0 996 KBytes [ 5] 9.00-10.00 sec 1.15 GBytes 9.90 Gbits/sec 0 996 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 11.5 GBytes 9.90 Gbits/sec 22 sender [ 5] 0.00-10.04 sec 11.5 GBytes 9.86 Gbits/sec receiver iperf Done. However, VMs stored on the vmstore gluster volume has poor write performances, oscillating between 100KBps and 30MBps. I almost always observe a write spike (180Mbps) at the beginning until around 500MB written, then it drastically falls at 10MBps, sometimes even less (100KBps). Hypervisors have 32 threads (2 sockets, 8 cores per socket, 2 threads per core). Here is the volume settings : Volume Name: vmstore Type: Replicate Volume ID: XXX Status: Started Snapshot Count: 0 Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: gluster-ov1:/gluster_bricks/vmstore/vmstore Brick2: gluster-ov2:/gluster_bricks/vmstore/vmstore Brick3: gluster-ov3:/gluster_bricks/vmstore/vmstore Options Reconfigured: performance.io-thread-count: 32 # was 16 by default. cluster.granular-entry-heal: enable storage.owner-gid: 36 storage.owner-uid: 36 cluster.lookup-optimize: off server.keepalive-count: 5 server.keepalive-interval: 2 server.keepalive-time: 10 server.tcp-user-timeout: 20 network.ping-timeout: 30 server.event-threads: 4 client.event-threads: 8 # was 4 by default cluster.choose-local: off features.shard: on cluster.shd-wait-qlength: 10000 cluster.shd-max-threads: 8 cluster.locking-scheme: granular cluster.data-self-heal-algorithm: full cluster.server-quorum-type: server cluster.quorum-type: auto cluster.eager-lock: enable performance.strict-o-direct: on network.remote-dio: off performance.low-prio-threads: 32 performance.io-cache: off performance.read-ahead: off performance.quick-read: off auth.allow: * user.cifs: off storage.fips-mode-rchecksum: on transport.address-family: inet nfs.disable: on performance.client-io-threads: on When I naively write directly on the logical volume, which is mounted on a material RAID5 3-disks array, I have interesting performances: # dd if=/dev/zero of=a bs=4M count=2048 2048+0 records in 2048+0 records out 8589934592 bytes (8.6 GB, 8.0 GiB) copied, 17.2485 s, 498 MB/s #urandom gives around 200MBps Moreover, hypervisors have SSD which have been configured as lvcache, but I'm unsure how to test it efficiently. I can't find where is the problem, as every piece of the chain is apparently doing well ... Thanks anyone for helping me :) -- [téïcée]<https://urldefense.com/v3/__https:/eur02.safelinks.protection.outlook.com/?…> Mathieu Valois Bureau Caen: Quartier Kœnig - 153, rue Géraldine MOCK - 14760 Bretteville-sur-Odon Bureau Vitré: Zone de la baratière - 12, route de Domalain - 35500 Vitré 02 72 34 13 20 | www.teicee.com<https://urldefense.com/v3/__https:/eur02.safelinks.protection.outlook.com/?…> [téïcée sur facebook]<https://urldefense.com/v3/__https:/eur02.safelinks.protection.outlook.com/?…>[téïcée sur twitter]<https://urldefense.com/v3/__https:/eur02.safelinks.protection.outlook.com/?…>[téïcée sur linkedin]<https://urldefense.com/v3/__https:/eur02.safelinks.protection.outlook.com/?…>[téïcée sur viadeo]<https://urldefense.com/v3/__https:/eur02.safelinks.protection.outlook.com/?…> [Datadocké] To view the terms under which this email is distributed, please go to:- https://leedsbeckett.ac.uk/disclaimer/email<https://urldefense.com/v3/__https:/eur02.safelinks.protection.outlook.com/?…> -- [téïcée]<https://urldefense.com/v3/__https:/eur02.safelinks.protection.outlook.com/?…> Mathieu Valois Bureau Caen: Quartier Kœnig - 153, rue Géraldine MOCK - 14760 Bretteville-sur-Odon Bureau Vitré: Zone de la baratière - 12, route de Domalain - 35500 Vitré 02 72 34 13 20 | www.teicee.com<https://urldefense.com/v3/__https:/eur02.safelinks.protection.outlook.com/?…> [téïcée sur facebook]<https://urldefense.com/v3/__https:/eur02.safelinks.protection.outlook.com/?…>[téïcée sur twitter]<https://urldefense.com/v3/__https:/eur02.safelinks.protection.outlook.com/?…>[téïcée sur linkedin]<https://urldefense.com/v3/__https:/eur02.safelinks.protection.outlook.com/?…>[téïcée sur viadeo]<https://urldefense.com/v3/__https:/eur02.safelinks.protection.outlook.com/?…> [Datadocké] To view the terms under which this email is distributed, please go to:- https://leedsbeckett.ac.uk/disclaimer/email<https://urldefense.com/v3/__https:/leedsbeckett.ac.uk/disclaimer/email__;!!…>

1 0

oVirt 2021 Conference is almost here!
by Sandro Bonazzola 08 Sep '21

08 Sep '21

Hello oVirt friends, The oVirt 2021 Conference is almost here! The videos are uploaded, live speakers tech-checked, and last but not least, the final schedule with the watch links is available on the oVirt website <https://blogs.ovirt.org/ovirt-2021-online-conference/>[1]. Most talks are pre-recorded, but the speakers will be in the chat and answer your questions during their talk. See you all there, The organizers [1] https://blogs.ovirt.org/ovirt-2021-online-conference/ -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

Re: Time Drift Issues
by Gianluca Cecchi 08 Sep '21

08 Sep '21

On Wed, Sep 8, 2021 at 1:19 PM Nur Imam Febrianto <nur_imam(a)outlook.com> wrote: > Hi Marcos, > > > > Want to clarify one thing. If I’m using and oVirt Node based host (not EL > Linux Based), does it already configured with ntp client or can I just > configure any ntp client on the host ? If I just configure ntp client on > the host either chronyc or ntpd, will it persist after upgrading the node ? > > > > Thanks before. > > > > Regards, > > Nur Imam Febrianto > > > > Just to clarify: when using oVirt node 4.4, during installation you have the usual way to configure ntp inside anaconda. In 4.4, based on CentOS (stream) 8, when done that way it will configure chronyd. In the recent past there was a bug where sometimes installation crashed when configuring ntp in certain conditions. If you don't configure ntp at install time you can configure it later by modifying /etc/chrony.conf and starting/enabling the service from the cockpit web interface (or command line with usual commands). It will remain across reboots and/or image updates. HIH, Gianluca

1 0