List of Queries related to RHV
by Suchitra Herwadkar
Hi Nir, Pavan
https://ovirt.org/develop/release-management/features/storage/incremental...
On reading more from the article, we have some questions which we will like to discuss in the call
1. Irrespective of the underlying storage, we will always restore the data as raw. But it’s not very clear to us how the disk format conversion will be taken care of internally by RHV.
1. Conversion from raw data to qcow format will only work for disk marked to be incremental or it is not specific to that only.
Would rhv stage the data before conversion to qcow2.
1. For this feature( incremental backup) to work, disks have to be marked as such. User needs to mark those disks as incrementals while creating VM, otherwise this will not work.
1. If we use incremental backups, then will there not be a backing file. Do we have to upload the entire image and then rhv would convert to qcow. What if we want to just upload just the incremental part/changes relative to some base image.
Thanks
Suchitra
2 years, 6 months
Request for reviews - vdsm-tool update-volume
by Germano Veit Michel
Hi,
I'm working an a tool (vdsm-tool update-volume) to make modifying SD
metadata easier and more importantly, safer. This is very useful to recover
from failed LSMs or snapshot issues.
The plan is to use the VDSM API (modified by some of these patches) and add
a tool (vdsm-tool) that talks to the API and modifies the volumes metadata
as required by the user. Currently this is done manually, i.e.: looking at
MD_XXX tags, doing dd, sed and then dd back to the storage. Any wrong
argument (like a skip in place of a seek) can ruin the entire metadata, so
this tool can be quite handy.
The code is not necessarily 100% finished yet, but I've been testing this
for some time and it seems ok from a functional point of view. I'm just not
sure everything I did (especially inside VDSM, example 94366) is correct.
Your comments on what can/should be improved are very welcome at this
point. Please see this series and help reviewing it.
https://gerrit.ovirt.org/#/q/topic:update-volume+(status:open+OR+status:m...
https://gerrit.ovirt.org/#/c/93258/
Thanks,
Germano
2 years, 6 months
error when running VDSM build
by Nir Levy
*I am installing over *
*CentOS Linux release 7.5.1804*
make pylint
which runs:
tox -e pylint static/usr/share/vdsm/sitecustomize.py lib/vdsm
lib/vdsmclient lib/yajsonrpc
failed for:
Problem importing module base.py: cannot import name get_node_last_lineno
*./.tox/pylint/lib/python2.7/site-packages/pylint/checkers/base.py:48*:
from pylint.checkers.utils import get_node_last_lineno
which is located at:
*./.tox/pylint-py3k/lib/python2.7/site-packages/pylint/checkers/utils.py:912:*
*def get_node_last_lineno(node):*
what can cause this error?
Regards,
*Nir.*
*-----------------*
*full output:*
*-----------------*
tox -e pylint \
static/usr/share/vdsm/sitecustomize.py \
lib/vdsm \
lib/vdsmclient \
lib/yajsonrpc \
pylint recreate: /home/root/vdsm/.tox/pylint
pylint installdeps: pylint==1.8
pylint installed:
astroid==1.6.5,backports.functools-lru-cache==1.5,backports.ssl-match-hostname==3.5.0.1,blivet==0.61.15.69,cffi==1.6.0,chardet==2.2.1,configobj==4.7.2,configparser==3.5.0,configshell-fb==1.1.23,coverage==4.4.1,cryptography==1.7.2,decorator==3.4.0,dnspython==1.12.0,enum34==1.1.6,ethtool==0.8,filelock==3.0.10,funcsigs==1.0.2,futures==3.2.0,gssapi==1.2.0,idna==2.4,iniparse==0.4,ioprocess==1.2.0,ipaclient==4.5.4,ipaddress==1.0.16,ipalib==4.5.4,ipaplatform==4.5.4,ipapython==4.5.4,IPy==0.75,isort==4.3.4,jwcrypto==0.4.2,kitchen==1.1.1,kmod==0.1,langtable==0.0.31,lazy-object-proxy==1.3.1,libvirt-python==3.9.0,Magic-file-extensions==0.2,mccabe==0.6.1,mock==2.0.0,netaddr==0.7.19,netifaces==0.10.4,ovirt-imageio-common==1.5.0,pbr==3.1.1,perf==0.1,pluggy==0.8.0,ply==3.4,policycoreutils-default-encoding==0.1,pthreading==0.1.3,py==1.7.0,pyasn1==0.1.9,pyasn1-modules==0.0.8,pycparser==2.14,pycurl==7.19.0,pygobject==3.22.0,pygpgme==0.3,pyinotify==0.9.4,pykickstart==1.99.66.18,pyliblzma==0.5.3,pylint==1.8.0,pyOpenSSL==0.13.1,pyparsing==1.5.6,pyparted==3.9,python-augeas==0.5.0,python-dateutil==2.4.2,python-ldap==2.4.15,python-linux-procfs==0.4.9,python-nss==0.16.0,python-yubico==1.2.3,pyudev==0.15,pyusb==1.0.0b1,pyxattr==0.5.1,PyYAML==3.10,qrcode==5.0.1,requests==2.6.0,rtslib-fb==2.1.63,sanlock-python==3.6.0,schedutils==0.4,seobject==0.1,sepolicy==1.1,singledispatch==3.4.0.3,six==1.10.0,slip==0.4.0,slip.dbus==0.4.0,SSSDConfig==1.16.0,subprocess32==3.2.6,targetcli-fb===2.1.fb46,toml==0.10.0,tox==2.9.1,urlgrabber==3.10,urllib3==1.10.2,urwid==1.1.1,virtualenv==16.1.0,WebOb==1.2.3,wrapt==1.10.11,yum-langpacks==0.4.2,yum-metadata-parser==1.1.4
pylint runtests: PYTHONHASHSEED='2893585131'
pylint runtests: commands[0] | pylint -j0 --reports=no --score=no
static/usr/share/vdsm/sitecustomize.py lib/vdsm lib/vdsmclient lib/yajsonrpc
Problem importing module base.py: cannot import name get_node_last_lineno
Problem importing module base.pyc: cannot import name get_node_last_lineno
Problem importing module base.py: cannot import name get_node_last_lineno
Problem importing module base.py: cannot import name get_node_last_lineno
Problem importing module base.py: cannot import name get_node_last_lineno
2 years, 6 months
Hosts in Unassigned state after upgrading libvirt to 4.9
by Nir Soffer
I updated 2 Fedora 28 hosts today, getting new ovirt-master-release.rpm,
which exposes new virt-preview repo providing libvirt 4.9 and qemu 3.1.
After the update, connecting with engine master (built few week ago) fail
with:
2018-11-26 22:07:51,702+02 WARN
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-94) [] Unexpected return
value: Status [code=-32603, message=Internal JSON-RPC error: {'reason':
"[Errno 2] No such file or directory: '/usr/share/libvirt/cpu_map.xml'"}]
Looks like contents of /usr/share/libvirt/ is different now:
$ ls -1 /usr/share/libvirt/cpu_map/*.xml | head
/usr/share/libvirt/cpu_map/index.xml
/usr/share/libvirt/cpu_map/ppc64_POWER6.xml
/usr/share/libvirt/cpu_map/ppc64_POWER7.xml
/usr/share/libvirt/cpu_map/ppc64_POWER8.xml
/usr/share/libvirt/cpu_map/ppc64_POWER9.xml
/usr/share/libvirt/cpu_map/ppc64_POWERPC_e5500.xml
/usr/share/libvirt/cpu_map/ppc64_POWERPC_e6500.xml
/usr/share/libvirt/cpu_map/ppc64_vendors.xml
/usr/share/libvirt/cpu_map/x86_486.xml
/usr/share/libvirt/cpu_map/x86_athlon.xml
Do we have a fix for this?
Nir
2 years, 6 months
[VDSM] New: warnings in build
by Nir Soffer
Since we moved to pytest 4.0, we have warnings summary at the end of the
build.
Here is example summary:
*00:10:06.166* =============================== warnings summary
===============================*00:10:06.167*
/home/jenkins/workspace/vdsm_master_check-patch-fc28-x86_64/vdsm/lib/vdsm/storage/mailbox.py:73*00:10:06.168*
/home/jenkins/workspace/vdsm_master_check-patch-fc28-x86_64/vdsm/lib/vdsm/storage/mailbox.py:73:
DeprecationWarning: fromstring() is deprecated. Use frombytes()
instead.*00:10:06.171* _zeroCheck = checksum(EMPTYMAILBOX,
CHECKSUM_BYTES)
...
You can see complete warnings here:
https://jenkins.ovirt.org/job/vdsm_master_check-patch-fc28-x86_64/2413/co...
We have only few warnings, but they are repeated a lot.
Nir
2 years, 6 months
mismatch in disk size while uploading a disk in chunks using Image Transfer
by Ranjit DSouza
Hi
I am trying to upload a snapshot disk in chunks. Everything seems to work fine, but observed that the actual_size after upload, is much lesser than the actual_size of the original disk.
Here are the steps:
1. Take a snapshot of a vm disk and download it (using Image Transfer mechanism). Save it on the file system somewhere. This disk name is 3gbdisk. It is Raw + sparse. Resides on nfs storage. The size of this downloaded file is 3 GB.
"actual_size" : "1389109248", //1 GB
"alias" : "3gbdisk",
"content_type" : "data",
"format" : "raw",
"image_id" : "8fbac55e-0c86-4c0b-911b-f5b0a6722834",
"propagate_errors" : "false",
"provisioned_size" : "3221225472",
"shareable" : "false",
"sparse" : "true",
"status" : "ok",
"storage_type" : "image",
"total_size" : "0",
"wipe_after_delete" : "false",
2. Now create a new floating disk, (raw + sparse), with provisioned_size = 3221225472, or 3 GB. This disk name is vmRestoreDisk
3. Upload to this disk using Image Transfer API, using libCurl in chunks of 128 MB. This is done in a while loop, sequentially reading portions of the file downloaded in step 1 and uploading these chunks via libcurl. I Use the Transfer URL, not proxy URL.
Here is the trace of the first chunk. Note the Content-Range and Content-Length headers. Start offset = 0, end offset = 134217727 (or 128 MB)
upload request for chunk, start offset: 0, end offset: 134217727
Upload Started
Header:Content-Range: bytes 0-134217727/3221225472
Header:Content-Length: 3221225472
* Trying 10.210.46.215...
* TCP_NODELAY set
* Connected to pnm86hpch30bl15.pne.ven.veritas.com (10.210.46.215) port 54322 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: O=pne.ven.veritas.com; CN=pnm86hpch30bl15.pne.ven.veritas.com
* start date: Oct 7 08:55:24 2018 GMT
* expire date: Oct 7 08:55:24 2023 GMT
* issuer: C=US; O=pne.ven.veritas.com; CN=pravauto20.pne.ven.veritas.com.59289
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> PUT /images/8ebc9fa8-d322-423e-8a14-5e46ca10ed4e HTTP/1.1
Host: pnm86hpch30bl15.pne.ven.veritas.com:54322
Accept: */*
Content-Length: 134217728
Expect: 100-continue
* Done waiting for 100-continue
* We are completely uploaded and fine
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Date: Fri, 23 Nov 2018 11:52:53 GMT
< Server: WSGIServer/0.1 Python/2.7.5
< Content-Type: application/json; charset=UTF-8
< Content-Length: 0
<
* Closing connection 0
http response code from curl 200
Upload Finished. Return Value: 0
4. Finalize the Image Transfer after all chunks are uploaded. Observed that the disk status goes from 'uploading via API' to finalizing to OK.
5. Do a GET call on the disk (vmRestoreDisk).
"actual_size" : "134217728", //128MB
"alias" : "vmRestoreDisk",
"content_type" : "data",
"format" : "raw",
"image_id" : "3eda3df2-514a-4e78-b999-1729216b25db",
"propagate_errors" : "false",
"provisioned_size" : "3221225472",
"shareable" : "false",
"sparse" : "true",
"status" : "ok",
"storage_type" : "image",
"total_size" : "0",
"wipe_after_delete" : "false",
As you can see, the actual size is just 128 MB, not 1 GB. I have attached the logs of the upload operation. I think I may be missing something, let me know in case you need further information.
Thanks
Ranjit
2 years, 6 months
