NFS import storage domain permissions error
by Joop
Hi All,
I've got an error when I try to import a storage domain from a Synology
NAS that used to be a 4.3.10 oVirt installation but no matter what I try
I can't get it to import. I keep getting a permission denied error while
with the same settings it worked for the previous version oVirt.
This is the log:
2020-06-27 19:39:38,113+0200 INFO (jsonrpc/0) [vdsm.api] START connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'password': '********', 'protocol_version': 'auto', 'port': '', 'iqn': '', 'conn
ection': 'pakhuis:/volume1/nfs/data', 'ipv6_enabled': 'false', 'id': '00000000-0000-0000-0000-000000000000', 'user': '', 'tpgt': '1'}], options=None) from=::ffff:192.168.11.14,53948, flow_id=c753ebc5-20a3-4e1c-bcd8-794ddde3ec69, task
_id=2a9ef4e7-e2c0-43d6-b03a-7f0d62f08e50 (api:48)
2020-06-27 19:39:38,113+0200 DEBUG (jsonrpc/0) [storage.Server.NFS] Using local locks for NFSv3 locks (storageServer:442)
2020-06-27 19:39:38,114+0200 INFO (jsonrpc/0) [storage.StorageServer.MountConnection] Creating directory '/rhev/data-center/mnt/pakhuis:_volume1_nfs_data' (storageServer:167)
2020-06-27 19:39:38,114+0200 INFO (jsonrpc/0) [storage.fileUtils] Creating directory: /rhev/data-center/mnt/pakhuis:_volume1_nfs_data mode: None (fileUtils:198)
2020-06-27 19:39:38,114+0200 INFO (jsonrpc/0) [storage.Mount] mounting pakhuis:/volume1/nfs/data at /rhev/data-center/mnt/pakhuis:_volume1_nfs_data (mount:207)
2020-06-27 19:39:38,360+0200 DEBUG (jsonrpc/0) [storage.Mount] /rhev/data-center/mnt/pakhuis:_volume1_nfs_data mounted: 0.25 seconds (utils:390)
2020-06-27 19:39:38,380+0200 DEBUG (jsonrpc/0) [storage.Mount] Waiting for udev mount events: 0.02 seconds (utils:390)
2020-06-27 19:39:38,381+0200 DEBUG (jsonrpc/0) [storage.oop] Creating ioprocess Global (outOfProcess:89)
2020-06-27 19:39:38,382+0200 INFO (jsonrpc/0) [IOProcessClient] (Global) Starting client (__init__:308)
2020-06-27 19:39:38,394+0200 INFO (ioprocess/10476) [IOProcess] (Global) Starting ioprocess (__init__:434)
2020-06-27 19:39:38,394+0200 DEBUG (ioprocess/10476) [IOProcess] (Global) Closing unrelated FDs... (__init__:432)
2020-06-27 19:39:38,394+0200 DEBUG (ioprocess/10476) [IOProcess] (Global) Opening communication channels... (__init__:432)
2020-06-27 19:39:38,394+0200 DEBUG (ioprocess/10476) [IOProcess] (Global) Queuing request (slotsLeft=20) (__init__:432)
2020-06-27 19:39:38,394+0200 DEBUG (ioprocess/10476) [IOProcess] (Global) (1) Start request for method 'access' (waitTime=49) (__init__:432)
2020-06-27 19:39:38,395+0200 DEBUG (ioprocess/10476) [IOProcess] (Global) (1) Finished request for method 'access' (runTime=421) (__init__:432)
2020-06-27 19:39:38,395+0200 DEBUG (ioprocess/10476) [IOProcess] (Global) (1) Dequeuing request (slotsLeft=21) (__init__:432)
2020-06-27 19:39:38,396+0200 WARN (jsonrpc/0) [storage.oop] Permission denied for directory: /rhev/data-center/mnt/pakhuis:_volume1_nfs_data with permissions:7 (outOfProcess:193)
2020-06-27 19:39:38,396+0200 INFO (jsonrpc/0) [storage.Mount] unmounting /rhev/data-center/mnt/pakhuis:_volume1_nfs_data (mount:215)
2020-06-27 19:39:38,435+0200 DEBUG (jsonrpc/0) [storage.Mount] /rhev/data-center/mnt/pakhuis:_volume1_nfs_data unmounted: 0.04 seconds (utils:390)
2020-06-27 19:39:38,453+0200 DEBUG (jsonrpc/0) [storage.Mount] Waiting for udev mount events: 0.02 seconds (utils:390)
2020-06-27 19:39:38,453+0200 ERROR (jsonrpc/0) [storage.HSM] Could not connect to storageServer (hsm:2421)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 80, in validateDirAccess
getProcPool().fileUtils.validateAccess(dirPath)
File "/usr/lib/python3.6/site-packages/vdsm/storage/outOfProcess.py", line 194, in validateAccess
raise OSError(errno.EACCES, os.strerror(errno.EACCES))
PermissionError: [Errno 13] Permission denied
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 2418, in connectStorageServer
conObj.connect()
File "/usr/lib/python3.6/site-packages/vdsm/storage/storageServer.py", line 449, in connect
return self._mountCon.connect()
File "/usr/lib/python3.6/site-packages/vdsm/storage/storageServer.py", line 190, in connect
six.reraise(t, v, tb)
File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/usr/lib/python3.6/site-packages/vdsm/storage/storageServer.py", line 183, in connect
self.getMountObj().getRecord().fs_file)
File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 91, in validateDirAccess
raise se.StorageServerAccessPermissionError(dirPath)
vdsm.storage.exception.StorageServerAccessPermissionError: Permission settings on the specified path do not allow access to the storage. Verify permission settings on the specified storage path.: 'path = /rhev/data-center/mnt/pakhuis
:_volume1_nfs_data'
2020-06-27 19:39:38,453+0200 DEBUG (jsonrpc/0) [storage.HSM] knownSDs: {a2691633-e3c9-454e-9ee4-a6f50f4e00fa: vdsm.storage.glusterSD.findDomain} (hsm:2470)
202
If I make a mistake in the path I get the mount command in the logs and
correcting for the mistake let me mount it at /mnt and then I'm able to
create files and folders using the vdsm and qemu accounts so I don't
know where that error is coming from. Had a look at the source of
vdsm/storage/* but I'm not a python programmer.
A couple of other 4.4 oVirt installs don't have this problem and I used
the same usb stick with Centos-8.2 for those as for this one. Followed
the same install instructions, same versions of packages as far as I can
see. Beats me
I'm tempted to reinstall Centos7 and oVirt4.3.10 just to see what
happens, but I'm hoping someone can point me to something that I must
have overlooked.
Regards,
Joop
4 years, 6 months
Re: ETL service aggregation error
by Ayansh Rocks
Thanks Paul.
On Sat, Jun 27, 2020, 1:19 PM Staniforth, Paul <
P.Staniforth(a)leedsbeckett.ac.uk> wrote:
> Hi Ayansh,
> as it's only changing the history database there isn't
> a high risk and you can run it while your system is being used,
> For piece of mind you can do a full backup before, or just backup the Data
> Warehouse.
>
>
> https://www.ovirt.org/documentation/administration_guide/#chap-Backups_an...
>
> Regards,
> Paul S.
>
> ------------------------------
> *From:* Ayansh Rocks <shashank123rastogi(a)gmail.com>
> *Sent:* 27 June 2020 06:42
> *To:* Staniforth, Paul <P.Staniforth(a)leedsbeckett.ac.uk>
> *Cc:* users <users(a)ovirt.org>; devel(a)ovirt.org <devel(a)ovirt.org>
> *Subject:* Re: [ovirt-users] Re: ETL service aggregation error
>
>
> *Caution External Mail:* Do not click any links or open any attachments
> unless you trust the sender and know that the content is safe.
> Hi Paul,
>
> Thanks, is it safe to run this update command. I am getting this issue on
> my production environment.
>
> On Sat, Jun 27, 2020, 12:51 AM Staniforth, Paul <
> P.Staniforth(a)leedsbeckett.ac.uk> wrote:
>
> Hello Ayansh,
> It looks like the lastHourAgg is wrong is should
> be 2 hours before the runTime but is 1:30 hours before runtime.
>
> Try
>
> systemctl stop ovirt-engine-dwhd
>
> in ovirt_engine_history database
>
> UPDATE history_configuration set var_datetime = var_datetime - interval '24 hour' WHERE var_name = 'lastHourAggr';
>
> UPDATE history_configuration set var_datetime = var_datetime - interval '1 day' WHERE var_name = 'lastDayAggr';
>
> systemctl stop ovirt-engine-dwhd
>
>
> see https://www.mail-archive.com/users@ovirt.org/msg53055.html
> <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mai...>
>
>
> Regards,
>
> Paul Staniforth
>
> School of Built Environment Engineering and Computing.
>
> Leeds Beckett University
>
> Networked Systems Analyst, Research and Engineering Support.
> Technical Lead Architect IMS and oVirt Virtualization
>
> tel: +44 (0)113 28123754
> email: p.staniforth(a)leedsbeckett.ac.uk
>
>
> <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gof...>
>
> ------------------------------
> *From:* Ayansh Rocks <shashank123rastogi(a)gmail.com>
> *Sent:* 26 June 2020 17:10
> *To:* Staniforth, Paul <P.Staniforth(a)leedsbeckett.ac.uk>; users <
> users(a)ovirt.org>; devel(a)ovirt.org <devel(a)ovirt.org>
> *Subject:* Re: [ovirt-users] Re: ETL service aggregation error
>
>
> *Caution External Mail:* Do not click any links or open any attachments
> unless you trust the sender and know that the content is safe.
> Can we fix this any how guys.
>
> On Thu, Jun 25, 2020 at 5:53 PM Ayansh Rocks <shashank123rastogi(a)gmail.com>
> wrote:
>
> Is it a bug ?
>
>
> On Thu, Jun 25, 2020 at 4:11 PM Ayansh Rocks <shashank123rastogi(a)gmail.com>
> wrote:
>
> See below as well if it helps -
>
> [root@iondelvm149 ~]# date
> Thu Jun 25 16:09:45 IST 2020
> [root@iondelvm149 ~]# psql -h localhost -U ovirt_engine_history -d
> ovirt_engine_history
> Password for user ovirt_engine_history:
> psql (9.2.24, server 10.6)
> WARNING: psql version 9.2, server version 10.0.
> Some psql features might not work.
> Type "help" for help.
>
> ovirt_engine_history=> select * from history_configuration;
> var_name | var_value | var_datetime
> -------------------+-----------+------------------------
> default_language | en_US |
> firstSync | false | 2018-03-23 12:27:00-04
> lastHourAggr | | 2020-06-25 05:00:00-04
> HourlyAggFailed | false |
> lastDayAggr | | 2020-06-24 01:00:00-04
> MinimalETLVersion | 4.3.0 |
> (6 rows)
>
> ovirt_engine_history=>
>
> On Thu, Jun 25, 2020 at 4:06 PM Staniforth, Paul <
> P.Staniforth(a)leedsbeckett.ac.uk> wrote:
>
> Hello Shashank,
> is it on the hour? it sounds like it got a
> problem aggregating the samples to create the hourly samples.
>
> I had to update the java memory usage as it was running out of heap memory.
>
> The defaults are
>
> DWH_HEAP_MIN=1g
> DWH_HEAP_MAX=1g
>
> see
>
> /usr/share/ovirt-engine-dwh/services/ovirt-engine-dwhd/ovirt-engine-dwhd.conf
>
> for settings.
>
> Regards,
> Paul S.
>
>
> ------------------------------
> *From:* Ayansh Rocks <shashank123rastogi(a)gmail.com>
> *Sent:* 25 June 2020 10:40
> *To:* Staniforth, Paul <P.Staniforth(a)leedsbeckett.ac.uk>; users <
> users(a)ovirt.org>
> *Subject:* Re: [ovirt-users] Re: ETL service aggregation error
>
>
> *Caution External Mail:* Do not click any links or open any attachments
> unless you trust the sender and know that the content is safe.
> Hi Paul,
>
> Yes i am able to connect to the database from ovirt engine machine.
>
> port is already open. what could be issue, i am getting this error in
> every hour.
>
> Thanks
>
> On Wed, Jun 24, 2020 at 6:30 PM Staniforth, Paul <
> P.Staniforth(a)leedsbeckett.ac.uk> wrote:
>
> Hello Shashank,
> it looks like it's had a problem for over 2
> years, is the Data Warehouse database local or remote? is there a firewall
> port open?
>
> Can you connect to the database from the engine machine?
>
> The credentials should be in
> /etc/ovirt-engine-dwh/ovirt-engine-dwhd.conf.d/10-setup-database.conf
>
> as your using 4.3 the version of postgresql is 10 and using scl
>
> scl enable rh-postgresql10 "psql -h localhost -U ovirt_engine_history
> -d ovirt_engine_history"
>
> if it's using the standard user, database and is local.
>
>
> Regards,
> Paul S.
> .
>
> ------------------------------
> *From:* Ayansh Rocks <shashank123rastogi(a)gmail.com>
> *Sent:* 24 June 2020 12:47
> *To:* Staniforth, Paul <P.Staniforth(a)leedsbeckett.ac.uk>; users <
> users(a)ovirt.org>
> *Subject:* Re: [ovirt-users] Re: ETL service aggregation error
>
>
> *Caution External Mail:* Do not click any links or open any attachments
> unless you trust the sender and know that the content is safe.
> Please find the attached error logs.
>
> On Thu, Jun 4, 2020 at 8:17 PM Staniforth, Paul <
> P.Staniforth(a)leedsbeckett.ac.uk> wrote:
>
> Hello Shashank,
> I can't see any of your images and also it
> would be better to have the log file as text.
>
> Regards,
> Paul S.
>
>
>
> ------------------------------
> *From:* Ayansh Rocks <shashank123rastogi(a)gmail.com>
> *Sent:* 04 June 2020 15:16
> *To:* users <users(a)ovirt.org>
> *Subject:* [ovirt-users] Re: ETL service aggregation error
>
>
> *Caution External Mail:* Do not click any links or open any attachments
> unless you trust the sender and know that the content is safe.
> Any update on this ?
>
> On Tue, May 26, 2020 at 1:41 PM Ayansh Rocks <shashank123rastogi(a)gmail.com>
> wrote:
>
> Hi,
>
> I am using 4.3.7 self hosted engine. From Few days i am getting regular
> below error messages :-
> [image: image.png]
>
> Logs in /var/log/ovirt-engine-dwh/ovirt-engine-dwhd.log
> [image: image.png]
>
> What could be the reason for this?
>
> Thanks
> Shashank
>
> To view the terms under which this email is distributed, please go to:-
> http://leedsbeckett.ac.uk/disclaimer/email/
> <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fleedsbec...>
>
> To view the terms under which this email is distributed, please go to:-
> http://leedsbeckett.ac.uk/disclaimer/email/
> <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fleedsbec...>
>
> To view the terms under which this email is distributed, please go to:-
> http://leedsbeckett.ac.uk/disclaimer/email/
> <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fleedsbec...>
>
> To view the terms under which this email is distributed, please go to:-
> http://leedsbeckett.ac.uk/disclaimer/email/
> <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fleedsbec...>
>
> To view the terms under which this email is distributed, please go to:-
> http://leedsbeckett.ac.uk/disclaimer/email/
>
>
4 years, 6 months
Re: ETL service aggregation error
by Ayansh Rocks
Hi Paul,
Thanks, is it safe to run this update command. I am getting this issue on
my production environment.
On Sat, Jun 27, 2020, 12:51 AM Staniforth, Paul <
P.Staniforth(a)leedsbeckett.ac.uk> wrote:
> Hello Ayansh,
> It looks like the lastHourAgg is wrong is should
> be 2 hours before the runTime but is 1:30 hours before runtime.
>
> Try
>
> systemctl stop ovirt-engine-dwhd
>
> in ovirt_engine_history database
>
> UPDATE history_configuration set var_datetime = var_datetime - interval '24 hour' WHERE var_name = 'lastHourAggr';
>
> UPDATE history_configuration set var_datetime = var_datetime - interval '1 day' WHERE var_name = 'lastDayAggr';
>
> systemctl stop ovirt-engine-dwhd
>
>
> see https://www.mail-archive.com/users@ovirt.org/msg53055.html
>
>
> Regards,
>
> Paul Staniforth
>
> School of Built Environment Engineering and Computing.
>
> Leeds Beckett University
>
> Networked Systems Analyst, Research and Engineering Support.
> Technical Lead Architect IMS and oVirt Virtualization
>
> tel: +44 (0)113 28123754
> email: p.staniforth(a)leedsbeckett.ac.uk
>
> <https://www.gofundme.com/f/bytes-for-heroes>
>
> ------------------------------
> *From:* Ayansh Rocks <shashank123rastogi(a)gmail.com>
> *Sent:* 26 June 2020 17:10
> *To:* Staniforth, Paul <P.Staniforth(a)leedsbeckett.ac.uk>; users <
> users(a)ovirt.org>; devel(a)ovirt.org <devel(a)ovirt.org>
> *Subject:* Re: [ovirt-users] Re: ETL service aggregation error
>
>
> *Caution External Mail:* Do not click any links or open any attachments
> unless you trust the sender and know that the content is safe.
> Can we fix this any how guys.
>
> On Thu, Jun 25, 2020 at 5:53 PM Ayansh Rocks <shashank123rastogi(a)gmail.com>
> wrote:
>
> Is it a bug ?
>
>
> On Thu, Jun 25, 2020 at 4:11 PM Ayansh Rocks <shashank123rastogi(a)gmail.com>
> wrote:
>
> See below as well if it helps -
>
> [root@iondelvm149 ~]# date
> Thu Jun 25 16:09:45 IST 2020
> [root@iondelvm149 ~]# psql -h localhost -U ovirt_engine_history -d
> ovirt_engine_history
> Password for user ovirt_engine_history:
> psql (9.2.24, server 10.6)
> WARNING: psql version 9.2, server version 10.0.
> Some psql features might not work.
> Type "help" for help.
>
> ovirt_engine_history=> select * from history_configuration;
> var_name | var_value | var_datetime
> -------------------+-----------+------------------------
> default_language | en_US |
> firstSync | false | 2018-03-23 12:27:00-04
> lastHourAggr | | 2020-06-25 05:00:00-04
> HourlyAggFailed | false |
> lastDayAggr | | 2020-06-24 01:00:00-04
> MinimalETLVersion | 4.3.0 |
> (6 rows)
>
> ovirt_engine_history=>
>
> On Thu, Jun 25, 2020 at 4:06 PM Staniforth, Paul <
> P.Staniforth(a)leedsbeckett.ac.uk> wrote:
>
> Hello Shashank,
> is it on the hour? it sounds like it got a
> problem aggregating the samples to create the hourly samples.
>
> I had to update the java memory usage as it was running out of heap memory.
>
> The defaults are
>
> DWH_HEAP_MIN=1g
> DWH_HEAP_MAX=1g
>
> see
>
> /usr/share/ovirt-engine-dwh/services/ovirt-engine-dwhd/ovirt-engine-dwhd.conf
>
> for settings.
>
> Regards,
> Paul S.
>
>
> ------------------------------
> *From:* Ayansh Rocks <shashank123rastogi(a)gmail.com>
> *Sent:* 25 June 2020 10:40
> *To:* Staniforth, Paul <P.Staniforth(a)leedsbeckett.ac.uk>; users <
> users(a)ovirt.org>
> *Subject:* Re: [ovirt-users] Re: ETL service aggregation error
>
>
> *Caution External Mail:* Do not click any links or open any attachments
> unless you trust the sender and know that the content is safe.
> Hi Paul,
>
> Yes i am able to connect to the database from ovirt engine machine.
>
> port is already open. what could be issue, i am getting this error in
> every hour.
>
> Thanks
>
> On Wed, Jun 24, 2020 at 6:30 PM Staniforth, Paul <
> P.Staniforth(a)leedsbeckett.ac.uk> wrote:
>
> Hello Shashank,
> it looks like it's had a problem for over 2
> years, is the Data Warehouse database local or remote? is there a firewall
> port open?
>
> Can you connect to the database from the engine machine?
>
> The credentials should be in
> /etc/ovirt-engine-dwh/ovirt-engine-dwhd.conf.d/10-setup-database.conf
>
> as your using 4.3 the version of postgresql is 10 and using scl
>
> scl enable rh-postgresql10 "psql -h localhost -U ovirt_engine_history
> -d ovirt_engine_history"
>
> if it's using the standard user, database and is local.
>
>
> Regards,
> Paul S.
> .
>
> ------------------------------
> *From:* Ayansh Rocks <shashank123rastogi(a)gmail.com>
> *Sent:* 24 June 2020 12:47
> *To:* Staniforth, Paul <P.Staniforth(a)leedsbeckett.ac.uk>; users <
> users(a)ovirt.org>
> *Subject:* Re: [ovirt-users] Re: ETL service aggregation error
>
>
> *Caution External Mail:* Do not click any links or open any attachments
> unless you trust the sender and know that the content is safe.
> Please find the attached error logs.
>
> On Thu, Jun 4, 2020 at 8:17 PM Staniforth, Paul <
> P.Staniforth(a)leedsbeckett.ac.uk> wrote:
>
> Hello Shashank,
> I can't see any of your images and also it
> would be better to have the log file as text.
>
> Regards,
> Paul S.
>
>
>
> ------------------------------
> *From:* Ayansh Rocks <shashank123rastogi(a)gmail.com>
> *Sent:* 04 June 2020 15:16
> *To:* users <users(a)ovirt.org>
> *Subject:* [ovirt-users] Re: ETL service aggregation error
>
>
> *Caution External Mail:* Do not click any links or open any attachments
> unless you trust the sender and know that the content is safe.
> Any update on this ?
>
> On Tue, May 26, 2020 at 1:41 PM Ayansh Rocks <shashank123rastogi(a)gmail.com>
> wrote:
>
> Hi,
>
> I am using 4.3.7 self hosted engine. From Few days i am getting regular
> below error messages :-
> [image: image.png]
>
> Logs in /var/log/ovirt-engine-dwh/ovirt-engine-dwhd.log
> [image: image.png]
>
> What could be the reason for this?
>
> Thanks
> Shashank
>
> To view the terms under which this email is distributed, please go to:-
> http://leedsbeckett.ac.uk/disclaimer/email/
> <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fleedsbec...>
>
> To view the terms under which this email is distributed, please go to:-
> http://leedsbeckett.ac.uk/disclaimer/email/
> <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fleedsbec...>
>
> To view the terms under which this email is distributed, please go to:-
> http://leedsbeckett.ac.uk/disclaimer/email/
> <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fleedsbec...>
>
> To view the terms under which this email is distributed, please go to:-
> http://leedsbeckett.ac.uk/disclaimer/email/
>
>
4 years, 6 months
How to suspend a stateless VM
by pascal＠butterflyit.com
Currently from the ovirt web interface it is not possible to suspend a stateless VM. However I have a need for it for example when I want to free idle VM (or migrate them to a different cluster) and still not lose the current status of the VM.
Thanks
4 years, 6 months
How to change the hosted engine configuration manually?
by Ricardo Alonso
After an update in the cluster configuration, the hosted engine ended up with an invalid configuration (bios x machine type) and it's not coming up. I need to return the bios to legacy mode, but I'm not finding a way to correct the xml generated to turn the machine back on. The storage domain is iSCSI.
Any help would be appreciate. (reinstall isn't an option).
4 years, 6 months
Re: ETL service aggregation error
by Ayansh Rocks
Is it a bug ?
On Thu, Jun 25, 2020 at 4:11 PM Ayansh Rocks <shashank123rastogi(a)gmail.com>
wrote:
> See below as well if it helps -
>
> [root@iondelvm149 ~]# date
> Thu Jun 25 16:09:45 IST 2020
> [root@iondelvm149 ~]# psql -h localhost -U ovirt_engine_history -d
> ovirt_engine_history
> Password for user ovirt_engine_history:
> psql (9.2.24, server 10.6)
> WARNING: psql version 9.2, server version 10.0.
> Some psql features might not work.
> Type "help" for help.
>
> ovirt_engine_history=> select * from history_configuration;
> var_name | var_value | var_datetime
> -------------------+-----------+------------------------
> default_language | en_US |
> firstSync | false | 2018-03-23 12:27:00-04
> lastHourAggr | | 2020-06-25 05:00:00-04
> HourlyAggFailed | false |
> lastDayAggr | | 2020-06-24 01:00:00-04
> MinimalETLVersion | 4.3.0 |
> (6 rows)
>
> ovirt_engine_history=>
>
> On Thu, Jun 25, 2020 at 4:06 PM Staniforth, Paul <
> P.Staniforth(a)leedsbeckett.ac.uk> wrote:
>
>> Hello Shashank,
>> is it on the hour? it sounds like it got a
>> problem aggregating the samples to create the hourly samples.
>>
>> I had to update the java memory usage as it was running out of heap
>> memory.
>>
>> The defaults are
>>
>> DWH_HEAP_MIN=1g
>> DWH_HEAP_MAX=1g
>>
>> see
>>
>> /usr/share/ovirt-engine-dwh/services/ovirt-engine-dwhd/ovirt-engine-dwhd.conf
>>
>> for settings.
>>
>> Regards,
>> Paul S.
>>
>>
>> ------------------------------
>> *From:* Ayansh Rocks <shashank123rastogi(a)gmail.com>
>> *Sent:* 25 June 2020 10:40
>> *To:* Staniforth, Paul <P.Staniforth(a)leedsbeckett.ac.uk>; users <
>> users(a)ovirt.org>
>> *Subject:* Re: [ovirt-users] Re: ETL service aggregation error
>>
>>
>> *Caution External Mail:* Do not click any links or open any attachments
>> unless you trust the sender and know that the content is safe.
>> Hi Paul,
>>
>> Yes i am able to connect to the database from ovirt engine machine.
>>
>> port is already open. what could be issue, i am getting this error in
>> every hour.
>>
>> Thanks
>>
>> On Wed, Jun 24, 2020 at 6:30 PM Staniforth, Paul <
>> P.Staniforth(a)leedsbeckett.ac.uk> wrote:
>>
>> Hello Shashank,
>> it looks like it's had a problem for over
>> 2 years, is the Data Warehouse database local or remote? is there a
>> firewall port open?
>>
>> Can you connect to the database from the engine machine?
>>
>> The credentials should be in
>> /etc/ovirt-engine-dwh/ovirt-engine-dwhd.conf.d/10-setup-database.conf
>>
>> as your using 4.3 the version of postgresql is 10 and using scl
>>
>> scl enable rh-postgresql10 "psql -h localhost -U ovirt_engine_history
>> -d ovirt_engine_history"
>>
>> if it's using the standard user, database and is local.
>>
>>
>> Regards,
>> Paul S.
>> .
>>
>> ------------------------------
>> *From:* Ayansh Rocks <shashank123rastogi(a)gmail.com>
>> *Sent:* 24 June 2020 12:47
>> *To:* Staniforth, Paul <P.Staniforth(a)leedsbeckett.ac.uk>; users <
>> users(a)ovirt.org>
>> *Subject:* Re: [ovirt-users] Re: ETL service aggregation error
>>
>>
>> *Caution External Mail:* Do not click any links or open any attachments
>> unless you trust the sender and know that the content is safe.
>> Please find the attached error logs.
>>
>> On Thu, Jun 4, 2020 at 8:17 PM Staniforth, Paul <
>> P.Staniforth(a)leedsbeckett.ac.uk> wrote:
>>
>> Hello Shashank,
>> I can't see any of your images and also
>> it would be better to have the log file as text.
>>
>> Regards,
>> Paul S.
>>
>>
>>
>> ------------------------------
>> *From:* Ayansh Rocks <shashank123rastogi(a)gmail.com>
>> *Sent:* 04 June 2020 15:16
>> *To:* users <users(a)ovirt.org>
>> *Subject:* [ovirt-users] Re: ETL service aggregation error
>>
>>
>> *Caution External Mail:* Do not click any links or open any attachments
>> unless you trust the sender and know that the content is safe.
>> Any update on this ?
>>
>> On Tue, May 26, 2020 at 1:41 PM Ayansh Rocks <
>> shashank123rastogi(a)gmail.com> wrote:
>>
>> Hi,
>>
>> I am using 4.3.7 self hosted engine. From Few days i am getting regular
>> below error messages :-
>> [image: image.png]
>>
>> Logs in /var/log/ovirt-engine-dwh/ovirt-engine-dwhd.log
>> [image: image.png]
>>
>> What could be the reason for this?
>>
>> Thanks
>> Shashank
>>
>> To view the terms under which this email is distributed, please go to:-
>> http://leedsbeckett.ac.uk/disclaimer/email/
>> <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fleedsbec...>
>>
>> To view the terms under which this email is distributed, please go to:-
>> http://leedsbeckett.ac.uk/disclaimer/email/
>> <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fleedsbec...>
>>
>> To view the terms under which this email is distributed, please go to:-
>> http://leedsbeckett.ac.uk/disclaimer/email/
>>
>>
4 years, 6 months
ovirt-provider-ovn syncrhonisation
by chris＠chrislaw.me
Hi - during the last reboot all my ovn networks appeared in oVirt but weren't reflected on the physical host, no VMs would boot that had an OVN network attached.
I was looking at my logs and I can see the following -
Failed to synchronize networks of Provider ovirt-provider-ovn.
Networks of Provider ovirt-provider-ovn were successfully synchronized.
There was an issue with this host/engine where they lost connection which explains the Failed to Sync. The problem is it's since all been rebooted and it doesn't look like the ovirt-provider-ovn has tried to sync since, as I understand it, it should sync every 5 minutes?
Is it just a case that successful syncs aren't always report in the events log? I've checked my timing properties for sync of external providers and all seems correct. So now I'm worried another reboot will result in losing all the ovn networks I've created through oVirt again!
Also worth noting AutoSync is turned on for the provider.
4 years, 6 months
Clean old mount points in hosts VDSM
by Vinícius Ferrão
Hello,
For reasons unknown one of my hosts is trying to mount an old storage point that’s been removed some time ago.
/var/log/vdsm/vdsm.log:2020-06-24 19:57:35,958-0300 INFO (tmap-65016/0) [IOProcessClient] (/192.168.10.6:_mnt_pool0_ovirt_he) Starting client (__init__:308)
/var/log/vdsm/vdsm.log:2020-06-24 19:57:35,968-0300 INFO (ioprocess/12115) [IOProcess] (/192.168.10.6:_mnt_pool0_ovirt_he) Starting ioprocess (__init__:434)
/var/log/vdsm/vdsm.log:2020-06-24 19:57:42,167-0300 INFO (jsonrpc/6) [vdsm.api] START connectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'protocol_version': u'auto', u'connection': u'192.168.10.6:/mnt/pool0/ovirt/he', u'user': u'kvm', u'id': u'e29cf818-5ee5-46e1-85c1-8aeefa33e95d'}], options=None) from=::1,59090, task_id=5ea81925-ec92-4031-aa36-bb6f436321d5 (api:48)
/var/log/vdsm/vdsm.log:2020-06-24 19:57:42,169-0300 INFO (jsonrpc/6) [storage.StorageServer.MountConnection] Creating directory u'/rhev/data-center/mnt/192.168.10.6:_mnt_pool0_ovirt_he' (storageServer:168)
/var/log/vdsm/vdsm.log:2020-06-24 19:57:42,169-0300 INFO (jsonrpc/6) [storage.fileUtils] Creating directory: /rhev/data-center/mnt/192.168.10.6:_mnt_pool0ovirt_he mode: None (fileUtils:199)
/var/log/vdsm/vdsm.log:2020-06-24 19:57:42,169-0300 INFO (jsonrpc/6) [storage.Mount] mounting 192.168.10.6:/mnt/pool0/ovirt/he at /rhev/data-center/mnt/192.168.10.6:_mnt_pool0_ovirt_he (mount:204)
/var/log/vdsm/vdsm.log:MountError: (32, ';mount.nfs: mounting 192.168.10.6:/mnt/pool0/ovirt/he failed, reason given by server: No such file or directory\n')
/var/log/vdsm/vdsm.log:2020-06-24 19:57:43,683-0300 INFO (jsonrpc/5) [vdsm.api] START connectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'protocol_version': u'auto', u'connection': u'192.168.10.6:/mnt/pool0/ovirt/he', u'user': u'kvm', u'id': u'e29cf818-5ee5-46e1-85c1-8aeefa33e95d'}], options=None) from=::1,59094, task_id=9ce61858-dea1-4059-b942-a52c8c82afdc (api:48)
/var/log/vdsm/vdsm.log:2020-06-24 19:57:43,685-0300 INFO (jsonrpc/5) [storage.StorageServer.MountConnection] Creating directory u'/rhev/data-center/mnt/192.168.10.6:_mnt_pool0_ovirt_he' (storageServer:168)
/var/log/vdsm/vdsm.log:2020-06-24 19:57:43,685-0300 INFO (jsonrpc/5) [storage.fileUtils] Creating directory: /rhev/data-center/mnt/192.168.10.6:_mnt_pool0ovirt_he mode: None (fileUtils:199)
/var/log/vdsm/vdsm.log:2020-06-24 19:57:43,685-0300 INFO (jsonrpc/5) [storage.Mount] mounting 192.168.10.6:/mnt/pool0/ovirt/he at /rhev/data-center/mnt/192.168.10.6:_mnt_pool0_ovirt_he (mount:204)
/var/log/vdsm/vdsm.log:MountError: (32, ';mount.nfs: mounting 192.168.10.6:/mnt/pool0/ovirt/he failed, reason given by server: No such file or directory\n’)
This only happens in one host and it’s spamming /var/log/vdsm/vdsm.log.
Any ideia on how to debug this and remove the entry?
Thanks,
4 years, 6 months
Re: ETL service aggregation error
by Ayansh Rocks
Hi Paul,
Yes i am able to connect to the database from ovirt engine machine.
port is already open. what could be issue, i am getting this error in every
hour.
Thanks
On Wed, Jun 24, 2020 at 6:30 PM Staniforth, Paul <
P.Staniforth(a)leedsbeckett.ac.uk> wrote:
> Hello Shashank,
> it looks like it's had a problem for over 2
> years, is the Data Warehouse database local or remote? is there a firewall
> port open?
>
> Can you connect to the database from the engine machine?
>
> The credentials should be in
> /etc/ovirt-engine-dwh/ovirt-engine-dwhd.conf.d/10-setup-database.conf
>
> as your using 4.3 the version of postgresql is 10 and using scl
>
> scl enable rh-postgresql10 "psql -h localhost -U ovirt_engine_history
> -d ovirt_engine_history"
>
> if it's using the standard user, database and is local.
>
>
> Regards,
> Paul S.
> .
>
> ------------------------------
> *From:* Ayansh Rocks <shashank123rastogi(a)gmail.com>
> *Sent:* 24 June 2020 12:47
> *To:* Staniforth, Paul <P.Staniforth(a)leedsbeckett.ac.uk>; users <
> users(a)ovirt.org>
> *Subject:* Re: [ovirt-users] Re: ETL service aggregation error
>
>
> *Caution External Mail:* Do not click any links or open any attachments
> unless you trust the sender and know that the content is safe.
> Please find the attached error logs.
>
> On Thu, Jun 4, 2020 at 8:17 PM Staniforth, Paul <
> P.Staniforth(a)leedsbeckett.ac.uk> wrote:
>
> Hello Shashank,
> I can't see any of your images and also it
> would be better to have the log file as text.
>
> Regards,
> Paul S.
>
>
>
> ------------------------------
> *From:* Ayansh Rocks <shashank123rastogi(a)gmail.com>
> *Sent:* 04 June 2020 15:16
> *To:* users <users(a)ovirt.org>
> *Subject:* [ovirt-users] Re: ETL service aggregation error
>
>
> *Caution External Mail:* Do not click any links or open any attachments
> unless you trust the sender and know that the content is safe.
> Any update on this ?
>
> On Tue, May 26, 2020 at 1:41 PM Ayansh Rocks <shashank123rastogi(a)gmail.com>
> wrote:
>
> Hi,
>
> I am using 4.3.7 self hosted engine. From Few days i am getting regular
> below error messages :-
> [image: image.png]
>
> Logs in /var/log/ovirt-engine-dwh/ovirt-engine-dwhd.log
> [image: image.png]
>
> What could be the reason for this?
>
> Thanks
> Shashank
>
> To view the terms under which this email is distributed, please go to:-
> http://leedsbeckett.ac.uk/disclaimer/email/
> <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fleedsbec...>
>
> To view the terms under which this email is distributed, please go to:-
> http://leedsbeckett.ac.uk/disclaimer/email/
>
>
4 years, 6 months
VDSM can't see StoragePool
by Erez Zarum
I have a Self-hosted Engine running on iSCSI as well as couple of Storage domains using iSCSI, both the SE and those Storage Domains uses the same target portals (two).
I can see the iSCSI sessions and multipath working well from the Host point of view.
Yesterday after doing a restart for the “ovirt-engine” all the hosts besides the one that runs the SE and the SPM went into “Unassigned” mode with an error stating the ovirt-engine can’t communicate with the hosts.
Network wise, everything is good, I can reach all the ports, all the network is well configured, so I ruled this.
Looking at the VDSM logs on those “Unassigned” hosts it looks like the VDSM can’t find the Storage Pool.
(vmrecovery) [vdsm.api] START getConnectedStoragePoolsList(options=None) from=internal, task_id=217ec32b-591c-4376-8dc0-8d62200557ee (api:48)
(vmrecovery) [vdsm.api] FINISH getConnectedStoragePoolsList return={'poollist': []} from=internal, task_id=217ec32b-591c-4376-8dc0-8d62200557ee (api:54)
(vmrecovery) [vds] recovery: waiting for storage pool to go up (clientIF:723)
If I look at the VDSM logs on the host where the SE and SPM is running, no issues there and the node appears up (green) in the ovirt-engine UI.
I managed to set the hosted-engine to maintenance, shut it down and then start it again on another Host, when it starts on that Host, the host goes “green” and if the SPM stays on the previous host, I have two hosts working and the rest remains “Unassigned”.
All the “ovirt-ha-agent”/”ovirt-ha-broker” services seems ok, I restarted them, I also tried to restart the VDSM on the hosts with no luck.
I have the VMs still running, I did shutdown one host (even used the “SSH restart” from the WebUI) to see if that helps, it came back and still went into “Unassigned”.
It seems like the hosts can’t see the Storage pool.
Where should I start to troubleshoot this?
Thanks
4 years, 6 months
