OVirt hosted engine setup dnf config

Hi all, I'm currently installing OVirt 4.4 on CentOS 8 in an hosted engine setup and it fails during ovirt-hosted-engine-setup. Because of our corporate proxy, some changes to the dnf.conf is needed (namely proxy_auth_method=basic) and the config is working fine when running dnf commands the normal way. Proxy configuration is provided via http_proxy, https_proxy and ftp_proxy env vars. The setup fails during a DNF call with status code 407 (proxy auth required). Is there another way to provide that config or any other way to resolve this? My googling didn't help me yet. On the same machine I had a running OVirt 4.3 on CentOS 7 running as a test and now wanted to upgrade (after a complete wipe). Thanks for your help, Raphael content of /etc/dnf/dnf.conf: [main] gpgcheck=1 installonly_limit=3 clean_requirements_on_remove=True best=True skip_if_unavailable=False proxy_auth_method=basic timeout=120 minrate=90 Log of ovirt-hosted-engine-setup: [ INFO ] Stage: Initializing [ INFO ] Stage: Environment setup During customization use CTRL-D to abort. Continuing will configure this host for serving as hypervisor and will create a local VM with a running engine. The locally running engine will be used to configure a new storage domain and create a VM there. At the end the disk of the local VM will be moved to the shared storage. Are you sure you want to continue? (Yes, No)[Yes]: Configuration files: Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20200818091958-tozsht.log Version: otopi-1.9.2 (otopi-1.9.2-1.el8) [ INFO ] DNF Downloading 1 files, 0.00KB [ INFO ] DNF Downloaded CentOS-8 - AppStream [ INFO ] DNF Errors during downloading metadata for repository 'AppStream': - Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.68) - Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.67) [ ERROR ] DNF Failed to download metadata for repo 'AppStream': Cannot prepare internal mirrorlist: Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.67) [ ERROR ] Failed to execute stage 'Environment setup': Failed to download metadata for repo 'AppStream': Cannot prepare internal mirrorlist: Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.67) [ INFO ] Stage: Clean up [Some more cleanup]

I guess you can experiment a little bit. Have you tried to 'inject' a task before the failing one to add the options you need via 'blockinfile' or 'lineinfile'. I think the simplest way is to add a 'pause' task , that will give you enough time to: - Find the IP of the local engine vm - ssh to that engine - add the necessary lines for dnf Best Regards, Strahil Nikolov На 18 август 2020 г. 10:28:46 GMT+03:00, "Raphael Höser" <raphael@hoeser.info> написа:
Hi all,
I'm currently installing OVirt 4.4 on CentOS 8 in an hosted engine setup and it fails during ovirt-hosted-engine-setup.
Because of our corporate proxy, some changes to the dnf.conf is needed (namely proxy_auth_method=basic) and the config is working fine when running dnf commands the normal way. Proxy configuration is provided via http_proxy, https_proxy and ftp_proxy env vars.
The setup fails during a DNF call with status code 407 (proxy auth required). Is there another way to provide that config or any other way to resolve this? My googling didn't help me yet.
On the same machine I had a running OVirt 4.3 on CentOS 7 running as a test and now wanted to upgrade (after a complete wipe).
Thanks for your help, Raphael
content of /etc/dnf/dnf.conf: [main] gpgcheck=1 installonly_limit=3 clean_requirements_on_remove=True best=True skip_if_unavailable=False proxy_auth_method=basic timeout=120 minrate=90
Log of ovirt-hosted-engine-setup: [ INFO ] Stage: Initializing [ INFO ] Stage: Environment setup During customization use CTRL-D to abort. Continuing will configure this host for serving as hypervisor and will create a local VM with a running engine. The locally running engine will be used to configure a new storage domain and create a VM there. At the end the disk of the local VM will be moved to the shared storage. Are you sure you want to continue? (Yes, No)[Yes]: Configuration files: Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20200818091958-tozsht.log Version: otopi-1.9.2 (otopi-1.9.2-1.el8) [ INFO ] DNF Downloading 1 files, 0.00KB [ INFO ] DNF Downloaded CentOS-8 - AppStream [ INFO ] DNF Errors during downloading metadata for repository 'AppStream': - Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.68) - Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.67) [ ERROR ] DNF Failed to download metadata for repo 'AppStream': Cannot prepare internal mirrorlist: Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.67) [ ERROR ] Failed to execute stage 'Environment setup': Failed to download metadata for repo 'AppStream': Cannot prepare internal mirrorlist: Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.67) [ INFO ] Stage: Clean up [Some more cleanup] _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/F2ZZVWCZ5WTSTT...

I don't use ansible for the deployment of the hosted engine right now, but like I said the ovirt-hosted-engine-setup:command. Can you tell me where I can find the required ansible plays, then I'll try.

Usually the cockpit deployment is generating an ansible play which you can edit before running it. Best Regards, Strahil Nikolov На 18 август 2020 г. 20:57:36 GMT+03:00, "Raphael Höser" <raphael@hoeser.info> написа:
I don't use ansible for the deployment of the hosted engine right now, but like I said the ovirt-hosted-engine-setup:command. Can you tell me where I can find the required ansible plays, then I'll try. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/UNGBX7GSHBZAYP...

On Tue, Aug 18, 2020 at 9:02 PM Raphael Höser <raphael@hoeser.info> wrote:
I don't use ansible for the deployment of the hosted engine right now, but like I said the ovirt-hosted-engine-setup:command.
You can try: hosted-engine --deploy --otopi-environment=OVEHOSTED_CORE/pauseonRestore=bool:True This is setting the ansible var he_pause_host: https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/
Can you tell me where I can find the required ansible plays, then I'll try.
Good luck and best regards, -- Didi

Hi Didi, thanks for your input, sadly this doesn't change anything of the described behavior. (Also just to mention: It takes way less then a second to get to the cleanup - in case this provides some hints) Stdout: [root@ovirt03 ~]# hosted-engine --deploy --otopi-environment=OVEHOSTED_CORE/pauseonRestore=bool:True [ INFO ] Stage: Initializing [ INFO ] Stage: Environment setup During customization use CTRL-D to abort. Continuing will configure this host for serving as hypervisor and will create a local VM with a running engine. The locally running engine will be used to configure a new storage domain and create a VM there. At the end the disk of the local VM will be moved to the shared storage. Are you sure you want to continue? (Yes, No)[Yes]: Configuration files: Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20200819092656-jh53ut.log Version: otopi-1.9.2 (otopi-1.9.2-1.el8) [ INFO ] DNF Downloading 1 files, 0.00KB [ INFO ] DNF Downloaded CentOS-8 - AppStream [ INFO ] DNF Errors during downloading metadata for repository 'AppStream': - Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.68) - Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.67) [ ERROR ] DNF Failed to download metadata for repo 'AppStream': Cannot prepare internal mirrorlist: Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.68) [ ERROR ] Failed to execute stage 'Environment setup': Failed to download metadata for repo 'AppStream': Cannot prepare internal mirrorlist: Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.68) [ INFO ] Stage: Clean up [ INFO ] Cleaning temporary resources [ INFO ] TASK [ovirt.hosted_engine_setup : Execute just a specific set of steps] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Force facts gathering] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch logs from the engine VM] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Set destination directory path] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Create destination directory] [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : include_tasks] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Find the local appliance image] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Set local_vm_disk_path] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Give the vm time to flush dirty buffers] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Copy engine logs] [ INFO ] TASK [ovirt.hosted_engine_setup : include_tasks] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Remove local vm dir] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Remove temporary entry in /etc/hosts for the local VM] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Clean local storage pools] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Destroy local storage-pool {{ he_local_vm_dir | basename }}] [ INFO ] TASK [ovirt.hosted_engine_setup : Undefine local storage-pool {{ he_local_vm_dir | basename }}] [ INFO ] TASK [ovirt.hosted_engine_setup : Destroy local storage-pool {{ local_vm_disk_path.split('/')[5] }}] [ INFO ] TASK [ovirt.hosted_engine_setup : Undefine local storage-pool {{ local_vm_disk_path.split('/')[5] }}] [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20200819092723.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Hosted Engine deployment failed Log file is located at /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20200819092656-jh53ut.log

On Wed, Aug 19, 2020 at 10:33 AM Raphael Höser <raphael@hoeser.info> wrote:
Hi Didi,
thanks for your input, sadly this doesn't change anything of the described behavior. (Also just to mention: It takes way less then a second to get to the cleanup - in case this provides some hints)
It does - and also the full log below. It means it fails on the host, not inside the engine vm. It probably fails because our internal "minidnf" does not support all the stuff you try to do to dnf. engine-setup has an option '--offline', but sadly hosted-engine --deploy does not have it. You can try: hosted-engine --deploy --otopi-environment=PACKAGER/dnfpackagerEnabled=bool:False Or perhaps: hosted-engine --deploy --otopi-environment="PACKAGER/dnfpackagerEnabled=bool:False PACKAGER/yumpackagerEnabled=bool:False" I think this should be enough, didn't try.
Stdout: [root@ovirt03 ~]# hosted-engine --deploy --otopi-environment=OVEHOSTED_CORE/pauseonRestore=bool:True [ INFO ] Stage: Initializing [ INFO ] Stage: Environment setup During customization use CTRL-D to abort. Continuing will configure this host for serving as hypervisor and will create a local VM with a running engine. The locally running engine will be used to configure a new storage domain and create a VM there. At the end the disk of the local VM will be moved to the shared storage. Are you sure you want to continue? (Yes, No)[Yes]: Configuration files: Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20200819092656-jh53ut.log Version: otopi-1.9.2 (otopi-1.9.2-1.el8) [ INFO ] DNF Downloading 1 files, 0.00KB [ INFO ] DNF Downloaded CentOS-8 - AppStream [ INFO ] DNF Errors during downloading metadata for repository 'AppStream': - Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.68) - Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.67) [ ERROR ] DNF Failed to download metadata for repo 'AppStream': Cannot prepare internal mirrorlist: Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.68) [ ERROR ] Failed to execute stage 'Environment setup': Failed to download metadata for repo 'AppStream': Cannot prepare internal mirrorlist: Status code: 407 for http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=AppStream&infra=$infra (IP: 10.201.210.68) [ INFO ] Stage: Clean up [ INFO ] Cleaning temporary resources [ INFO ] TASK [ovirt.hosted_engine_setup : Execute just a specific set of steps] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Force facts gathering] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch logs from the engine VM] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Set destination directory path] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Create destination directory] [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : include_tasks] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Find the local appliance image] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Set local_vm_disk_path] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Give the vm time to flush dirty buffers] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Copy engine logs] [ INFO ] TASK [ovirt.hosted_engine_setup : include_tasks] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Remove local vm dir] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Remove temporary entry in /etc/hosts for the local VM] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Clean local storage pools] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Destroy local storage-pool {{ he_local_vm_dir | basename }}] [ INFO ] TASK [ovirt.hosted_engine_setup : Undefine local storage-pool {{ he_local_vm_dir | basename }}] [ INFO ] TASK [ovirt.hosted_engine_setup : Destroy local storage-pool {{ local_vm_disk_path.split('/')[5] }}] [ INFO ] TASK [ovirt.hosted_engine_setup : Undefine local storage-pool {{ local_vm_disk_path.split('/')[5] }}] [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20200819092723.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Hosted Engine deployment failed Log file is located at /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20200819092656-jh53ut.log _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/Q4ESHPIUURINFM...
-- Didi

Thanks Didi,
On Wed, Aug 19, 2020 at 10:33 AM Raphael Höser <raphael(a)hoeser.info> wrote:
It does - and also the full log below. It means it fails on the host, not inside the engine vm. I meant, that the output wasn't changed by this.
It probably fails because our internal "minidnf" does not support all the stuff you try to do to dnf. This seems to be the case.
You can try:
hosted-engine --deploy --otopi-environment=PACKAGER/dnfpackagerEnabled=bool:False This Works!
Thanks a lot for your support! For anyone stumbling upon this at a later point: This workaround might work for you, if you made changes to /etc/dnf/dnf.conf for dnf to work on your CentOS8 instance. In our case the problem was the proxy authentication method dnf uses. --- Raphael

On Wed, Aug 19, 2020 at 12:01 PM Raphael Höser <raphael@hoeser.info> wrote:
Thanks Didi,
On Wed, Aug 19, 2020 at 10:33 AM Raphael Höser <raphael(a)hoeser.info> wrote:
It does - and also the full log below. It means it fails on the host, not inside the engine vm. I meant, that the output wasn't changed by this.
It probably fails because our internal "minidnf" does not support all the stuff you try to do to dnf. This seems to be the case.
You can try:
hosted-engine --deploy --otopi-environment=PACKAGER/dnfpackagerEnabled=bool:False This Works!
Thanks a lot for your support!
You are welcome, thanks for the report! :-)
For anyone stumbling upon this at a later point: This workaround might work for you, if you made changes to /etc/dnf/dnf.conf for dnf to work on your CentOS8 instance. In our case the problem was the proxy authentication method dnf uses.
Actually, AFAICT, we never really need dnfpackager anymore, as all the very little logic around installing/removing the appliance is now done in ovirt-ansible-hosted-engine-setup. So it's safe to always use (and thus skip the lengthy dnf cache update etc.). Feel free to open a bug to get this removed and use an "offline packager" (like --offline in engine-setup). Best regards, -- Didi
participants (3)
-
Raphael Höser
-
Strahil Nikolov
-
Yedidyah Bar David