Ales, that was the issue. The config below was as installed by the
hosting center, and rewritten by nmcli. I had not touched the ipv6 part.
After your input below I looked in Cockpit, and removed the dynamic
ipv6. I was able to run the wizard to prepare VM which successfully
created the bridge but then failed on something else during the last
step of the deploy although the engine is running..
Den 2020-09-08 kl. 07:42, skrev Ales Musil:
On Mon, Sep 7, 2020 at 1:16 PM Sverker Abrahamsson
<sverker(a)abrahamsson.com <mailto:sverker@abrahamsson.com>> wrote:
It is static configured:
[root@h1-mgmt network-scripts]# cat ifcfg-enp4s0
NAME=enp4s0
DEVICE=enp4s0
ONBOOT=yes
UUID=af7ccb53-011b-4c36-998a-1878b4ae7100
TYPE=Ethernet
BOOTPROTO=none
IPADDR=144.76.84.73
PREFIX=32
GATEWAY=144.76.84.65
DEFROUTE=yes
DNS1=213.133.98.98
IPV6INIT=yes
IPV6ADDR=2a01:4f8:192:1148::2/64
IPV6_DEFAULTGW=fe80::1
IPV6_DEFROUTE=yes
ZONE=public
PROXY_METHOD=none
BROWSER_ONLY=no
IPV4_FAILURE_FATAL=no
IPV6_AUTOCONF=yes
IPV6_FAILURE_FATAL=no
Any reason to have dynamic ipv6 enabled along with static address?
[root@h1-mgmt network-scripts]# cat ifcfg-enp4s0.4000
NAME=enp4s0.4000
DEVICE=enp4s0.4000
ONBOOT=yes
VLAN=yes
BOOTPROTO=none
IPADDR=172.27.1.1
PREFIX=24
ZONE=internal
IPV6INIT=no
TYPE=Vlan
PHYSDEV=enp4s0
VLAN_ID=4000
REORDER_HDR=yes
GVRP=no
MVRP=no
HWADDR=
PROXY_METHOD=none
BROWSER_ONLY=no
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6_DISABLED=yes
UUID=ecc8064d-18c1-99b7-3fe4-9c5a593ece6f
I think that what would happen if it didn't fail due to ipv6 route
is that enp4s0 would be without any ip's, based on what is logged
as the desired state by vdsm:
{
'interfaces': [{
'name': 'enp4s0',
'state': 'up',
'mtu': 1500
}, {
'vlan': {
'id': 4000,
'base-iface': 'enp4s0'
},
'name': 'enp4s0.4000',
'type': 'vlan',
'state': 'up',
'mtu': 1500,
'ipv4': {
'enabled': False
},
'ipv6': {
'enabled': False
}
}, {
'name': 'ovirtmgmt',
'type': 'linux-bridge',
'state': 'up',
'mtu': 1500,
'bridge': {
'port': [{
'name': 'enp4s0.4000'
}
],
'options': {
'stp': {
'enabled': False
}
}
},
'ipv4': {
'enabled': True,
'address': [{
'ip': '172.27.1.1',
'prefix-length': 24
}
],
'dhcp': False
},
'ipv6': {
'enabled': False
}
}
],
'dns-resolver': {
'config': {
'server': ['213.133.98.98']
}
}
The desired state should be something like this (although I don't
know the correct names for the missing keys and there may be ):
{
'interfaces': [{
'name': 'enp4s0',
'state': 'up',
'mtu': 1500,
'ipv4': {
'enabled': True,
'address': [{
'ip': '144.76.84.73',
'prefix-length': 32,
'gateway': '144.76.84.65'
}
],
'dhcp': False
},
'ipv6': {
'enabled': True,
'address': [{
'ip': '2a01:4f8:192:1148::2',
'prefix-length': 64,
'gateway': 'fe80::1'
}
}
}, {
'vlan': {
'id': 4000,
'base-iface': 'enp4s0'
},
'name': 'enp4s0.4000',
'type': 'vlan',
'state': 'up',
'mtu': 1500,
'ipv4': {
'enabled': False
},
'ipv6': {
'enabled': False
}
}, {
'name': 'ovirtmgmt',
'type': 'linux-bridge',
'state': 'up',
'mtu': 1500,
'bridge': {
'port': [{
'name': 'enp4s0.4000'
}
],
'options': {
'stp': {
'enabled': False
}
}
},
'ipv4': {
'enabled': True,
'address': [{
'ip': '172.27.1.1',
'prefix-length': 24
}
],
'dhcp': False
},
'ipv6': {
'enabled': False
}
}
],
'dns-resolver': {
'config': {
'server': ['213.133.98.98']
}
}
}
I.e. issue is that vdsm assumes that the underlying enp4s0
interface should not have any addresses when it should maintain
all settings it has before. It's only the addresses of enp4s0.4000
that should move to the ovirtmgmt bridge. Alternatively enp4s0
should not get involved at all in the desired state, only enp4s0.4000
That's not the case. In the desired state there is no removal of
address on enp4s0 at all. We are requesting 2 things on enp4s0:
1) Set the interface up
2) Set the MTU to 1500
Everything else should be kept as is. That means nmstate will take
what is requested and what is configured and merge those states which
should result only in change of those two things mentioned.
There are two things that come to my mind currently that might be
wrong. First is the definition of the static and dynamic. This might
actually result in flush of ip address on down/up interface cycle.
The second is that enp4s0 has a default route but ovirtmgmt tries to
be the default route as well.
/Sverker
Den 2020-09-07 kl. 12:54, skrev Dominik Holler:
>
>
> On Mon, Sep 7, 2020 at 12:22 PM Sverker Abrahamsson
> <sverker(a)abrahamsson.com <mailto:sverker@abrahamsson.com>> wrote:
>
> Unforturnatly it is a blocker, I have not found a way to work
> around it.
>
>
> Is there a change in behavior, if IPv6 on the non-vlanned
> interface (enp4s0) is configured in a static way?
>
> Den 2020-09-04 kl. 09:47, skrev Dominik Holler:
>> Sverker, is this bug blocking you, or can you work around it?
>>
>> On Thu, Sep 3, 2020 at 8:52 PM Dominik Holler
>> <dholler(a)redhat.com <mailto:dholler@redhat.com>> wrote:
>>
>> Sverker, thanks!
>>
>> On Thu, Sep 3, 2020 at 6:50 PM Sverker Abrahamsson
>> <sverker(a)abrahamsson.com
>> <mailto:sverker@abrahamsson.com>> wrote:
>>
>> Hi Dominik,
>> bug filed at
>>
https://bugzilla.redhat.com/show_bug.cgi?id=1875520.
>> I'm doing a new install to get fresh vdsm and
>> supervdsm logs which will be attached as soon as
>> they've failed.
>> /Sverker
>>
>> Den 2020-09-03 kl. 18:03, skrev Dominik Holler:
>>>
>>>
>>> On Thu, Sep 3, 2020 at 12:42 PM Sverker Abrahamsson
>>> <sverker(a)abrahamsson.com
>>> <mailto:sverker@abrahamsson.com>> wrote:
>>>
>>> Hi Ales,
>>> this is a CentOS 8 so my impression was that
>>> you always have NetworkManager then? At least
>>> my attempt to remove it failed miserably.
>>>
>>>
>>> Yes, on CentOS 8 hosts oVirt requires the
>>> interfaces managed by NetworkManager.
>>>
>>> The enp4s0 config was created by the install,
>>> so it should be controlled by NetworkManager.
>>>
>>>
>>> This should work. Can you please report a bug on
>>> vdsm [1]?
>>> Would be helpful if the vdsm.log and supervdsm.log
>>> would be attached to this bug.
>>>
>>> [1]
>>>
https://bugzilla.redhat.com/enter_bug.cgi?product=vdsm
>>>
>>> /Sverker
>>>
>>> Den 2020-09-03 kl. 12:29, skrev Ales Musil:
>>>>
>>>>
>>>> On Thu, Sep 3, 2020 at 12:21 PM Sverker
>>>> Abrahamsson <sverker(a)abrahamsson.com
>>>> <mailto:sverker@abrahamsson.com>> wrote:
>>>>
>>>> Hi Ales,
>>>> right now I have a manually created
>>>> ovirtmgmt bridge (virbr0 and vnet0 seems
>>>> to be created during the failed attempt to
>>>> deploy hosted engine):
>>>>
>>>> [root@h1-mgmt ~]# nmcli con show
>>>> NAME UUID
>>>> TYPE DEVICE
>>>> enp4s0
>>>> af7ccb53-011b-4c36-998a-1878b4ae7100
>>>> ethernet enp4s0
>>>> Bridge ovirtmgmt
>>>> 9a0b07c0-2983-fe97-ec7f-ad2b51c3a3f0
>>>> bridge ovirtmgmt
>>>> virbr0
>>>> aa593151-2c12-4cf7-985b-f105b3575d09
>>>> bridge virbr0
>>>> enp4s0.4000
>>>> ecc8064d-18c1-99b7-3fe4-9c5a593ece6f
>>>> vlan enp4s0.4000
>>>> vnet0
>>>> a6db45bd-93c8-4c37-85fc-0c58ba3e9d00
>>>> tun vnet0
>>>> [root@h1-mgmt ~]# nmstatectl show
>>>> ---
>>>> dns-resolver:
>>>> config:
>>>> search: []
>>>> server:
>>>> - 213.133.98.98
>>>> running:
>>>> search: []
>>>> server:
>>>> - 213.133.98.98
>>>> route-rules:
>>>> config: []
>>>> routes:
>>>> config:
>>>> - destination: 0.0.0.0/0
<
http://0.0.0.0/0>
>>>> metric: -1
>>>> next-hop-address: 144.76.84.65
>>>> next-hop-interface: enp4s0
>>>> table-id: 0
>>>> - destination: ::/0
>>>> metric: -1
>>>> next-hop-address: fe80::1
>>>> next-hop-interface: enp4s0
>>>> table-id: 0
>>>> running:
>>>> - destination: 0.0.0.0/0
<
http://0.0.0.0/0>
>>>> metric: 100
>>>> next-hop-address: 144.76.84.65
>>>> next-hop-interface: enp4s0
>>>> table-id: 254
>>>> - destination: 144.76.84.65/32
>>>> <
http://144.76.84.65/32>
>>>> metric: 100
>>>> next-hop-address: ''
>>>> next-hop-interface: enp4s0
>>>> table-id: 254
>>>> - destination: 172.27.1.0/24
>>>> <
http://172.27.1.0/24>
>>>> metric: 425
>>>> next-hop-address: ''
>>>> next-hop-interface: ovirtmgmt
>>>> table-id: 254
>>>> - destination: 192.168.1.0/24
>>>> <
http://192.168.1.0/24>
>>>> metric: 0
>>>> next-hop-address: ''
>>>> next-hop-interface: virbr0
>>>> table-id: 254
>>>> - destination: 2a01:4f8:192:1148::/64
>>>> metric: 100
>>>> next-hop-address: ''
>>>> next-hop-interface: enp4s0
>>>> table-id: 254
>>>> - destination: ::/0
>>>> metric: 100
>>>> next-hop-address: fe80::1
>>>> next-hop-interface: enp4s0
>>>> table-id: 254
>>>> - destination: fe80::/64
>>>> metric: 100
>>>> next-hop-address: ''
>>>> next-hop-interface: enp4s0
>>>> table-id: 254
>>>> - destination: ff00::/8
>>>> metric: 256
>>>> next-hop-address: ''
>>>> next-hop-interface: enp4s0
>>>> table-id: 255
>>>> interfaces:
>>>> - name: ;vdsmdummy;
>>>> type: linux-bridge
>>>> state: down
>>>> ipv4:
>>>> enabled: false
>>>> ipv6:
>>>> enabled: false
>>>> mac-address: DE:D3:A8:24:27:F6
>>>> mtu: 1500
>>>> - name: br-int
>>>> type: unknown
>>>> state: down
>>>> ipv4:
>>>> enabled: false
>>>> ipv6:
>>>> enabled: false
>>>> mac-address: 6E:37:94:63:E0:4B
>>>> mtu: 1500
>>>> - name: enp4s0
>>>> type: ethernet
>>>> state: up
>>>> ethernet:
>>>> auto-negotiation: true
>>>> duplex: full
>>>> speed: 1000
>>>> ipv4:
>>>> address:
>>>> - ip: 144.76.84.73
>>>> prefix-length: 32
>>>> dhcp: false
>>>> enabled: true
>>>> ipv6:
>>>> address:
>>>> - ip: 2a01:4f8:192:1148::2
>>>> prefix-length: 64
>>>> - ip: fe80::62a4:4cff:fee9:4ac
>>>> prefix-length: 64
>>>> auto-dns: true
>>>> auto-gateway: true
>>>> auto-routes: true
>>>> autoconf: true
>>>> dhcp: true
>>>> enabled: true
>>>> mac-address: 60:A4:4C:E9:04:AC
>>>> mtu: 1500
>>>> - name: enp4s0.4000
>>>> type: vlan
>>>> state: up
>>>> ipv4:
>>>> dhcp: false
>>>> enabled: false
>>>> ipv6:
>>>> enabled: false
>>>> mac-address: 60:A4:4C:E9:04:AC
>>>> mtu: 1500
>>>> vlan:
>>>> base-iface: enp4s0
>>>> id: 4000
>>>> - name: lo
>>>> type: unknown
>>>> state: down
>>>> ipv4:
>>>> enabled: false
>>>> ipv6:
>>>> enabled: false
>>>> mtu: 65536
>>>> - name: ovirtmgmt
>>>> type: linux-bridge
>>>> state: up
>>>> bridge:
>>>> options:
>>>> group-forward-mask: 0
>>>> mac-ageing-time: 300
>>>> multicast-snooping: true
>>>> stp:
>>>> enabled: false
>>>> forward-delay: 15
>>>> hello-time: 2
>>>> max-age: 20
>>>> priority: 32768
>>>> port:
>>>> - name: enp4s0.4000
>>>> stp-hairpin-mode: false
>>>> stp-path-cost: 100
>>>> stp-priority: 32
>>>> ipv4:
>>>> address:
>>>> - ip: 172.27.1.1
>>>> prefix-length: 24
>>>> dhcp: false
>>>> enabled: true
>>>> ipv6:
>>>> autoconf: false
>>>> dhcp: false
>>>> enabled: false
>>>> mac-address: 60:A4:4C:E9:04:AC
>>>> mtu: 1500
>>>> - name: ovs-system
>>>> type: unknown
>>>> state: down
>>>> ipv4:
>>>> enabled: false
>>>> ipv6:
>>>> enabled: false
>>>> mac-address: D2:D3:5B:BA:58:38
>>>> mtu: 1500
>>>> - name: virbr0
>>>> type: linux-bridge
>>>> state: up
>>>> bridge:
>>>> options:
>>>> group-forward-mask: 0
>>>> mac-ageing-time: 300
>>>> multicast-snooping: true
>>>> stp:
>>>> enabled: true
>>>> forward-delay: 2
>>>> hello-time: 2
>>>> max-age: 20
>>>> priority: 32768
>>>> port:
>>>> - name: virbr0-nic
>>>> stp-hairpin-mode: false
>>>> stp-path-cost: 100
>>>> stp-priority: 32
>>>> - name: vnet0
>>>> stp-hairpin-mode: false
>>>> stp-path-cost: 100
>>>> stp-priority: 32
>>>> ipv4:
>>>> address:
>>>> - ip: 192.168.1.1
>>>> prefix-length: 24
>>>> dhcp: false
>>>> enabled: true
>>>> ipv6:
>>>> autoconf: false
>>>> dhcp: false
>>>> enabled: false
>>>> mac-address: 52:54:00:C0:A5:20
>>>> mtu: 1500
>>>> - name: virbr0-nic
>>>> type: unknown
>>>> state: down
>>>> ipv4:
>>>> enabled: false
>>>> ipv6:
>>>> enabled: false
>>>> mac-address: 52:54:00:C0:A5:20
>>>> mtu: 1500
>>>> - name: vnet0
>>>> type: unknown
>>>> state: up
>>>> ipv4:
>>>> enabled: false
>>>> ipv6:
>>>> enabled: false
>>>> mac-address: FE:16:3E:1B:21:0B
>>>> mtu: 1500
>>>>
>>>> Reverting the configuration back to not
>>>> have the static bridge and a reboot I get
>>>> this:
>>>>
>>>> [root@h1-mgmt ~]# nmcli con show
>>>> NAME UUID
>>>> TYPE DEVICE
>>>> enp4s0
>>>> af7ccb53-011b-4c36-998a-1878b4ae7100
>>>> ethernet enp4s0
>>>> enp4s0.4000
>>>> ecc8064d-18c1-99b7-3fe4-9c5a593ece6f
>>>> vlan enp4s0.4000
>>>> [root@h1-mgmt ~]# nmstatectl show
>>>> ---
>>>> dns-resolver:
>>>> config:
>>>> search: []
>>>> server:
>>>> - 213.133.98.98
>>>> running:
>>>> search: []
>>>> server:
>>>> - 213.133.98.98
>>>> route-rules:
>>>> config: []
>>>> routes:
>>>> config:
>>>> - destination: 0.0.0.0/0
<
http://0.0.0.0/0>
>>>> metric: -1
>>>> next-hop-address: 144.76.84.65
>>>> next-hop-interface: enp4s0
>>>> table-id: 0
>>>> - destination: ::/0
>>>> metric: -1
>>>> next-hop-address: fe80::1
>>>> next-hop-interface: enp4s0
>>>> table-id: 0
>>>> running:
>>>> - destination: 0.0.0.0/0
<
http://0.0.0.0/0>
>>>> metric: 100
>>>> next-hop-address: 144.76.84.65
>>>> next-hop-interface: enp4s0
>>>> table-id: 254
>>>> - destination: 144.76.84.65/32
>>>> <
http://144.76.84.65/32>
>>>> metric: 100
>>>> next-hop-address: ''
>>>> next-hop-interface: enp4s0
>>>> table-id: 254
>>>> - destination: 172.27.1.0/24
>>>> <
http://172.27.1.0/24>
>>>> metric: 400
>>>> next-hop-address: ''
>>>> next-hop-interface: enp4s0.4000
>>>> table-id: 254
>>>> - destination: 2a01:4f8:192:1148::/64
>>>> metric: 100
>>>> next-hop-address: ''
>>>> next-hop-interface: enp4s0
>>>> table-id: 254
>>>> - destination: ::/0
>>>> metric: 100
>>>> next-hop-address: fe80::1
>>>> next-hop-interface: enp4s0
>>>> table-id: 254
>>>> - destination: fe80::/64
>>>> metric: 100
>>>> next-hop-address: ''
>>>> next-hop-interface: enp4s0
>>>> table-id: 254
>>>> - destination: ff00::/8
>>>> metric: 256
>>>> next-hop-address: ''
>>>> next-hop-interface: enp4s0
>>>> table-id: 255
>>>> interfaces:
>>>> - name: ;vdsmdummy;
>>>> type: linux-bridge
>>>> state: down
>>>> ipv4:
>>>> enabled: false
>>>> ipv6:
>>>> enabled: false
>>>> mac-address: B2:9E:E0:61:71:88
>>>> mtu: 1500
>>>> - name: br-int
>>>> type: unknown
>>>> state: down
>>>> ipv4:
>>>> enabled: false
>>>> ipv6:
>>>> enabled: false
>>>> mac-address: 6E:37:94:63:E0:4B
>>>> mtu: 1500
>>>> - name: enp4s0
>>>> type: ethernet
>>>> state: up
>>>> ethernet:
>>>> auto-negotiation: true
>>>> duplex: full
>>>> speed: 1000
>>>> ipv4:
>>>> address:
>>>> - ip: 144.76.84.73
>>>> prefix-length: 32
>>>> dhcp: false
>>>> enabled: true
>>>> ipv6:
>>>> address:
>>>> - ip: 2a01:4f8:192:1148::2
>>>> prefix-length: 64
>>>> - ip: fe80::62a4:4cff:fee9:4ac
>>>> prefix-length: 64
>>>> auto-dns: true
>>>> auto-gateway: true
>>>> auto-routes: true
>>>> autoconf: true
>>>> dhcp: true
>>>> enabled: true
>>>> mac-address: 60:A4:4C:E9:04:AC
>>>> mtu: 1500
>>>> - name: enp4s0.4000
>>>> type: vlan
>>>> state: up
>>>> ipv4:
>>>> address:
>>>> - ip: 172.27.1.1
>>>> prefix-length: 24
>>>> dhcp: false
>>>> enabled: true
>>>> ipv6:
>>>> autoconf: false
>>>> dhcp: false
>>>> enabled: false
>>>> mac-address: 60:A4:4C:E9:04:AC
>>>> mtu: 1500
>>>> vlan:
>>>> base-iface: enp4s0
>>>> id: 4000
>>>> - name: lo
>>>> type: unknown
>>>> state: down
>>>> ipv4:
>>>> enabled: false
>>>> ipv6:
>>>> enabled: false
>>>> mtu: 65536
>>>> - name: ovs-system
>>>> type: unknown
>>>> state: down
>>>> ipv4:
>>>> enabled: false
>>>> ipv6:
>>>> enabled: false
>>>> mac-address: A2:35:7A:6C:B7:EF
>>>> mtu: 1500
>>>>
>>>> /Sverker
>>>>
>>>>
>>>> Was the enp4s0 always managed by
>>>> NetworkManager or only after the attempt to
>>>> make the ovirtmgmt? If not that would explain
>>>> the failure.
>>>> Also the workaround would be to configure the
>>>> interface via NetworkManager and then run the
>>>> host deploy again.
>>>>
>>>> Thanks,
>>>> Ales
>>>>
>>>> Den 2020-09-03 kl. 11:54, skrev Ales Musil:
>>>>>
>>>>>
>>>>> On Thu, Sep 3, 2020 at 11:51 AM Sverker
>>>>> Abrahamsson via Users <users(a)ovirt.org
>>>>> <mailto:users@ovirt.org>> wrote:
>>>>>
>>>>> Hi Dominik
>>>>> That is my issue, I don't get to
>>>>> where I can get the ovirtmgmt bridge
>>>>> established because vdsm insists on
>>>>> creating it. It used to be possible
>>>>> to create that bridge statically and
>>>>> vdsm would just skip it but seems to
>>>>> be broken now.
>>>>>
>>>>> If it would be possible to use OVN
>>>>> for the management network that would
>>>>> solve my issue and would be the
>>>>> preferable solution, but as you write
>>>>> that isn't possible which was what I
>>>>> suspected.
>>>>>
>>>>> Do you have any other suggestion on
>>>>> how to solve this issue? That I get
>>>>> the external interface untagged and
>>>>> the internal network tagged is not
>>>>> possible to change.
>>>>>
>>>>> /Sverker
>>>>>
>>>>>
>>>>> Hello Sverker,
>>>>>
>>>>> can you please share output from "nmcli
>>>>> con show" and "nmstatectl
show"?
>>>>>
>>>>> Thank you.
>>>>> Regards,
>>>>> Ales
>>>>>
>>>>> Den 2020-09-03 kl. 10:52, skrev
>>>>> Dominik Holler:
>>>>>>
>>>>>>
>>>>>> On Wed, Sep 2, 2020 at 10:38 PM
>>>>>> Sverker Abrahamsson via Users
>>>>>> <users(a)ovirt.org
>>>>>> <mailto:users@ovirt.org>>
wrote:
>>>>>>
>>>>>> Well, unforturnatly I don't
have
>>>>>> a choise since it is out of my
>>>>>> control.
>>>>>> I only have one physical network
>>>>>> port where the external traffic
is
>>>>>> untagged and the internal vlan
>>>>>> is tagged. If I could run with
OVN
>>>>>>
>>>>>>
>>>>>> OVN is for VM traffic only, not
>>>>>> usable for the management network.
>>>>>>
>>>>>> instead I wouldn't need that
>>>>>> tagged vlan, but I haven't
been
>>>>>> able to get
>>>>>> that to work neither.
>>>>>>
>>>>>>
>>>>>> Please let us know if OVN does not
>>>>>> work for VM traffic for you.
>>>>>>
>>>>>> It's perfectly possible to
have
>>>>>> both tagged and untagged traffic
>>>>>> on the
>>>>>> same switch port, issue is that
>>>>>> vdsm tries to take control over
the
>>>>>> network without being able to be
>>>>>> flexible enough.. I'm
attempting
>>>>>> now to
>>>>>> have ovirtmgmt bridge created
>>>>>> before, that used to be possible
>>>>>> but
>>>>>> according to previous mails on
>>>>>> the list it went broken
>>>>>> somewhere at 4.x.
>>>>>>
>>>>>> /Sverker
>>>>>>
>>>>>> Den 2020-09-02 kl. 21:39, skrev
>>>>>> Strahil Nikolov:
>>>>>> > Switchports can either be
>>>>>> tagged or untagged.
>>>>>> > I'm not sure that your
setup
>>>>>> is supported at all.
>>>>>> >
>>>>>> > Best Regards,
>>>>>> > Strahil Nikolov
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > В сряда, 2 септември 2020
г.,
>>>>>> 20:41:57 Гринуич+3, Sverker
>>>>>> Abrahamsson via Users
>>>>>> <users(a)ovirt.org
>>>>>>
<mailto:users@ovirt.org>> написа:
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > Pretty formatting the
"desired
>>>>>> state" it seems that vdsm
tries
>>>>>> to remove
>>>>>> > the ip of my underlying
>>>>>> interface, that is enp4s0:
>>>>>> >
>>>>>>
>>>>>> > {
>>>>>> > 'interfaces': [{
>>>>>> > 'name':
'enp4s0',
>>>>>> > 'state':
'up',
>>>>>> > 'mtu': 1500
>>>>>> > }, {
>>>>>> > 'vlan': {
>>>>>> > 'id': 4000,
>>>>>> > 'base-iface':
'enp4s0'
>>>>>> > },
>>>>>> > 'name':
'enp4s0.4000',
>>>>>> > 'type':
'vlan',
>>>>>> > 'state':
'up',
>>>>>> > 'mtu': 1500,
>>>>>> > 'ipv4': {
>>>>>> > 'enabled': False
>>>>>> > },
>>>>>> > 'ipv6': {
>>>>>> > 'enabled': False
>>>>>> > }
>>>>>> > }, {
>>>>>> > 'name':
'ovirtmgmt',
>>>>>> > 'type':
'linux-bridge',
>>>>>> > 'state':
'up',
>>>>>> > 'mtu': 1500,
>>>>>> > 'bridge': {
>>>>>> > 'port': [{
>>>>>> > 'name':
'enp4s0.4000'
>>>>>> > }
>>>>>> > ],
>>>>>> > 'options': {
>>>>>> > 'stp': {
>>>>>> > 'enabled':
False
>>>>>> > }
>>>>>> > }
>>>>>> > },
>>>>>> > 'ipv4': {
>>>>>> > 'enabled': True,
>>>>>> > 'address': [{
>>>>>> > 'ip':
'172.27.1.1',
>>>>>> > 'prefix-length': 24
>>>>>> > }
>>>>>> > ],
>>>>>> > 'dhcp': False
>>>>>> > },
>>>>>> > 'ipv6': {
>>>>>> > 'enabled': False
>>>>>> > }
>>>>>> > }
>>>>>> > ],
>>>>>> > 'dns-resolver': {
>>>>>> > 'config'
>>>>>> > : {
>>>>>> > 'server':
['213.133.98.98']
>>>>>> > }
>>>>>> > }
>>>>>> > }
>>>>>> >
>>>>>>
>>>>>>
>>>>>> Thanks, this is helpful information.
>>>>>> Can you please share the
>>>>>> getCapabilities result sent from
>>>>>> vdsm to Engine directly before the
>>>>>> setupNetworks request,
>>>>>> and the parameters of the
>>>>>> setupNetworks request from Engine to
>>>>>> vdsm?
>>>>>> Both are in the vdsm.log during
>>>>>> adding the host.
>>>>>>
>>>>>> >
>>>>>> > This is my interfaces
before
>>>>>> vdsm attemtpts to change the
config:
>>>>>> >
>>>>>> > enp4s0:
>>>>>>
flags=4163<UP,BROADCAST,RUNNING,MULTICAST>
>>>>>> mtu 1500
>>>>>> > inet 144.76.84.73 netmask
>>>>>> 255.255.255.255 broadcast
0.0.0.0
>>>>>> > inet6
fe80::62a4:4cff:fee9:4ac
>>>>>> prefixlen 64 scopeid
0x20<link>
>>>>>> > inet6 2a01:4f8:192:1148::2
>>>>>> prefixlen 64 scopeid
0x0<global>
>>>>>> > ether 60:a4:4c:e9:04:ac
>>>>>> txqueuelen 1000 (Ethernet)
>>>>>> > RX packets 293442 bytes
>>>>>> 385541799 (367.6 MiB)
>>>>>> > RX errors 0 dropped 0
overruns
>>>>>> 0 frame 0
>>>>>> > TX packets 91095 bytes
>>>>>> 31160348 (29.7 MiB)
>>>>>> > TX errors 0 dropped 0
overruns
>>>>>> 0 carrier 0 collisions 0
>>>>>> > device interrupt 17 memory
>>>>>> 0xf7d00000-f7d20000
>>>>>> >
>>>>>> > enp4s0.4000:
>>>>>>
flags=4163<UP,BROADCAST,RUNNING,MULTICAST>
>>>>>> mtu 1500
>>>>>> > inet 172.27.1.1 netmask
>>>>>> 255.255.255.0 broadcast
172.27.1.255
>>>>>> > inet6
fe80::62a4:4cff:fee9:4ac
>>>>>> prefixlen 64 scopeid
0x20<link>
>>>>>> > ether 60:a4:4c:e9:04:ac
>>>>>> txqueuelen 1000 (Ethernet)
>>>>>> > RX packets 0 bytes 0 (0.0
B)
>>>>>> > RX errors 0 dropped 0
overruns
>>>>>> 0 frame 0
>>>>>> > TX packets 13 bytes 938
(938.0 B)
>>>>>> > TX errors 0 dropped 0
overruns
>>>>>> 0 carrier 0 collisions 0
>>>>>> >
>>>>>> > I.e. enp4s0 is the external
>>>>>> interface that must not be
>>>>>> changed, bridge
>>>>>> > must be created on the vlan
>>>>>> interface. I would prefer to
>>>>>> create the
>>>>>> > bridge manually and not
>>>>>> through vdsm if that is
possible.
>>>>>> >
>>>>>> > /Sverker
>>>>>> >
>>>>>> > Den 2020-09-02 kl. 19:14,
>>>>>> skrev Sverker Abrahamsson via
Users:
>>>>>> >> Hi,
>>>>>> >> I'm attempting to
install
>>>>>> hosted engine but getting this
>>>>>> failure:
>>>>>> >>
>>>>>> >> [ ERROR ] fatal:
[localhost]:
>>>>>> FAILED! =>
{"changed": false, "msg":
>>>>>> >> "The host has been
set in
>>>>>> non_operational status,
>>>>>> deployment errors:
>>>>>> >> code 505: Host
>>>>>>
h1-mgmt.limetransit.com
>>>>>>
<
http://h1-mgmt.limetransit.com>
>>>>>> installation failed. Failed to
>>>>>> >> configure management
network
>>>>>> on the host., code 1120:
>>>>>> Failed to
>>>>>> >> configure management
network
>>>>>> on host
h1-mgmt.limetransit.com
>>>>>>
<
http://h1-mgmt.limetransit.com>
>>>>>> due to
>>>>>> >> setup networks failure.,
code
>>>>>> 9000: Failed to verify Power
>>>>>> Management
>>>>>> >> configuration for Host
>>>>>>
h1-mgmt.limetransit.com
>>>>>>
<
http://h1-mgmt.limetransit.com>.,
>>>>>> code 10802: VDSM
>>>>>> >>
h1-mgmt.limetransit.com
>>>>>>
<
http://h1-mgmt.limetransit.com>
>>>>>> command HostSetupNetworksVDS
>>>>>> failed: Internal
>>>>>> >> JSON-RPC error:
{'reason':
>>>>>> 'Unexpected failure of libnm
>>>>>> when running
>>>>>> >> the mainloop: run
>>>>>> execution'}, fix accordingly
and
>>>>>> re-deploy."}
>>>>>> >>
>>>>>> >> Looking in vdsm.log I
find
>>>>>> this which I believe is the root
>>>>>> cause:
>>>>>> >>
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >>
>>>>>>
16:38:25,897::supervdsm_server::93::SuperVdsm.ServerCallback::(wrapper)
>>>>>> >> call setupNetworks with
>>>>>> ({'ovirtmgmt':
{'vlan': '4000',
>>>>>> 'netmask':
>>>>>> >>
'255.255.255.0',
>>>>>> 'ipv6autoconf': False,
>>>>>> >> 'nic':
'enp4s0', 'bridged':
>>>>>> 'true', 'ipaddr':
'172.27.1.1',
>>>>>> >> 'defaultRoute':
True,
>>>>>> 'dhcpv6': False,
'STP': 'no',
>>>>>> 'mtu': 1500,
>>>>>> >> 'switch':
'legacy'}}, {},
>>>>>> {'connectivityTimeout':
120,
>>>>>> >>
'commitOnSuccess': True,
>>>>>> 'connectivityCh
>>>>>> >> eck':
'true'}) {}
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::INFO::2020-09-02
>>>>>> >>
>>>>>>
16:38:25,897::api::220::root::(setupNetworks)
>>>>>> Setting up network
>>>>>> >> according to
configuration:
>>>>>> networks:{'ovirtmgmt':
{'vlan':
>>>>>> '4000',
>>>>>> >> 'netmask':
'255.255.255.0',
>>>>>> 'ipv6autoconf': Fal
>>>>>> >> se, 'nic':
'enp4s0',
>>>>>> 'bridged':
'true', 'ipaddr':
>>>>>> '172.27.1.1',
>>>>>> >> 'defaultRoute':
True,
>>>>>> 'dhcpv6': False,
'STP': 'no',
>>>>>> 'mtu': 1500,
>>>>>> >> 'switch':
'legacy'}},
>>>>>> bondings:{},
>>>>>>
options:{'connectivityTimeout':
>>>>>> >> 120,
'commitOnSuccess':
>>>>>> >> True,
'connectivityCheck':
>>>>>> 'true'}
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >>
>>>>>>
16:38:25,902::cmdutils::130::root::(exec_cmd)
>>>>>> /sbin/tc qdisc show (cwd
>>>>>> >> None)
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >>
>>>>>>
16:38:25,906::cmdutils::138::root::(exec_cmd)
>>>>>> SUCCESS: <err> =
b'';
>>>>>> >> <rc> = 0
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >>
>>>>>>
16:38:25,945::vsctl::74::root::(commit)
>>>>>> Executing commands:
>>>>>> >> /usr/bin/ovs-vsctl
>>>>>> --timeout=5 --oneline
>>>>>> --format=json -- list Bridge
>>>>>> >> -- list Port -- list
Interface
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >>
>>>>>>
16:38:25,945::cmdutils::130::root::(exec_cmd)
>>>>>> /usr/bin/ovs-vsctl
>>>>>> >> --timeout=5 --oneline
>>>>>> --format=json -- list Bridge --
>>>>>> list Port --
>>>>>> >> list Interface (cwd
None)
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >>
>>>>>>
16:38:25,952::cmdutils::138::root::(exec_cmd)
>>>>>> SUCCESS: <err> =
b'';
>>>>>> >> <rc> = 0
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::INFO::2020-09-02
>>>>>> >>
>>>>>>
16:38:25,957::netconfpersistence::58::root::(setNetwork)
>>>>>> Adding
>>>>>> >> network
ovirtmgmt({'vlan':
>>>>>> 4000, 'netmask':
'255.255.255.0',
>>>>>> >> 'ipv6autoconf':
False, 'nic':
>>>>>> 'enp4s0',
'bridged': True
>>>>>> >> , 'ipaddr':
'172.27.1.1',
>>>>>> 'defaultRoute': True,
'dhcpv6':
>>>>>> False,
>>>>>> >> 'mtu': 1500,
'switch':
>>>>>> 'legacy', 'stp':
False,
>>>>>> 'bootproto':
'none',
>>>>>> >> 'nameservers':
>>>>>> ['213.133.98.98']})
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >>
>>>>>>
16:38:25,958::commands::153::common.commands::(start)
>>>>>> /usr/bin/taskset
>>>>>> >> --cpu-list 0-7
>>>>>>
/usr/libexec/vdsm/hooks/before_network_setup/50_fcoe
>>>>>> >> (cwd None)
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::INFO::2020-09-02
>>>>>> >>
>>>>>>
16:38:26,154::hooks::122::root::(_runHooksDir)
>>>>>> >>
>>>>>>
/usr/libexec/vdsm/hooks/before_network_setup/50_fcoe:
>>>>>> rc=0 err=b''
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::INFO::2020-09-02
>>>>>> >>
>>>>>>
16:38:26,155::configurator::195::root::(_setup_nmstate)
>>>>>> Processing
>>>>>> >> setup through nmstate
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::INFO::2020-09-02
>>>>>> >>
>>>>>>
16:38:26,175::configurator::197::root::(_setup_nmstate)
>>>>>> Desired state:
>>>>>> >> {'interfaces':
[{'name':
>>>>>> 'enp4s0',
'state': 'up', 'mtu':
>>>>>> 1500},
>>>>>> >> {'vlan':
{'id': 4000,
>>>>>> 'base-iface':
'enp4s0'}
>>>>>> >> , 'name':
'enp4s0.4000',
>>>>>> 'type': 'vlan',
'state': 'up',
>>>>>> 'mtu': 1500,
>>>>>> >> 'ipv4':
{'enabled': False},
>>>>>> 'ipv6':
{'enabled': False}},
>>>>>> {'name':
>>>>>> >> 'ovirtmgmt',
'type':
>>>>>> 'linux-bridge',
'state': 'up',
>>>>>> 'mtu': 1500,
>>>>>> >> 'bridge':
{'port': [
>>>>>> >> {'name':
'enp4s0.4000'}],
>>>>>> 'options':
{'stp': {'enabled':
>>>>>> False}}},
>>>>>> >> 'ipv4':
{'enabled': True,
>>>>>> 'address':
[{'ip': '172.27.1.1',
>>>>>> >> 'prefix-length':
24}],
>>>>>> 'dhcp': False},
'ipv6':
>>>>>> {'enabled': False}}],
>>>>>> >> 'dns-resolver':
{'config'
>>>>>> >> : {'server':
['213.133.98.98']}}}
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >>
>>>>>>
16:38:26,217::checkpoint::121::root::(create)
>>>>>> Checkpoint
>>>>>> >>
>>>>>>
/org/freedesktop/NetworkManager/Checkpoint/1
>>>>>> created for all devices: 60
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >>
>>>>>>
16:38:26,218::netapplier::239::root::(_add_interfaces)
>>>>>> Adding new
>>>>>> >> interfaces:
['ovirtmgmt']
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >>
>>>>>>
16:38:26,221::netapplier::251::root::(_edit_interfaces)
>>>>>> Editing
>>>>>> >> interfaces:
['enp4s0.4000',
>>>>>> 'enp4s0']
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >>
>>>>>>
16:38:26,224::nmclient::136::root::(execute_next_action)
>>>>>> Executing NM
>>>>>> >> action:
func=add_connection_async
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >>
>>>>>>
16:38:26,231::connection::329::root::(_add_connection_callback)
>>>>>> >> Connection adding
succeeded:
>>>>>> dev=ovirtmgmt
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >>
>>>>>>
16:38:26,232::nmclient::136::root::(execute_next_action)
>>>>>> Executing NM
>>>>>> >> action:
func=commit_changes_async
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::ERROR::2020-09-02
>>>>>> >>
>>>>>>
16:38:26,233::nmclient::200::root::(quit)
>>>>>> NM main-loop aborted:
>>>>>> >> Connection update
failed:
>>>>>> error=nm-connection-error-quark:
>>>>>> >> ipv6.gateway: gateway
cannot
>>>>>> be set if there are no addresses
>>>>>> >> configured (7),
>>>>>> dev=enp4s0/<enum
>>>>>> NM_DEVICE_STATE_ACTIVATED of
type
>>>>>> >> NM.DeviceState>
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >>
>>>>>>
16:38:26,246::checkpoint::164::root::(rollback)
>>>>>> Checkpoint
>>>>>> >>
>>>>>>
/org/freedesktop/NetworkManager/Checkpoint/1
>>>>>> rollback executed:
>>>>>> >>
>>>>>>
dbus.Dictionary({dbus.String('/org/freedesktop/NetworkManager/Devices/6'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >>
>>>>>>
dbus.String('/org/freedesktop/NetworkManager/Devices/7'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >>
>>>>>>
dbus.String('/org/freedesktop/NetworkManager/Devices/8'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >>
>>>>>>
dbus.String('/org/freedesktop/NetworkManager/Devices/9'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >>
>>>>>>
dbus.String('/org/freedesktop/NetworkManager/Devices/3'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >>
>>>>>>
dbus.String('/org/freedesktop/NetworkManager/Devices/2'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >>
>>>>>>
dbus.String('/org/freedesktop/NetworkManager/Devices/4'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >>
>>>>>>
dbus.String('/org/freedesktop/NetworkManager/Devices/1'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >>
>>>>>>
dbus.String('/org/freedesktop/NetworkManager/Devices/5'):
>>>>>> >> dbus.UInt32(0)},
>>>>>>
signature=dbus.Signature('su'))
>>>>>> >>
>>>>>>
MainProcess|jsonrpc/0::ERROR::2020-09-02
>>>>>> >>
>>>>>>
16:38:31,251::supervdsm_server::97::SuperVdsm.ServerCallback::(wrapper)
>>>>>> >> Error in setupNetworks
>>>>>> >> Traceback (most recent
call
>>>>>> last):
>>>>>> >> File
>>>>>>
"/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py",
>>>>>> >> line 95, in wrapper
>>>>>> >> res = func(*args,
**kwargs)
>>>>>> >> File
>>>>>>
"/usr/lib/python3.6/site-packages/vdsm/network/api.py",
>>>>>> line
>>>>>> >> 241, in setupNetworks
>>>>>> >>
_setup_networks(networks,
>>>>>> bondings, options, net_info)
>>>>>> >> File
>>>>>>
"/usr/lib/python3.6/site-packages/vdsm/network/api.py",
>>>>>> line
>>>>>> >> 266, in _setup_networks
>>>>>> >> networks, bondings,
options,
>>>>>> net_info, in_rollback
>>>>>> >> File
>>>>>> >>
>>>>>>
"/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py",
>>>>>> >> line 154, in setup
>>>>>> >>
_setup_nmstate(networks,
>>>>>> bondings, options, in_rollback)
>>>>>> >> File
>>>>>> >>
>>>>>>
"/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py",
>>>>>> >> line 199, in
_setup_nmstate
>>>>>> >>
nmstate.setup(desired_state,
>>>>>> verify_change=not in_rollback)
>>>>>> >> File
>>>>>>
"/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py",
>>>>>> >> line 63, in setup
>>>>>> >>
state_apply(desired_state,
>>>>>> verify_change=verify_change)
>>>>>> >> File
>>>>>>
"/usr/lib/python3.6/site-packages/libnmstate/deprecation.py",
>>>>>> >> line 40, in wrapper
>>>>>> >> return func(*args,
**kwargs)
>>>>>> >> File
>>>>>>
"/usr/lib/python3.6/site-packages/libnmstate/nm/nmclient.py",
>>>>>> >> line 96, in wrapped
>>>>>> >> ret = func(*args,
**kwargs)
>>>>>> >> File
>>>>>>
"/usr/lib/python3.6/site-packages/libnmstate/netapplier.py",
>>>>>> >> line 73, in apply
>>>>>> >>
state.State(desired_state),
>>>>>> verify_change, commit,
>>>>>> rollback_timeout
>>>>>> >> File
>>>>>>
"/usr/lib/python3.6/site-packages/libnmstate/netapplier.py",
>>>>>> >> line 163, in
_apply_ifaces_state
>>>>>> >>
>>>>>> con_profiles=ifaces_add_configs
>>>>>> + ifaces_edit_configs,
>>>>>> >> File
>>>>>>
"/usr/lib64/python3.6/contextlib.py",
>>>>>> line 88, in __exit__
>>>>>> >> next(self.gen)
>>>>>> >> File
>>>>>>
"/usr/lib/python3.6/site-packages/libnmstate/netapplier.py",
>>>>>> >> line 232, in
_setup_providers
>>>>>> >>
>>>>>>
mainloop.run(timeout=MAINLOOP_TIMEOUT)
>>>>>> >> File
>>>>>>
"/usr/lib/python3.6/site-packages/libnmstate/nm/nmclient.py",
>>>>>> >> line 177, in run
>>>>>> >> f"Unexpected
failure of libnm
>>>>>> when running the mainloop:
{err}"
>>>>>> >>
>>>>>>
libnmstate.error.NmstateLibnmError:
>>>>>> Unexpected failure of libnm when
>>>>>> >> running the mainloop:
run
>>>>>> execution
>>>>>> >>
>>>>>> >> My host have a main
network
>>>>>> interface and a vlan interface
>>>>>> on the same
>>>>>> >> physical interface which
is
>>>>>> an internal management network
which
>>>>>> >> should be used as
management
>>>>>> network. The vlan interface
>>>>>> doesn't have
>>>>>> >> any ipv6 interface, but
>>>>>> nmclient fails on setting ipv6
>>>>>> gateway. This
>>>>>> >> seems like a bug?
>>>>>> >>
>>>>>> >> /Sverker
>>>>>> >>
>>>>>> >>
>>>>>>
_______________________________________________
>>>>>> >> discuss mailing list
>>>>>> >> discuss(a)openvswitch.org
>>>>>>
<mailto:discuss@openvswitch.org>
>>>>>> >>
>>>>>>
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>>>>>> >>
>>>>>>
_______________________________________________
>>>>>> >> Users mailing list --
>>>>>> users(a)ovirt.org
>>>>>> <mailto:users@ovirt.org>
>>>>>> >> To unsubscribe send an
email
>>>>>> to users-leave(a)ovirt.org
>>>>>>
<mailto:users-leave@ovirt.org>
>>>>>> >> Privacy Statement:
>>>>>>
https://www.ovirt.org/privacy-policy.html
>>>>>> >> oVirt Code of Conduct:
>>>>>> >>
>>>>>>
https://www.ovirt.org/community/about/community-guidelines/
>>>>>> >> List Archives:
>>>>>> >>
>>>>>>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/R4YSTOQM7I4...
>>>>>> >
>>>>>>
_______________________________________________
>>>>>> > Users mailing list --
>>>>>> users(a)ovirt.org
>>>>>> <mailto:users@ovirt.org>
>>>>>> > To unsubscribe send an
email
>>>>>> to users-leave(a)ovirt.org
>>>>>>
<mailto:users-leave@ovirt.org>
>>>>>> > Privacy Statement:
>>>>>>
https://www.ovirt.org/privacy-policy.html
>>>>>> > oVirt Code of Conduct:
>>>>>>
https://www.ovirt.org/community/about/community-guidelines/
>>>>>> > List Archives:
>>>>>> >
>>>>>>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/UMXFPVLJHYO...
>>>>>> >
>>>>>>
_______________________________________________
>>>>>> Users mailing list --
>>>>>> users(a)ovirt.org
>>>>>> <mailto:users@ovirt.org>
>>>>>> To unsubscribe send an email to
>>>>>> users-leave(a)ovirt.org
>>>>>>
<mailto:users-leave@ovirt.org>
>>>>>> Privacy Statement:
>>>>>>
https://www.ovirt.org/privacy-policy.html
>>>>>> oVirt Code of Conduct:
>>>>>>
https://www.ovirt.org/community/about/community-guidelines/
>>>>>> List Archives:
>>>>>>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/RQIPLZGVOHX...
>>>>>>
>>>>>
_______________________________________________
>>>>> Users mailing list -- users(a)ovirt.org
>>>>> <mailto:users@ovirt.org>
>>>>> To unsubscribe send an email to
>>>>> users-leave(a)ovirt.org
>>>>> <mailto:users-leave@ovirt.org>
>>>>> Privacy Statement:
>>>>>
https://www.ovirt.org/privacy-policy.html
>>>>> oVirt Code of Conduct:
>>>>>
https://www.ovirt.org/community/about/community-guidelines/
>>>>> List Archives:
>>>>>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/ED2YKQAKVG6...
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Ales Musil
>>>>>
>>>>> Software Engineer - RHV Network
>>>>>
>>>>> Red Hat EMEA <
https://www.redhat.com>
>>>>>
>>>>> amusil(a)redhat.com
>>>>> <mailto:amusil@redhat.com> IM: amusil
>>>>>
>>>>> <
https://red.ht/sig>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Ales Musil
>>>>
>>>> Software Engineer - RHV Network
>>>>
>>>> Red Hat EMEA <
https://www.redhat.com>
>>>>
>>>> amusil(a)redhat.com <mailto:amusil@redhat.com>
>>>> IM: amusil
>>>>
>>>> <
https://red.ht/sig>
>>>>
--
Ales Musil
Software Engineer - RHV Network
Red Hat EMEA <
https://www.redhat.com>
amusil(a)redhat.com <mailto:amusil@redhat.com> IM: amusil
<
https://red.ht/sig>