Marko, Joey, Zhenyu, maybe worth preparing a feature proposal for oVirt 4.5.0 for aarch64 support (see https://github.com/oVirt/ovirt-site/pull/2549 as example)

Il giorno gio 15 lug 2021 alle ore 10:38 Marko Vrgotic <M.Vrgotic@activevideo.com> ha scritto:

Thank you all.

 

-----

kind regards/met vriendelijke groeten

 

Marko Vrgotic
Sr. System Engineer @ System Administration


ActiveVideo

o: +31 (35) 6774131

m: +31 (65) 5734174

e: m.vrgotic@activevideo.com
w: www.activevideo.com

 

ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited.  If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.

 

 

 

From: Arik Hadas <ahadas@redhat.com>
Date: Wednesday, 14 July 2021 at 15:26
To: Milan Zamazal <mzamazal@redhat.com>
Cc: Marko Vrgotic <M.Vrgotic@activevideo.com>, Sandro Bonazzola <sbonazzo@redhat.com>, Evgheni Dereveanchin <ederevea@redhat.com>, Zhenyu Zheng <zhengzhenyulixi@gmail.com>, Joey Ma <majunjiev@gmail.com>, users@ovirt.org <users@ovirt.org>
Subject: Re: oVirt and ARM

***CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender!!!***

 

 

On Wed, Jul 14, 2021 at 3:14 PM Milan Zamazal <mzamazal@redhat.com> wrote:

Arik Hadas <ahadas@redhat.com> writes:

> On Wed, Jul 14, 2021 at 10:36 AM Milan Zamazal <mzamazal@redhat.com> wrote:
>
>> Marko Vrgotic <M.Vrgotic@activevideo.com> writes:
>>
>> > Dear Arik and Milan,
>> >
>> > In the meantime, I was asked to check if in current 4.4 version or
>> > coming 4.5, are/will there any capabilities or options of emulating
>> > aarch64 on x86_64 platform and if so, what would be the steps to
>> > test/enable it.
>> >
>> > Can you provide some information?
>>
>> Hi Marko,
>>
>> I don't think there is a way to emulate a non-native architecture.
>> Engine doesn't have ARM support and it cannot handle ARM (native or
>> emulated) hosts.  You could try to run emulated ARM VMs presented as x86
>> to Engine using Vdsm hooks but I doubt it would work.
>>
>
> Oh I just sent a draft I had in my mailbox without noticing this comment
> and I see we both mentioned Vdsm hook
> What is the source of the doubts about Vdsm hooks to work for this?

It's possible to override the domain XML obtained from Engine to change
it from x86 to ARM but Engine will get back the non-x86 domain XML.
Engine may not care about the emulator but perhaps it can be confused by
the reported CPU etc.  There can be problems with devices and
architecture specific settings in both the directions.  Engine will base
assumptions about the VM capabilities based on x86, which won't exactly
match ARM.

 

Yeah, it would be best to try and see what the misalignment between what the engine thinks and what the VM runs with leads to

But from the top of my head I don't think a misalignment in regards to CPU or capabilities could be problematic since the engine doesn't check what the VM is actually set with, it just assumes that the VM is set with what it wrote to its domain XML

The engine certainly looks at the devices but as long as they preserve their user-alias, managed devices won't be unplugged. Other devices would be added as unmanaged. So I think that should also be fine.

 


Maybe it'd be possible to simply run and stop a VM with some effort and
it would be enough for certain purposes.  But for more than that it's a
question whether the effort would be better spent on implementing a
proper architecture support. 


>> I'm afraid the only way is to add ARM support to oVirt.  My former
>> colleague has played with running oVirt on Raspberry Pi hosts some years
>> ago (there are traces of that effort in Vdsm) and I think adding ARM
>> support should be, at least in theory, possible.  Particular features
>> available would be mostly dependent on ARM support in QEMU and libvirt.
>
>
>> Regards,
>> Milan
>>
>> > -----
>> > kind regards/met vriendelijke groeten
>> >
>> > Marko Vrgotic
>> > Sr. System Engineer @ System Administration
>> >
>> > ActiveVideo
>> > o: +31 (35) 6774131
>> > m: +31 (65) 5734174
>> > e: m.vrgotic@activevideo.com<mailto:m.vrgotic@activevideo.com>
>> > w: www.activevideo.com<http://www.activevideo.com>
>> >
>> > ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein
>> > 1.1217 WJ Hilversum, The Netherlands. The information contained in
>> > this message may be legally privileged and confidential. It is
>> > intended to be read only by the individual or entity to whom it is
>> > addressed or by their designee. If the reader of this message is not
>> > the intended recipient, you are on notice that any distribution of
>> > this message, in any form, is strictly prohibited.  If you have
>> > received this message in error, please immediately notify the sender
>> > and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and
>> > delete or destroy any copy of this message.
>> >
>> >
>> >
>> > From: Sandro Bonazzola <sbonazzo@redhat.com>
>> > Date: Friday, 9 July 2021 at 15:37
>> > To: Marko Vrgotic <M.Vrgotic@activevideo.com>, Arik Hadas
>> > <ahadas@redhat.com>, Milan Zamazal <mzamazal@redhat.com>
>> > Cc: Evgheni Dereveanchin <ederevea@redhat.com>, Zhenyu Zheng
>> > <zhengzhenyulixi@gmail.com>, Joey Ma <majunjiev@gmail.com>,
>> > users@ovirt.org <users@ovirt.org>
>> > Subject: Re: [ovirt-users] oVirt and ARM
>> >
>> > ***CAUTION: This email originated from outside of the organization. Do
>> > not click links or open attachments unless you recognize the
>> > sender!!!***
>> >
>> >
>> > Il giorno ven 9 lug 2021 alle ore 11:00 Marko Vrgotic
>> > <M.Vrgotic@activevideo.com<mailto:M.Vrgotic@activevideo.com>> ha
>> > scritto:
>> > Hi Sandro and the rest of oVirt gurus,
>> >
>> > My managers are positive regarding helping provide some ARM hardware,
>> > but it would not happened earlier than three months from now, as we
>> > are in process of establishing certain relationship with ARM HW
>> > vendor.
>> >
>> > T news!
>> >
>> >
>> > In the meantime, I was asked to check if in current 4.4 version or
>> > coming 4.5, are/will there any capabilities or options of emulating
>> > aarch64 on x86_64 platform and if so, what would be the steps to
>> > test/enable it.
>> >
>> > +Arik Hadas<mailto:ahadas@redhat.com> , +Milan Zamazal<mailto:
>> mzamazal@redhat.com> ?
>> >
>> >
>> > Kindly awaiting your reply.
>> >
>> > Marko Vrgotic
>> >
>> > From: Marko Vrgotic <M.Vrgotic@activevideo.com<mailto:
>> M.Vrgotic@activevideo.com>>
>> > Date: Monday, 28 June 2021 at 15:38
>> > To: Sandro Bonazzola
>> > <sbonazzo@redhat.com<mailto:sbonazzo@redhat.com>>, Evgheni
>> > Dereveanchin <ederevea@redhat.com<mailto:ederevea@redhat.com>>
>> > Cc: Zhenyu Zheng
>> > <zhengzhenyulixi@gmail.com<mailto:zhengzhenyulixi@gmail.com>>, Joey Ma
>> > <majunjiev@gmail.com<mailto:majunjiev@gmail.com>>,
>> > users@ovirt.org<mailto:users@ovirt.org>
>> > <users@ovirt.org<mailto:users@ovirt.org>>
>> > Subject: Re: [ovirt-users] oVirt and ARM
>> > Hi Sandro,
>> >
>> > I will check with my managers if we have and could spare some hardware
>> > to contribute developing for oVirt.
>> >
>> >
>> > -----
>> > kind regards/met vriendelijke groeten
>> >
>> > Marko Vrgotic
>> > Sr. System Engineer @ System Administration
>> >
>> > ActiveVideo
>> > o: +31 (35) 6774131
>> > m: +31 (65) 5734174
>> > e: m.vrgotic@activevideo.com<mailto:m.vrgotic@activevideo.com>
>> > w: www.activevideo.com<http://www.activevideo.com>
>> >
>> > ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein
>> > 1.1217 WJ Hilversum, The Netherlands. The information contained in
>> > this message may be legally privileged and confidential. It is
>> > intended to be read only by the individual or entity to whom it is
>> > addressed or by their designee. If the reader of this message is not
>> > the intended recipient, you are on notice that any distribution of
>> > this message, in any form, is strictly prohibited.  If you have
>> > received this message in error, please immediately notify the sender
>> > and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and
>> > delete or destroy any copy of this message.
>> >
>> >
>> >
>> > From: Sandro Bonazzola <sbonazzo@redhat.com<mailto:sbonazzo@redhat.com>>
>> > Date: Friday, 25 June 2021 at 15:26
>> > To: Marko Vrgotic
>> > <M.Vrgotic@activevideo.com<mailto:M.Vrgotic@activevideo.com>>, Evgheni
>> > Dereveanchin <ederevea@redhat.com<mailto:ederevea@redhat.com>>
>> > Cc: Zhenyu Zheng
>> > <zhengzhenyulixi@gmail.com<mailto:zhengzhenyulixi@gmail.com>>, Joey Ma
>> > <majunjiev@gmail.com<mailto:majunjiev@gmail.com>>,
>> > users@ovirt.org<mailto:users@ovirt.org>
>> > <users@ovirt.org<mailto:users@ovirt.org>>
>> > Subject: Re: [ovirt-users] oVirt and ARM
>> >
>> > ***CAUTION: This email originated from outside of the organization. Do
>> > not click links or open attachments unless you recognize the
>> > sender!!!***
>> >
>> >
>> > Il giorno ven 25 giu 2021 alle ore 14:20 Marko Vrgotic
>> > <M.Vrgotic@activevideo.com<mailto:M.Vrgotic@activevideo.com>> ha
>> > scritto:
>> > Hi Sandro,
>> >
>> > Thank you for the update. I am not equipped to help on development
>> > side, but I can most certainly do test deployments, once there is
>> > something available.
>> >
>> > We are big oVirt shop and moving to ARM64 with new product, it would
>> > be great if oVirt would start supporting it.
>> >
>> > If we are able to help somehow, let me know.
>> >
>> > I guess a start could be adding some arm64 machine to oVirt
>> > infrastructure so developers can build for it.
>> > You can have a look at
>> > https://ovirt.org/community/get-involved/donate-hardware.html<
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fovirt.org%2Fcommunity%2Fget-involved%2Fdonate-hardware.html&data=04%7C01%7Cm.vrgotic%40activevideo.com%7C0339802ecf2947fd60b208d942dea85e%7C214268a3e1214486acd4545c9faf2252%7C0%7C0%7C637614346362967472%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=kEpGhQOJV0eCwI%2BrorwWPIQvReC7f8Tw8KL82HjKVZQ%3D&reserved=0
>> >
>> > Looping in +Evgheni Dereveanchin<mailto:ederevea@redhat.com> in case
>> you can share some resources.
>> >
>> >
>> >
>> >
>> > -----
>> > kind regards/met vriendelijke groeten
>> >
>> > Marko Vrgotic
>> > Sr. System Engineer @ System Administration
>> >
>> > ActiveVideo
>> > e: m.vrgotic@activevideo.com<mailto:m.vrgotic@activevideo.com>
>> > w: www.activevideo.com<http://www.activevideo.com>
>> >
>> > ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein
>> > 1.1217 WJ Hilversum, The Netherlands. The information contained in
>> > this message may be legally privileged and confidential. It is
>> > intended to be read only by the individual or entity to whom it is
>> > addressed or by their designee. If the reader of this message is not
>> > the intended recipient, you are on notice that any distribution of
>> > this message, in any form, is strictly prohibited.  If you have
>> > received this message in error, please immediately notify the sender
>> > and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and
>> > delete or destroy any copy of this message.
>> >
>> >
>> >
>> > From: Sandro Bonazzola <sbonazzo@redhat.com<mailto:sbonazzo@redhat.com>>
>> > Date: Thursday, 24 June 2021 at 18:21
>> > To: Marko Vrgotic
>> > <M.Vrgotic@activevideo.com<mailto:M.Vrgotic@activevideo.com>>, Zhenyu
>> > Zheng <zhengzhenyulixi@gmail.com<mailto:zhengzhenyulixi@gmail.com>>,
>> > Joey Ma <majunjiev@gmail.com<mailto:majunjiev@gmail.com>>
>> > Cc: users@ovirt.org<mailto:users@ovirt.org> <users@ovirt.org<mailto:
>> users@ovirt.org>>
>> > Subject: Re: [ovirt-users] oVirt and ARM
>> >
>> > ***CAUTION: This email originated from outside of the organization. Do
>> > not click links or open attachments unless you recognize the
>> > sender!!!***
>> >
>> >
>> > Il giorno gio 24 giu 2021 alle ore 16:34 Marko Vrgotic
>> > <M.Vrgotic@activevideo.com<mailto:M.Vrgotic@activevideo.com>> ha
>> > scritto:
>> > Hi oVirt,
>> >
>> > Where can I find if there are any information about oVirt supporting
>> arm64 CPU architecture?
>> >
>> > Right now oVirt is not supporting arm64. There was an initiative about
>> > supporting it started some time ago from openEuler ovirt SIG.
>> > I didn't got any further updates on this topic, looping in those who I
>> remember being looking into it.
>> > I think that if someone contributes arm64 support it would also be a
>> feature worth a 4.5 release :-)
>> >
>> >
>> > -----
>> > kind regards/met vriendelijke groeten
>> >
>> > Marko Vrgotic
>> > Sr. System Engineer @ System Administration
>> >
>> > ActiveVideo
>> > o: +31 (35) 6774131
>> > m: +31 (65) 5734174
>> > e: m.vrgotic@activevideo.com<mailto:m.vrgotic@activevideo.com>
>> > w: www.activevideo.com<http://www.activevideo.com>
>> >
>> > ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein
>> > 1.1217 WJ Hilversum, The Netherlands. The information contained in
>> > this message may be legally privileged and confidential. It is
>> > intended to be read only by the individual or entity to whom it is
>> > addressed or by their designee. If the reader of this message is not
>> > the intended recipient, you are on notice that any distribution of
>> > this message, in any form, is strictly prohibited.  If you have
>> > received this message in error, please immediately notify the sender
>> > and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and
>> > delete or destroy any copy of this message.
>> >
>> >
>> > _______________________________________________
>> > Users mailing list -- users@ovirt.org<mailto:users@ovirt.org>
>> > To unsubscribe send an email to users-leave@ovirt.org<mailto:
>> users-leave@ovirt.org>
>> > Privacy Statement:
>> > https://www.ovirt.org/privacy-policy.html<
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ovirt.org%2Fprivacy-policy.html&data=04%7C01%7Cm.vrgotic%40activevideo.com%7C0339802ecf2947fd60b208d942dea85e%7C214268a3e1214486acd4545c9faf2252%7C0%7C0%7C637614346362977427%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=fCfm3M7tegII2jbEbXrws1lPuLkf9BFM6A9eEye%2Fixw%3D&reserved=0
>> >
>> > oVirt Code of Conduct:
>> > https://www.ovirt.org/community/about/community-guidelines/<
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ovirt.org%2Fcommunity%2Fabout%2Fcommunity-guidelines%2F&data=04%7C01%7Cm.vrgotic%40activevideo.com%7C0339802ecf2947fd60b208d942dea85e%7C214268a3e1214486acd4545c9faf2252%7C0%7C0%7C637614346362977427%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W2%2BL3NWoC9%2FnUg51S95nrP1phhxJxba3Q6SV3ErA30c%3D&reserved=0
>> >
>> > List Archives:
>> >
>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/MQ3XND2NKIL7VV4NLIC72NAZCJH4DGCB/
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.ovirt.org%2Farchives%2Flist%2Fusers%40ovirt.org%2Fmessage%2FMQ3XND2NKIL7VV4NLIC72NAZCJH4DGCB%2F&data=04%7C01%7Cm.vrgotic%40activevideo.com%7C0339802ecf2947fd60b208d942dea85e%7C214268a3e1214486acd4545c9faf2252%7C0%7C0%7C637614346362987392%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=K%2Far3yiwlHgMv5DDpaASE16micKbc9T2z9uacUFbTkw%3D&reserved=0
>> >
>> >
>> >
>> > --
>> >
>> > Sandro Bonazzola
>> >
>> > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
>> >
>> > Red Hat
>> > EMEA<
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2F&data=04%7C01%7Cm.vrgotic%40activevideo.com%7C0339802ecf2947fd60b208d942dea85e%7C214268a3e1214486acd4545c9faf2252%7C0%7C0%7C637614346362987392%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PwOF8E7JZR%2B7vAYcCTzD7oM1bymoBXhJvyo088TE0wQ%3D&reserved=0
>> >
>> >
>> > sbonazzo@redhat.com<mailto:sbonazzo@redhat.com>
>> > Error! Filename not
>> > specified.<
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2F&data=04%7C01%7Cm.vrgotic%40activevideo.com%7C0339802ecf2947fd60b208d942dea85e%7C214268a3e1214486acd4545c9faf2252%7C0%7C0%7C637614346362997339%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ykfRg5DoW8rsPfoKy%2FxWJ0UQcCCRw9v1HUyJQiRkMGg%3D&reserved=0
>> >
>> > Red Hat respects your work life balance. Therefore there is no need to
>> > answer this email out of your office hours.
>> >
>> >
>> >
>> > --
>> >
>> > Sandro Bonazzola
>> >
>> > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
>> >
>> > Red Hat
>> > EMEA<
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2F&data=04%7C01%7Cm.vrgotic%40activevideo.com%7C0339802ecf2947fd60b208d942dea85e%7C214268a3e1214486acd4545c9faf2252%7C0%7C0%7C637614346362997339%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ykfRg5DoW8rsPfoKy%2FxWJ0UQcCCRw9v1HUyJQiRkMGg%3D&reserved=0
>> >
>> >
>> > sbonazzo@redhat.com<mailto:sbonazzo@redhat.com>
>> > Error! Filename not
>> > specified.<
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2F&data=04%7C01%7Cm.vrgotic%40activevideo.com%7C0339802ecf2947fd60b208d942dea85e%7C214268a3e1214486acd4545c9faf2252%7C0%7C0%7C637614346363007298%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=oqZUeFYvcmmWvpqxS8rXiUY7bbUpquPleuApFMEGnt4%3D&reserved=0
>> >
>> > Red Hat respects your work life balance. Therefore there is no need to
>> > answer this email out of your office hours.
>> >
>> >
>> >
>> > --
>> >
>> > Sandro Bonazzola
>> >
>> > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
>> >
>> > Red Hat EMEA<
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2F&data=04%7C01%7Cm.vrgotic%40activevideo.com%7C0339802ecf2947fd60b208d942dea85e%7C214268a3e1214486acd4545c9faf2252%7C0%7C0%7C637614346363007298%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=oqZUeFYvcmmWvpqxS8rXiUY7bbUpquPleuApFMEGnt4%3D&reserved=0
>> >
>> >
>> > sbonazzo@redhat.com<mailto:sbonazzo@redhat.com>
>> > [https://static.redhat.com/libs/redhat/brand-assets/2/corp/logo--200.png
>> ]<
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2F&data=04%7C01%7Cm.vrgotic%40activevideo.com%7C0339802ecf2947fd60b208d942dea85e%7C214268a3e1214486acd4545c9faf2252%7C0%7C0%7C637614346363017253%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zsygeDT1sfOZ23BaO9VTRB7spSXGzZOG1mo8C%2Bm0IuU%3D&reserved=0
>> >
>> > Red Hat respects your work life balance. Therefore there is no need to
>> answer this email out of your office hours.
>> > <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmojo.redhat.com%2Fdocs%2FDOC-1199578&data=04%7C01%7Cm.vrgotic%40activevideo.com%7C0339802ecf2947fd60b208d942dea85e%7C214268a3e1214486acd4545c9faf2252%7C0%7C0%7C637614346363017253%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=haQxDQr6uKUEqAgZUbEEhLOeKfVPVaoy8Wl2JgyPvRs%3D&reserved=0
>> >
>>
>>



--