[Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
Hi All, Our second approach for trusted compute pools integration with oVirt seems more simple and sensible than previous VM level approach. Welcome any comments on our latest design. Thanks in advance. Link is here, http://www.ovirt.org/Trusted_compute_pools Best Regards, Dave Chen
On 04/19/2013 12:21 PM, Chen, Wei D wrote:
Hi All,
Our second approach for trusted compute pools integration with oVirt seems more simple and sensible than previous VM level approach. Welcome any comments on our latest design. Thanks in advance.
Link is here, http://www.ovirt.org/Trusted_compute_pools
a few nits: 1. last updated date isn't updated... 2. from reading it top to bottom, hard to understand the 2nd approach is the one to be used and not the first (vm level). 3. cluster dialog - the 'trusted' should be a checkbox, not radio button, and should only be enabled if virt service was chosen. thanks, Itamar
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Wei D Chen" <wei.d.chen@intel.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, "engine-devel@ovirt.org" <engine-devel@ovirt.org> Sent: Saturday, April 20, 2013 5:49:47 PM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
On 04/19/2013 12:21 PM, Chen, Wei D wrote:
Hi All,
Our second approach for trusted compute pools integration with oVirt seems more simple and sensible than previous VM level approach. Welcome any comments on our latest design. Thanks in advance.
Link is here, http://www.ovirt.org/Trusted_compute_pools
a few nits: 1. last updated date isn't updated... 2. from reading it top to bottom, hard to understand the 2nd approach is the one to be used and not the first (vm level). 3. cluster dialog - the 'trusted' should be a checkbox, not radio button, and should only be enabled if virt service was chosen.
I'd also consider putting this property in a different side tab. Perhaps "Cluster policy" side tab would fit? (dividing it into two sections "scheduling policy" and "additional properties" or something similar. What do you think about that?
thanks, Itamar _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Oved Ourfalli" <ovedo@redhat.com> To: "Itamar Heim" <iheim@redhat.com>, "Wei D Chen" <wei.d.chen@intel.com> Cc: engine-devel@ovirt.org Sent: Sunday, April 21, 2013 8:41:50 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Wei D Chen" <wei.d.chen@intel.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, "engine-devel@ovirt.org" <engine-devel@ovirt.org> Sent: Saturday, April 20, 2013 5:49:47 PM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
On 04/19/2013 12:21 PM, Chen, Wei D wrote:
Hi All,
Our second approach for trusted compute pools integration with oVirt seems more simple and sensible than previous VM level approach. Welcome any comments on our latest design. Thanks in advance.
Link is here, http://www.ovirt.org/Trusted_compute_pools
a few nits: 1. last updated date isn't updated... 2. from reading it top to bottom, hard to understand the 2nd approach is the one to be used and not the first (vm level). 3. cluster dialog - the 'trusted' should be a checkbox, not radio button, and should only be enabled if virt service was chosen.
I'd also consider putting this property in a different side tab. Perhaps "Cluster policy" side tab would fit? (dividing it into two sections "scheduling policy" and "additional properties" or something similar.
What do you think about that?
thanks, Itamar
Hi, One more thing we need to think about for the second approach - aggregated query. On engine start we need to determine the trust state of all the hosts. sending a separate query for each host will overload the attestation host and the network. an initial aggregated query needs to be send when the engine starts. Same thing can happen after management network fail and so on. Maybe we can run a quartz job every x minutes, checking if a large part of the hosts in the cluster (like 30%) are untrusted - in that case run the aggregated query. Ofri
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 04/21/2013 10:13 AM, Ofri Masad wrote:
Hi, One more thing we need to think about for the second approach - aggregated query. On engine start we need to determine the trust state of all the hosts. sending a separate query for each host will overload the attestation host and the network. an initial aggregated query needs to be send when the engine starts. Same thing can happen after management network fail and so on. Maybe we can run a quartz job every x minutes, checking if a large part of the hosts in the cluster (like 30%) are untrusted - in that case run the aggregated query.
are we sure this optimization is needed? how heavy/latent is the call to the attestation service?
Dave, If I'm not mistaking, there is a big difference between separated queries to the attestation server and aggregated one? Is it true? Thanks, Ofri ----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Ofri Masad" <omasad@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, "Wei D Chen" <wei.d.chen@intel.com>, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 10:20:17 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
On 04/21/2013 10:13 AM, Ofri Masad wrote:
Hi, One more thing we need to think about for the second approach - aggregated query. On engine start we need to determine the trust state of all the hosts. sending a separate query for each host will overload the attestation host and the network. an initial aggregated query needs to be send when the engine starts. Same thing can happen after management network fail and so on. Maybe we can run a quartz job every x minutes, checking if a large part of the hosts in the cluster (like 30%) are untrusted - in that case run the aggregated query.
are we sure this optimization is needed? how heavy/latent is the call to the attestation service?
Ofri, Absolutely right, aggregated query has a significantly time improve compared to separated queries. I agree a aggregated query on engine's starting. Is it possible to invoke attestation service in engine's initialization code block instead of "quartz job"? Is there any class similar with " InitVdsOnUpCommand " for engine's initialization? Best Regards, Dave Chen -----Original Message----- From: Ofri Masad [mailto:omasad@redhat.com] Sent: Sunday, April 21, 2013 3:29 PM To: Chen, Wei D Cc: Oved Ourfalli; engine-devel@ovirt.org; Itamar Heim Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated Dave, If I'm not mistaking, there is a big difference between separated queries to the attestation server and aggregated one? Is it true? Thanks, Ofri ----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Ofri Masad" <omasad@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, "Wei D Chen" <wei.d.chen@intel.com>, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 10:20:17 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
On 04/21/2013 10:13 AM, Ofri Masad wrote:
Hi, One more thing we need to think about for the second approach - aggregated query. On engine start we need to determine the trust state of all the hosts. sending a separate query for each host will overload the attestation host and the network. an initial aggregated query needs to be send when the engine starts. Same thing can happen after management network fail and so on. Maybe we can run a quartz job every x minutes, checking if a large part of the hosts in the cluster (like 30%) are untrusted - in that case run the aggregated query.
are we sure this optimization is needed? how heavy/latent is the call to the attestation service?
----- Original Message -----
From: "Wei D Chen" <wei.d.chen@intel.com> To: "Ofri Masad" <omasad@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 4:00:55 PM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
Ofri,
Absolutely right, aggregated query has a significantly time improve compared to separated queries. I agree a aggregated query on engine's starting. Is it possible to invoke attestation service in engine's initialization code block instead of "quartz job"? Is there any class similar with " InitVdsOnUpCommand " for engine's initialization?
Best Regards, Dave Chen
org.ovirt.engine.core.bll.Backend.Initialize() Note you cannot block this method while waiting for results. Instead I suggest you fire a one-time background request from this method.
-----Original Message----- From: Ofri Masad [mailto:omasad@redhat.com] Sent: Sunday, April 21, 2013 3:29 PM To: Chen, Wei D Cc: Oved Ourfalli; engine-devel@ovirt.org; Itamar Heim Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
Dave,
If I'm not mistaking, there is a big difference between separated queries to the attestation server and aggregated one? Is it true?
Thanks, Ofri
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Ofri Masad" <omasad@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, "Wei D Chen" <wei.d.chen@intel.com>, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 10:20:17 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
On 04/21/2013 10:13 AM, Ofri Masad wrote:
Hi, One more thing we need to think about for the second approach - aggregated query. On engine start we need to determine the trust state of all the hosts. sending a separate query for each host will overload the attestation host and the network. an initial aggregated query needs to be send when the engine starts. Same thing can happen after management network fail and so on. Maybe we can run a quartz job every x minutes, checking if a large part of the hosts in the cluster (like 30%) are untrusted - in that case run the aggregated query.
are we sure this optimization is needed? how heavy/latent is the call to the attestation service?
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Hi, Our current consideration is add a new thread in engine's side to attest all of hosts (aggregated query from attestation sever) one time in case of engine's rebooting. There is still one potential issue under extreme condition, saying, hundreds of nodes in a datacenter, attest all of hosts still may take couple of mins, let's say, one hacked untrusted node before receiving the latest status may considered as a trusted host, so, the worst case in a datacenter with hundreds of nodes is, 1. engine is down for some reasons and boot up again, some trusted nodes may be hacked and rebooted during this period. 2. our thread is running to get all of node's status (trust /untrusted), may take couple of mins in large datacenter. 2. user create VMs on these hacked nodes and believe these VMs are trusted VMs launched on trusted nodes. 3. our thread get the correct status of these untrusted nodes, set these nodes as non-operational. 4. all of these "trusted" VMs running on these untrusted nodes are expected to migrate to other trusted node. So, the question is in a trusted cluster with hundreds of nodes some VMs expected to create on trusted nodes may actually create on untrusted nodes instead, and this time may last for couple of mins. (worst case in my view is 10 mins with 1000 nodes). Does this acceptable from your point of view? Or any other suggestion? Best Regards, Dave Chen
-----Original Message----- From: Doron Fediuck [mailto:dfediuck@redhat.com] Sent: Sunday, April 21, 2013 11:58 PM To: Chen, Wei D Cc: Ofri Masad; Oved Ourfalli; engine-devel@ovirt.org Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
----- Original Message -----
From: "Wei D Chen" <wei.d.chen@intel.com> To: "Ofri Masad" <omasad@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 4:00:55 PM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
Ofri,
Absolutely right, aggregated query has a significantly time improve compared to separated queries. I agree a aggregated query on engine's starting. Is it possible to invoke attestation service in engine's initialization code block instead of "quartz job"? Is there any class similar with " InitVdsOnUpCommand " for engine's initialization?
Best Regards, Dave Chen
org.ovirt.engine.core.bll.Backend.Initialize()
Note you cannot block this method while waiting for results. Instead I suggest you fire a one-time background request from this method.
-----Original Message----- From: Ofri Masad [mailto:omasad@redhat.com] Sent: Sunday, April 21, 2013 3:29 PM To: Chen, Wei D Cc: Oved Ourfalli; engine-devel@ovirt.org; Itamar Heim Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
Dave,
If I'm not mistaking, there is a big difference between separated queries to the attestation server and aggregated one? Is it true?
Thanks, Ofri
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Ofri Masad" <omasad@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, "Wei D Chen" <wei.d.chen@intel.com>, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 10:20:17 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
On 04/21/2013 10:13 AM, Ofri Masad wrote:
Hi, One more thing we need to think about for the second approach - aggregated query. On engine start we need to determine the trust state of all the hosts. sending a separate query for each host will overload the attestation host and the network. an initial aggregated query needs to be send when the engine starts. Same thing can happen after management network fail and so on. Maybe we can run a quartz job every x minutes, checking if a large part of the hosts in the cluster (like 30%) are untrusted - in that case run the aggregated query.
are we sure this optimization is needed? how heavy/latent is the call to the attestation service?
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Hi Dave, Just to make sure I fully understand, I'll repeat your basic arguments; 1. It takes time to query a big number of hosts (hundreds). 2. When backend is booting, a user may start a VM on a host which was hacked during the downtime of the engine. If the above is your concern, it shouldn't be so. The reason is, that no host will become operational before you get a response from the attestation server and allow it to become operational. So a user cannot start a new VM on a non-operational host. What this means is that your thread may need to update the user by sending a periodic event that a large scale attestation operation is in progress. Other than that, maybe your thread can work in smaller groups if it gets better results? ie- instead of one query for 300 hosts, maybe you can run 3 serialized queries for 100 hosts each? If this does not help, maybe you can run a short query for something like 10 hosts, which should get an answer relatively fast. The you can issue a query for the other 290 hosts which will take longer. In this way the system may get 10 hosts to work with quite fast, and later on the other 290 hosts will join... So this can actually be configurable to a 2-phase process; a short query and a longer one. The admin can choose the short query size based on his setup, and the longer query can pick up all the other hosts. What do you think? Doron ----- Original Message -----
From: "Wei D Chen" <wei.d.chen@intel.com> To: "Doron Fediuck" <dfediuck@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, engine-devel@ovirt.org Sent: Saturday, April 27, 2013 9:36:44 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
Hi,
Our current consideration is add a new thread in engine's side to attest all of hosts (aggregated query from attestation sever) one time in case of engine's rebooting. There is still one potential issue under extreme condition, saying, hundreds of nodes in a datacenter, attest all of hosts still may take couple of mins, let's say, one hacked untrusted node before receiving the latest status may considered as a trusted host, so, the worst case in a datacenter with hundreds of nodes is, 1. engine is down for some reasons and boot up again, some trusted nodes may be hacked and rebooted during this period. 2. our thread is running to get all of node's status (trust /untrusted), may take couple of mins in large datacenter. 2. user create VMs on these hacked nodes and believe these VMs are trusted VMs launched on trusted nodes. 3. our thread get the correct status of these untrusted nodes, set these nodes as non-operational. 4. all of these "trusted" VMs running on these untrusted nodes are expected to migrate to other trusted node.
So, the question is in a trusted cluster with hundreds of nodes some VMs expected to create on trusted nodes may actually create on untrusted nodes instead, and this time may last for couple of mins. (worst case in my view is 10 mins with 1000 nodes). Does this acceptable from your point of view? Or any other suggestion?
Best Regards, Dave Chen
-----Original Message----- From: Doron Fediuck [mailto:dfediuck@redhat.com] Sent: Sunday, April 21, 2013 11:58 PM To: Chen, Wei D Cc: Ofri Masad; Oved Ourfalli; engine-devel@ovirt.org Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
----- Original Message -----
From: "Wei D Chen" <wei.d.chen@intel.com> To: "Ofri Masad" <omasad@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 4:00:55 PM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
Ofri,
Absolutely right, aggregated query has a significantly time improve compared to separated queries. I agree a aggregated query on engine's starting. Is it possible to invoke attestation service in engine's initialization code block instead of "quartz job"? Is there any class similar with " InitVdsOnUpCommand " for engine's initialization?
Best Regards, Dave Chen
org.ovirt.engine.core.bll.Backend.Initialize()
Note you cannot block this method while waiting for results. Instead I suggest you fire a one-time background request from this method.
-----Original Message----- From: Ofri Masad [mailto:omasad@redhat.com] Sent: Sunday, April 21, 2013 3:29 PM To: Chen, Wei D Cc: Oved Ourfalli; engine-devel@ovirt.org; Itamar Heim Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
Dave,
If I'm not mistaking, there is a big difference between separated queries to the attestation server and aggregated one? Is it true?
Thanks, Ofri
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Ofri Masad" <omasad@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, "Wei D Chen" <wei.d.chen@intel.com>, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 10:20:17 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
On 04/21/2013 10:13 AM, Ofri Masad wrote:
Hi, One more thing we need to think about for the second approach - aggregated query. On engine start we need to determine the trust state of all the hosts. sending a separate query for each host will overload the attestation host and the network. an initial aggregated query needs to be send when the engine starts. Same thing can happen after management network fail and so on. Maybe we can run a quartz job every x minutes, checking if a large part of the hosts in the cluster (like 30%) are untrusted - in that case run the aggregated query.
are we sure this optimization is needed? how heavy/latent is the call to the attestation service?
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 04/28/2013 11:34 AM, Doron Fediuck wrote:
Hi Dave,
Just to make sure I fully understand, I'll repeat your basic arguments;
1. It takes time to query a big number of hosts (hundreds).
2. When backend is booting, a user may start a VM on a host which was hacked during the downtime of the engine.
If the above is your concern, it shouldn't be so. The reason is, that no host will become operational before you get a response from the attestation server and allow it to become operational. So a user cannot start a new VM on a non-operational host.
i'd do the queries in groups of "cluster", so cluste-by-cluster they get unblocked. cluster without attestation service shouldn't block on this of course.
What this means is that your thread may need to update the user by sending a periodic event that a large scale attestation operation is in progress. Other than that, maybe your thread can work in smaller groups if it gets better results? ie- instead of one query for 300 hosts, maybe you can run 3 serialized queries for 100 hosts each? If this does not help, maybe you can run a short query for something like 10 hosts, which should get an answer relatively fast. The you can issue a query for the other 290 hosts which will take longer. In this way the system may get 10 hosts to work with quite fast, and later on the other 290 hosts will join... So this can actually be configurable to a 2-phase process; a short query and a longer one. The admin can choose the short query size based on his setup, and the longer query can pick up all the other hosts. What do you think?
Doron
----- Original Message -----
From: "Wei D Chen" <wei.d.chen@intel.com> To: "Doron Fediuck" <dfediuck@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, engine-devel@ovirt.org Sent: Saturday, April 27, 2013 9:36:44 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
Hi,
Our current consideration is add a new thread in engine's side to attest all of hosts (aggregated query from attestation sever) one time in case of engine's rebooting. There is still one potential issue under extreme condition, saying, hundreds of nodes in a datacenter, attest all of hosts still may take couple of mins, let's say, one hacked untrusted node before receiving the latest status may considered as a trusted host, so, the worst case in a datacenter with hundreds of nodes is, 1. engine is down for some reasons and boot up again, some trusted nodes may be hacked and rebooted during this period. 2. our thread is running to get all of node's status (trust /untrusted), may take couple of mins in large datacenter. 2. user create VMs on these hacked nodes and believe these VMs are trusted VMs launched on trusted nodes. 3. our thread get the correct status of these untrusted nodes, set these nodes as non-operational. 4. all of these "trusted" VMs running on these untrusted nodes are expected to migrate to other trusted node.
So, the question is in a trusted cluster with hundreds of nodes some VMs expected to create on trusted nodes may actually create on untrusted nodes instead, and this time may last for couple of mins. (worst case in my view is 10 mins with 1000 nodes). Does this acceptable from your point of view? Or any other suggestion?
Best Regards, Dave Chen
-----Original Message----- From: Doron Fediuck [mailto:dfediuck@redhat.com] Sent: Sunday, April 21, 2013 11:58 PM To: Chen, Wei D Cc: Ofri Masad; Oved Ourfalli; engine-devel@ovirt.org Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
----- Original Message -----
From: "Wei D Chen" <wei.d.chen@intel.com> To: "Ofri Masad" <omasad@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 4:00:55 PM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
Ofri,
Absolutely right, aggregated query has a significantly time improve compared to separated queries. I agree a aggregated query on engine's starting. Is it possible to invoke attestation service in engine's initialization code block instead of "quartz job"? Is there any class similar with " InitVdsOnUpCommand " for engine's initialization?
Best Regards, Dave Chen
org.ovirt.engine.core.bll.Backend.Initialize()
Note you cannot block this method while waiting for results. Instead I suggest you fire a one-time background request from this method.
-----Original Message----- From: Ofri Masad [mailto:omasad@redhat.com] Sent: Sunday, April 21, 2013 3:29 PM To: Chen, Wei D Cc: Oved Ourfalli; engine-devel@ovirt.org; Itamar Heim Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
Dave,
If I'm not mistaking, there is a big difference between separated queries to the attestation server and aggregated one? Is it true?
Thanks, Ofri
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Ofri Masad" <omasad@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, "Wei D Chen" <wei.d.chen@intel.com>, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 10:20:17 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
On 04/21/2013 10:13 AM, Ofri Masad wrote:
Hi, One more thing we need to think about for the second approach - aggregated query. On engine start we need to determine the trust state of all the hosts. sending a separate query for each host will overload the attestation host and the network. an initial aggregated query needs to be send when the engine starts. Same thing can happen after management network fail and so on. Maybe we can run a quartz job every x minutes, checking if a large part of the hosts in the cluster (like 30%) are untrusted - in that case run the aggregated query.
are we sure this optimization is needed? how heavy/latent is the call to the attestation service?
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 04/28/2013 11:34 AM, Doron Fediuck wrote:
Hi Dave, =20 Just to make sure I fully understand, I'll repeat your basic = arguments; =20 1. It takes time to query a big number of hosts (hundreds). =20 2. When backend is booting, a user may start a VM on a host which was hacked during the downtime of the engine. =20 If the above is your concern, it shouldn't be so. The reason is, that no host will become operational before you get a response from the attestation server and allow it to become operational. So a = user cannot start a new VM on a non-operational host. =20 i'd do the queries in groups of "cluster", so cluste-by-cluster they = get unblocked. cluster without attestation service shouldn't block on this of course. =20 =20 What this means is that your thread may need to update the user by sending a periodic event that a large scale attestation operation is in =
Other than that, maybe your thread can work in smaller groups if it = gets better results? ie- instead of one query for 300 hosts, maybe you can = run 3 serialized queries for 100 hosts each? If this does not help, maybe you can run a short query for something =
10 hosts, which should get an answer relatively fast. The you can = issue a query for the other 290 hosts which will take longer. In this way the system may get 10 hosts to work with quite fast, and later on the other 290 hosts will join... So this can actually be configurable to a 2-phase =
a short query and a longer one. The admin can choose the short query = size based on his setup, and the longer query can pick up all the other = hosts. What do you think? =20 Doron =20 ----- Original Message -----
From: "Wei D Chen" <wei.d.chen@intel.com> To: "Doron Fediuck" <dfediuck@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, engine-devel@ovirt.org Sent: Saturday, April 27, 2013 9:36:44 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute =
integration with oVirt has been updated =20 Hi, =20 Our current consideration is add a new thread in engine's side to attest all of hosts (aggregated query from attestation sever) one = time in case of engine's rebooting. There is still one potential issue under extreme condition, saying, hundreds of nodes in a datacenter, attest all of hosts still may take couple of mins, let's say, one hacked untrusted node before receiving the latest status may considered as a trusted host, so, the worst case in a datacenter = with hundreds of nodes is, 1. engine is down for some reasons and boot up again, some trusted nodes may be hacked and rebooted during this period. 2. our thread is running to get all of node's status (trust /untrusted), may take couple of mins in large datacenter. 2. user create VMs on these hacked nodes and believe these VMs are trusted = VMs launched on trusted nodes. 3. our thread get the correct status of these untrusted nodes, set these nodes as non-operational. 4. all of these "trusted" VMs running on these untrusted nodes are expected to migrate to other trusted node. =20 So, the question is in a trusted cluster with hundreds of nodes some VMs expected to create on trusted nodes may actually create on untrusted nodes instead, and this time may last for couple of mins. (worst case in my view is 10 mins with 1000 nodes). Does this acceptable from your point of view? Or any other suggestion? =20 =20 Best Regards, Dave Chen =20 =20 Doron Fediuck wrote on 2013-04-21:
integration with oVirt has been updated =20 =20 =20 ----- Original Message -----
From: "Wei D Chen" <wei.d.chen@intel.com> To: "Ofri Masad" <omasad@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 4:00:55 PM Subject: Re: [Engine-devel] Design wiki page for trusted compute =
integration with oVirt has been updated =20 Ofri, =20 Absolutely right, aggregated query has a significantly time = improve compared to separated queries. I agree a aggregated query on engine's starting. Is it possible to invoke attestation service in engine's initialization code block instead of "quartz job"? Is =
------=_NextPart_000_00BD_01CE4443.79F17D10 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I like the ideas of 2-phase aggregated attestation & cluster-by-cluster order. But I want to understand the process more clearly. Without TCP, how does engine handle the states of existing hosts during engine booting? Will engine put all existing hosts in non-operational = state and then perform some check via VDSM then turn it into operational = state? Put host in non-operational state will cause VM migration, right? Or there is a global state in engine to indicate whether user is allowed = to create VM? Thanks Jimmy Itamar Heim wrote on=A02013-04-28: progress. like process; pools pools there
any class similar with " InitVdsOnUpCommand " for engine's initialization? =20 Best Regards, Dave Chen =20 org.ovirt.engine.core.bll.Backend.Initialize() =20 Note you cannot block this method while waiting for results. = Instead I suggest you fire a one-time background request from this method. =20 =20 Ofri Masad wrote on 2013-04-21: integration with oVirt has been updated =20 Dave, =20 If I'm not mistaking, there is a big difference between separated queries to the attestation server and aggregated one? Is it true? =20 Thanks, Ofri =20 ----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Ofri Masad" <omasad@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, "Wei D Chen" <wei.d.chen@intel.com>, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 10:20:17 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated =20 On 04/21/2013 10:13 AM, Ofri Masad wrote: > Hi, > One more thing we need to think about for the second approach - > aggregated query. On engine start we need to determine the trust > state of all the hosts. sending a separate query for each host > will overload the attestation host and the network. an initial > aggregated query needs to be send when the engine starts. > Same thing can happen after management network fail and so on. > Maybe we can run a quartz job every x minutes, checking if a = large > part of the hosts in the cluster (like 30%) are untrusted - in > that case run the aggregated query. =20 are we sure this optimization is needed? how heavy/latent is the call to the attestation service? =20
Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel =20
Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel =20
Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel =20 =20
Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Jimmy ------=_NextPart_000_00BD_01CE4443.79F17D10 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIdxjCCAyAw ggKJoAMCAQICBDXe9M8wDQYJKoZIhvcNAQEFBQAwTjELMAkGA1UEBhMCVVMxEDAOBgNVBAoTB0Vx dWlmYXgxLTArBgNVBAsTJEVxdWlmYXggU2VjdXJlIENlcnRpZmljYXRlIEF1dGhvcml0eTAeFw05 ODA4MjIxNjQxNTFaFw0xODA4MjIxNjQxNTFaME4xCzAJBgNVBAYTAlVTMRAwDgYDVQQKEwdFcXVp ZmF4MS0wKwYDVQQLEyRFcXVpZmF4IFNlY3VyZSBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwgZ8wDQYJ KoZIhvcNAQEBBQADgY0AMIGJAoGBAMFdsVhnCGLuoJotHwhtkRRomAoe/toEbxOEYiHD0XzOnwXg uAHwTjTs4oqVBGSs8WtTXwWzy2eAv0ICjv7dAQns4QAUT/z78AzdQ7pbK+EfgHCZFVeTFvEPl2q3 wmgjHMxNWTCsUR47ryvW7mNFe8XZX1DS41APOojnvxT94Me5AgMBAAGjggEJMIIBBTBwBgNVHR8E aTBnMGWgY6BhpF8wXTELMAkGA1UEBhMCVVMxEDAOBgNVBAoTB0VxdWlmYXgxLTArBgNVBAsTJEVx dWlmYXggU2VjdXJlIENlcnRpZmljYXRlIEF1dGhvcml0eTENMAsGA1UEAxMEQ1JMMTAaBgNVHRAE EzARgQ8yMDE4MDgyMjE2NDE1MVowCwYDVR0PBAQDAgEGMB8GA1UdIwQYMBaAFEjmaPkr0rKV10fY IyAQTzOYkJ/UMB0GA1UdDgQWBBRI5mj5K9KylddH2CMgEE8zmJCf1DAMBgNVHRMEBTADAQH/MBoG CSqGSIb2fQdBAAQNMAsbBVYzLjBjAwIGwDANBgkqhkiG9w0BAQUFAAOBgQBYzinq/Pfetc4CuRe1 hdG54+CVzCUxDQCmkm5/tpJjnlCV0Zpv5BHeY4VumO6o/1rI01WyZnFX3sAh6z0qpyNJAQSGQnv8 7n+iFlK1Z2fTQNs7JliyKHc9rhR3Ydb6KmYnoA36p3Nc6nDxlCFlRF/6/O8paKmih3nvee9PrAd3 ODCCAz0wggKmoAMCAQICAwWw/zANBgkqhkiG9w0BAQUFADBOMQswCQYDVQQGEwJVUzEQMA4GA1UE ChMHRXF1aWZheDEtMCsGA1UECxMkRXF1aWZheCBTZWN1cmUgQ2VydGlmaWNhdGUgQXV0aG9yaXR5 MB4XDTA2MDIxNjE4MDEzMFoXDTE2MDIxOTE4MDEzMFowUjELMAkGA1UEBhMCVVMxGjAYBgNVBAoT EUludGVsIENvcnBvcmF0aW9uMScwJQYDVQQDEx5JbnRlbCBFeHRlcm5hbCBCYXNpYyBQb2xpY3kg Q0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDBpd/XOb9QVqEZ8mQ1042TdOIq3ATD IsV2xDyt30yLyMR5Wjtus0bn3B+he89BiNO/LP6+rFzEwlD55PlX+HLGIKeNNG97dqyc30FElEUj ZzTZFq2N4e3kVJ/XAEEgANzV8v9qp7qWwxugPgfc3z9BkYot+CifozexHLb/hEZj+yISCU61kRZv uSQ0E11yYL4dRgcglJeaHo3oX57rvIckaLsYV5/1Aj+R8DM1Ppk965XQAKsHfnyT7C4S50T4lVn4 lz36wOdNZn/zegG1zp41lnoTFfT4KuKVJH5x7YD1p6KbgJCKLovnujGuohquBNfdXKpZkvz6pGv+ iC1HawJdAgMBAAGjgaAwgZ0wDgYDVR0PAQH/BAQDAgEGMB0GA1UdDgQWBBQaxgxKxEdvqNutK/D0 Vgaj7TdUDDA6BgNVHR8EMzAxMC+gLaArhilodHRwOi8vY3JsLmdlb3RydXN0LmNvbS9jcmxzL3Nl Y3VyZWNhLmNybDAfBgNVHSMEGDAWgBRI5mj5K9KylddH2CMgEE8zmJCf1DAPBgNVHRMBAf8EBTAD AQH/MA0GCSqGSIb3DQEBBQUAA4GBABMQOK2kVKVIlUWwLTdywJ+e2O+PC/uQltK2F3lRyrPfBn69 tOkIP4SgDJOfsxyobIrPLe75kBLw+Dom13OBDp/EMZJZ1CglQfVV8co9mT3aZMjSGGQiMgkJLR3j Mfr900fXZKj5XeqCJ+JP0mEhJGEdVCY+FFlksJjV86fDrq1QMIIFijCCBHKgAwIBAgIKYR6AtwAA AAAABzANBgkqhkiG9w0BAQUFADBSMQswCQYDVQQGEwJVUzEaMBgGA1UEChMRSW50ZWwgQ29ycG9y YXRpb24xJzAlBgNVBAMTHkludGVsIEV4dGVybmFsIEJhc2ljIFBvbGljeSBDQTAeFw0wOTA1MTUx OTI1MTNaFw0xNTA1MTUxOTM1MTNaMFYxCzAJBgNVBAYTAlVTMRowGAYDVQQKExFJbnRlbCBDb3Jw b3JhdGlvbjErMCkGA1UEAxMiSW50ZWwgRXh0ZXJuYWwgQmFzaWMgSXNzdWluZyBDQSAzQTCCASIw DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMGPgGLnOO5IOzlHRfr1XfCVb97V4BR2QVpPZ7Cr cIQ+FGa2KHD/6dPjwxOIrtFTdfW4BYikdFmxUZVBWRWZ5Vye2cCdGzFWqIEOE1e17nNx1jM8Z6GZ EqbDUS+vBuPlBFHKQoVm5BaNIHpyn2XZxqwjV9j5/crIfPrCGstk+2ztUhVS8OHEgzO784PgD9pO gBnnAbZHmEM1FYYmQ6ibS+gVCHzobDYG+YReRiHpFKWBxpUuP+X0WYFw/Ja1JW7N8pELAFDw0UFB WFgiv1QIusdLvSy8mcsLJ5wy050OVcxShqoUxhw/wvyuuoQxvmEPjhRa1C2oSCmGN0003GMhQWMC AwEAAaOCAlwwggJYMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFKoWZq+3PVZTYK4Nwu3z7gfL UWB+MAsGA1UdDwQEAwIBhjASBgkrBgEEAYI3FQEEBQIDAQABMCMGCSsGAQQBgjcVAgQWBBTCKwhT x+hdMsKCgOmWwLgjQsAV+TAZBgkrBgEEAYI3FAIEDB4KAFMAdQBiAEMAQTAfBgNVHSMEGDAWgBQa xgxKxEdvqNutK/D0Vgaj7TdUDDCBvQYDVR0fBIG1MIGyMIGvoIGsoIGphk5odHRwOi8vd3d3Lmlu dGVsLmNvbS9yZXBvc2l0b3J5L0NSTC9JbnRlbCUyMEV4dGVybmFsJTIwQmFzaWMlMjBQb2xpY3kl MjBDQS5jcmyGV2h0dHA6Ly9jZXJ0aWZpY2F0ZXMuaW50ZWwuY29tL3JlcG9zaXRvcnkvQ1JML0lu dGVsJTIwRXh0ZXJuYWwlMjBCYXNpYyUyMFBvbGljeSUyMENBLmNybDCB4wYIKwYBBQUHAQEEgdYw gdMwYwYIKwYBBQUHMAKGV2h0dHA6Ly93d3cuaW50ZWwuY29tL3JlcG9zaXRvcnkvY2VydGlmaWNh dGVzL0ludGVsJTIwRXh0ZXJuYWwlMjBCYXNpYyUyMFBvbGljeSUyMENBLmNydDBsBggrBgEFBQcw AoZgaHR0cDovL2NlcnRpZmljYXRlcy5pbnRlbC5jb20vcmVwb3NpdG9yeS9jZXJ0aWZpY2F0ZXMv SW50ZWwlMjBFeHRlcm5hbCUyMEJhc2ljJTIwUG9saWN5JTIwQ0EuY3J0MA0GCSqGSIb3DQEBBQUA A4IBAQCUY/1d0MS6VPTlIcOho1XWh193PD5kJDJSPdphLHQdM1oKA+whMdIBoY1VzTDDK+C+Ey4J cyna7fpC8uVmn/Rz/i9MZtyc7qezPtZTn9UyORvJmddH+Ox/RycGwe3ags8jUdspECorYOkJyZks nDIlTVUvbR7wyY+gGJYqxWXqrcVFEiMsWu8/OIlf7F2gAYMBw1kZ55dn4lWBIM0WqvReWpPvhYeN 7Y+3MKEdSMkQ7TZiNbfdZ5D/8KfWNMTJ4VHltOgCL1lA5tx/F4R1920skpL5eu3Sj650RUe3rOXs aV5NyJzBwB31+1zsmleVdFD0k/Fw9HxXbAQE35ucN/7CMIIFijCCBHKgAwIBAgIKYSCKYgAAAAAA CDANBgkqhkiG9w0BAQUFADBSMQswCQYDVQQGEwJVUzEaMBgGA1UEChMRSW50ZWwgQ29ycG9yYXRp b24xJzAlBgNVBAMTHkludGVsIEV4dGVybmFsIEJhc2ljIFBvbGljeSBDQTAeFw0wOTA1MTUxOTI3 MjZaFw0xNTA1MTUxOTM3MjZaMFYxCzAJBgNVBAYTAlVTMRowGAYDVQQKExFJbnRlbCBDb3Jwb3Jh dGlvbjErMCkGA1UEAxMiSW50ZWwgRXh0ZXJuYWwgQmFzaWMgSXNzdWluZyBDQSAzQjCCASIwDQYJ KoZIhvcNAQEBBQADggEPADCCAQoCggEBAKQEM1Wn9TU9vc9C+/Tc7KB+eiYElmrcEWE32WUdHvWG +IcQHVQsikTmMyKKojNLw2B5s6Iekc8ivDo/wCfjZzX9JyftMnc+AArc0la87Olybzm8K9jXEfTB vTnUSFSiI9ZYefITdiUgqlAFuljFZEHYKYtLuhrRacpmQfP4mV63NKdc2bT804HRf6YptZFa4k6Y N94zlrGNrBuQQ74WFzz/jLBusbUpEkro6Mu/ZYFOFWQrV9lBhF9Ruk8yN+3N6n9fUo/qBigiF2kE n9xVh1ykl7SCGL2jBUkXx4qgV27a6Si8lRRdgrHGtN/HWnSWlLXTH5l575H4Lq++77OFv38CAwEA AaOCAlwwggJYMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFA7GKvdZsggQkCVvw939imYxMCvF MAsGA1UdDwQEAwIBhjASBgkrBgEEAYI3FQEEBQIDAQABMCMGCSsGAQQBgjcVAgQWBBQ5oFY2ekKQ /5Ktim+VdMeSWb4QWTAZBgkrBgEEAYI3FAIEDB4KAFMAdQBiAEMAQTAfBgNVHSMEGDAWgBQaxgxK xEdvqNutK/D0Vgaj7TdUDDCBvQYDVR0fBIG1MIGyMIGvoIGsoIGphk5odHRwOi8vd3d3LmludGVs LmNvbS9yZXBvc2l0b3J5L0NSTC9JbnRlbCUyMEV4dGVybmFsJTIwQmFzaWMlMjBQb2xpY3klMjBD QS5jcmyGV2h0dHA6Ly9jZXJ0aWZpY2F0ZXMuaW50ZWwuY29tL3JlcG9zaXRvcnkvQ1JML0ludGVs JTIwRXh0ZXJuYWwlMjBCYXNpYyUyMFBvbGljeSUyMENBLmNybDCB4wYIKwYBBQUHAQEEgdYwgdMw YwYIKwYBBQUHMAKGV2h0dHA6Ly93d3cuaW50ZWwuY29tL3JlcG9zaXRvcnkvY2VydGlmaWNhdGVz L0ludGVsJTIwRXh0ZXJuYWwlMjBCYXNpYyUyMFBvbGljeSUyMENBLmNydDBsBggrBgEFBQcwAoZg aHR0cDovL2NlcnRpZmljYXRlcy5pbnRlbC5jb20vcmVwb3NpdG9yeS9jZXJ0aWZpY2F0ZXMvSW50 ZWwlMjBFeHRlcm5hbCUyMEJhc2ljJTIwUG9saWN5JTIwQ0EuY3J0MA0GCSqGSIb3DQEBBQUAA4IB AQCxtQEHchVQhXyjEqtMVUMe6gkmPsIczHxSeqNbo9dsD+6xbT65JT+oYgpIAtfEsYXeUJu1cChq pb22U5bMAz7eaQcW5bzefufWvA6lg2048B8oczBj/q+5P5NpYrUO8jOmN4jTjfJq3ElZ7yFWpy7r B3Vm/aN6ATYqWfMbS/xfh+JCxmH3droUmMJI0/aZJHsLtjbjFnNsHDNrJZX1vxlM78Lb1hjskTEN PmhbVbfTj5i/ZGnhv4tmI8QZPCNtcegXJrfhRl2D9bWpdTOPrWiLDUqzy1Z6KL7TcOS/PCl8RHCJ XkPau/thTQCpIoDa2+c+3XA++gRTfAQ4svTO260NMIIF+zCCBOOgAwIBAgIKHtX06gABAACWPTAN BgkqhkiG9w0BAQUFADBWMQswCQYDVQQGEwJVUzEaMBgGA1UEChMRSW50ZWwgQ29ycG9yYXRpb24x KzApBgNVBAMTIkludGVsIEV4dGVybmFsIEJhc2ljIElzc3VpbmcgQ0EgM0IwHhcNMTIwNjA4MDgw NTExWhcNMTUwNTE1MTkzNzI2WjA3MRIwEAYDVQQDEwlXZWksIEdhbmcxITAfBgkqhkiG9w0BCQEW Emdhbmcud2VpQGludGVsLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALNyfS4y C6aDo3DZ/oId96Dvi8CB5SJyDUcMhpKWZtzqPX2mMOqQNgv4qAtHUjt4ibyPSjGZ+0EM3r63384g GcVR8+uxiuijBIOCkis6oGQ+TmBl1i28KobkE4jNnLCES0keisfNdzO8vAOxIFbT9KxQl1f1Mfvs ZfyGfFYB53gHCh1VxdZ7a2XKaON+l2YYx2p5xGGZtDDb61ajXSGvdHK+qMIfo7LMoZmY42t5Nawg izwcqBPUOLR+JXOtyGGiXZx3wZPeRmZx/eCMPBhSlfewpvUrK8W0kL591Lv0HeUVEJye2bOmlLo1 DeIp6KH9JujFB33KhHXvNsugc9IYUVMCAwEAAaOCAugwggLkMAsGA1UdDwQEAwIHgDA8BgkrBgEE AYI3FQcELzAtBiUrBgEEAYI3FQiGw4x1hJnlUYP9gSiFjp9TgpHACWeB3r05lfBDAgFkAgEIMB0G A1UdDgQWBBQYdG5bBKSgjlBUQ6dpUm2vjlkEKDAfBgNVHSMEGDAWgBQOxir3WbIIEJAlb8Pd/Ypm MTArxTCBzwYDVR0fBIHHMIHEMIHBoIG+oIG7hldodHRwOi8vd3d3LmludGVsLmNvbS9yZXBvc2l0 b3J5L0NSTC9JbnRlbCUyMEV4dGVybmFsJTIwQmFzaWMlMjBJc3N1aW5nJTIwQ0ElMjAzQigxKS5j cmyGYGh0dHA6Ly9jZXJ0aWZpY2F0ZXMuaW50ZWwuY29tL3JlcG9zaXRvcnkvQ1JML0ludGVsJTIw RXh0ZXJuYWwlMjBCYXNpYyUyMElzc3VpbmclMjBDQSUyMDNCKDEpLmNybDCB9QYIKwYBBQUHAQEE gegwgeUwbAYIKwYBBQUHMAKGYGh0dHA6Ly93d3cuaW50ZWwuY29tL3JlcG9zaXRvcnkvY2VydGlm aWNhdGVzL0ludGVsJTIwRXh0ZXJuYWwlMjBCYXNpYyUyMElzc3VpbmclMjBDQSUyMDNCKDEpLmNy dDB1BggrBgEFBQcwAoZpaHR0cDovL2NlcnRpZmljYXRlcy5pbnRlbC5jb20vcmVwb3NpdG9yeS9j ZXJ0aWZpY2F0ZXMvSW50ZWwlMjBFeHRlcm5hbCUyMEJhc2ljJTIwSXNzdWluZyUyMENBJTIwM0Io MSkuY3J0MB8GA1UdJQQYMBYGCCsGAQUFBwMEBgorBgEEAYI3CgMMMCkGCSsGAQQBgjcVCgQcMBow CgYIKwYBBQUHAwQwDAYKKwYBBAGCNwoDDDBBBgNVHREEOjA4oCIGCisGAQQBgjcUAgOgFAwSZ2Fu Zy53ZWlAaW50ZWwuY29tgRJnYW5nLndlaUBpbnRlbC5jb20wDQYJKoZIhvcNAQEFBQADggEBAHuy cX8AxjwfC5zmWDh0QpY8vDSgyLXaUDYKm2+ATDJDn5kALJgxAqaThvqGTH+oz73HQ7L8v7QxM0Yp 1IQd/k5GeqMzhuXEoPM4rcORlOlvRqxBJNZUuYwxvyYaUpLU1W8EsOB2zB31ykzdXH93b6ZpfJk7 8eqZuq00xHxU9mw4PXlWPnn1NDBYD1JH/ufCmpFk6sBE2bBf2u2miBEwHoRUyoH1nbu78aOs4mE6 fRC9NutIriNPI2790R3FAY8dLWl3nrpXs80TrUCptat61uNRJDH06KXe81QCtvDVlBGbZ4gqWR3P ZGsnJKeOLOO38PQvFFm1Xjs4DVYiPVYyCTIwggZCMIIFKqADAgECAgoUmebGAAEAAFPVMA0GCSqG SIb3DQEBBQUAMFYxCzAJBgNVBAYTAlVTMRowGAYDVQQKExFJbnRlbCBDb3Jwb3JhdGlvbjErMCkG A1UEAxMiSW50ZWwgRXh0ZXJuYWwgQmFzaWMgSXNzdWluZyBDQSAzQTAeFw0xMDEyMDEwOTMxMzla Fw0xMzExMTUwOTMxMzlaMDcxEjAQBgNVBAMTCVdlaSwgR2FuZzEhMB8GCSqGSIb3DQEJARYSZ2Fu Zy53ZWlAaW50ZWwuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAlkDjEHBe0Gg1 dZhmebTYRY4qWGcBCKuEGfp8DuoVXyv3gW6bXc8TcWRSXj0vxL/+/RKObwVc17Cu2XcO0MTU1EOo ohTEl/QDnBvj0qeqKnkZGmAM0cCU7+m07N0ATPEzg0t2gJUA8dVO37iAo1KMQ72ovMJ1LU+x7tno d6PhrgXLmbjkEjQZnHwLvuI2TdiKKDvJtd6xz6DQuIBSPyn4wYsckGQcb5NlEiQCkCaPM/ZtcKcm Kxu0g1TfW7zIWuRhkPIOyin/bsN8RBTTq8lL74yFgTB4ybAMp13wtROVAN5FLYE31CAgJ69EBNix xTBNsLtv+KM6sXsbDccp3H5JwQIDAQABo4IDLzCCAyswCwYDVR0PBAQDAgQwMD0GCSsGAQQBgjcV BwQwMC4GJisGAQQBgjcVCIbDjHWEmeVRg/2BKIWOn1OCkcAJZ4S52UGHhP9OAgFkAgEMMEQGCSqG SIb3DQEJDwQ3MDUwDgYIKoZIhvcNAwICAgCAMA4GCCqGSIb3DQMEAgIAgDAHBgUrDgMCBzAKBggq hkiG9w0DBzAdBgNVHQ4EFgQUwmKbKrf26Ot7Acv/+2aZDzj7gaQwHwYDVR0jBBgwFoAUqhZmr7c9 VlNgrg3C7fPuB8tRYH4wgc8GA1UdHwSBxzCBxDCBwaCBvqCBu4ZXaHR0cDovL3d3dy5pbnRlbC5j b20vcmVwb3NpdG9yeS9DUkwvSW50ZWwlMjBFeHRlcm5hbCUyMEJhc2ljJTIwSXNzdWluZyUyMENB JTIwM0EoMSkuY3JshmBodHRwOi8vY2VydGlmaWNhdGVzLmludGVsLmNvbS9yZXBvc2l0b3J5L0NS TC9JbnRlbCUyMEV4dGVybmFsJTIwQmFzaWMlMjBJc3N1aW5nJTIwQ0ElMjAzQSgxKS5jcmwwgfUG CCsGAQUFBwEBBIHoMIHlMGwGCCsGAQUFBzAChmBodHRwOi8vd3d3LmludGVsLmNvbS9yZXBvc2l0 b3J5L2NlcnRpZmljYXRlcy9JbnRlbCUyMEV4dGVybmFsJTIwQmFzaWMlMjBJc3N1aW5nJTIwQ0El MjAzQSgxKS5jcnQwdQYIKwYBBQUHMAKGaWh0dHA6Ly9jZXJ0aWZpY2F0ZXMuaW50ZWwuY29tL3Jl cG9zaXRvcnkvY2VydGlmaWNhdGVzL0ludGVsJTIwRXh0ZXJuYWwlMjBCYXNpYyUyMElzc3Vpbmcl MjBDQSUyMDNBKDEpLmNydDAfBgNVHSUEGDAWBggrBgEFBQcDBAYKKwYBBAGCNwoDBDApBgkrBgEE AYI3FQoEHDAaMAoGCCsGAQUFBwMEMAwGCisGAQQBgjcKAwQwQQYDVR0RBDowOKAiBgorBgEEAYI3 FAIDoBQMEmdhbmcud2VpQGludGVsLmNvbYESZ2FuZy53ZWlAaW50ZWwuY29tMA0GCSqGSIb3DQEB BQUAA4IBAQAPFjqbkKIbnvnYwFG/QpKUm2yGzieys9IJny7PU2fVV4ksTL7emcpCfhFtlg7cPEgU 7Rx6gG4hO+ZBx9oojImSGWj44Pj3EzSyRRaN+UK1VVESHgE2HKmETsAnRN/wulP9ahPJNmEOklBg oX6BJg9NYTqp8gDS6m1QBQoh/C2/51MGcmTecM4g5BuEpgmvxovlPDHlnSq390NXwXtdh1lM5qbo V9FBWFCR1Q7mf7n2Y6b/9m9bn9gMzbdwjMrnpYIiC5rN5WBDQtuzAios8OheGSTX0A1npuGZ3cvP NlPbRAoD7TVAdGhr+ZztVLA0+IFYmQqHV2C8q3TybgPnp1/uMYIDhjCCA4ICAQEwZDBWMQswCQYD VQQGEwJVUzEaMBgGA1UEChMRSW50ZWwgQ29ycG9yYXRpb24xKzApBgNVBAMTIkludGVsIEV4dGVy bmFsIEJhc2ljIElzc3VpbmcgQ0EgM0ICCh7V9OoAAQAAlj0wCQYFKw4DAhoFAKCCAfcwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTMwNDI4MTEwNjIzWjAjBgkqhkiG 9w0BCQQxFgQUqcFbs1qkmAyeNXOQpfMJFvRjkv0wcwYJKwYBBAGCNxAEMWYwZDBWMQswCQYDVQQG EwJVUzEaMBgGA1UEChMRSW50ZWwgQ29ycG9yYXRpb24xKzApBgNVBAMTIkludGVsIEV4dGVybmFs IEJhc2ljIElzc3VpbmcgQ0EgM0ECChSZ5sYAAQAAU9UwdQYLKoZIhvcNAQkQAgsxZqBkMFYxCzAJ BgNVBAYTAlVTMRowGAYDVQQKExFJbnRlbCBDb3Jwb3JhdGlvbjErMCkGA1UEAxMiSW50ZWwgRXh0 ZXJuYWwgQmFzaWMgSXNzdWluZyBDQSAzQQIKFJnmxgABAABT1TCBqwYJKoZIhvcNAQkPMYGdMIGa MAsGCWCGSAFlAwQBKjALBglghkgBZQMEARYwCgYIKoZIhvcNAwcwCwYJYIZIAWUDBAECMA4GCCqG SIb3DQMCAgIAgDAHBgUrDgMCBzANBggqhkiG9w0DAgIBQDANBggqhkiG9w0DAgIBKDAHBgUrDgMC GjALBglghkgBZQMEAgMwCwYJYIZIAWUDBAICMAsGCWCGSAFlAwQCATANBgkqhkiG9w0BAQEFAASC AQAJxx1uoXlXP/YcToOlGZx3H1SkHLKIj2ogwD/DFKBrb9/9MYtIsmP9MEuEXUpM1AdfSEY+erc5 4kRgboMoB7OZ/YiOTh2Zy/3ZXcE9iHoL9IDoeShnogrRSCb91mk7jNut1mMTByEw2Ykuu/w9g7i+ SgCpdarAhkCgNXn9haWHdWTnaBVHHiql2DRJIdsfvZVyJSfuOmIEF7k7QJK/Zcwx/4aaL/rcA6M/ GgdEGMaWrUVTcvVwDI95zq6nDcCouqO/lOSUt/YcGVbfesV+0GpE0EwerOlpjukEi3eRKLQtxiBv j1ZARtGOnzUrzAE518CfYYidYcxM8RjxBXPs7qTcAAAAAAAA ------=_NextPart_000_00BD_01CE4443.79F17D10--
Hi Jimmy, When the engine starts, it will start 'learning' the current state of known hosts. So if we want to secure all hosts in 'up' state, you will need to change it to 'unassigned' only if cluster.trusted == true in- org.ovirt.engine.core.vdsbroker.ResourceManager.AddVds(VDS, boolean) This means the host will be picked up by: org.ovirt.engine.core.vdsbroker.VirtMonitoringStrategy.isMonitoringNeeded(VDS) and the attestation query should get the results and update the trust level and accordingly the vds status. Note that we assume that trust level takes precedence on other functionalities, as this flow will cause storage connections to reinitialize as well. Basically it means that booting will take longer, but this is the price of security. ----- Original Message -----
From: "Gang Wei" <gang.wei@intel.com> To: "Itamar Heim" <iheim@redhat.com>, "Doron Fediuck" <dfediuck@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, engine-devel@ovirt.org Sent: Sunday, April 28, 2013 2:06:24 PM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
I like the ideas of 2-phase aggregated attestation & cluster-by-cluster order.
But I want to understand the process more clearly.
Without TCP, how does engine handle the states of existing hosts during engine booting? Will engine put all existing hosts in non-operational state and then perform some check via VDSM then turn it into operational state? Put host in non-operational state will cause VM migration, right?
Or there is a global state in engine to indicate whether user is allowed to create VM?
Thanks Jimmy
Itamar Heim wrote on 2013-04-28:
On 04/28/2013 11:34 AM, Doron Fediuck wrote:
Hi Dave,
Just to make sure I fully understand, I'll repeat your basic arguments;
1. It takes time to query a big number of hosts (hundreds).
2. When backend is booting, a user may start a VM on a host which was hacked during the downtime of the engine.
If the above is your concern, it shouldn't be so. The reason is, that no host will become operational before you get a response from the attestation server and allow it to become operational. So a user cannot start a new VM on a non-operational host.
i'd do the queries in groups of "cluster", so cluste-by-cluster they get unblocked. cluster without attestation service shouldn't block on this of course.
What this means is that your thread may need to update the user by
sending
a periodic event that a large scale attestation operation is in progress. Other than that, maybe your thread can work in smaller groups if it gets better results? ie- instead of one query for 300 hosts, maybe you can run 3 serialized queries for 100 hosts each? If this does not help, maybe you can run a short query for something like 10 hosts, which should get an answer relatively fast. The you can issue a query for the other 290 hosts which will take longer. In this way the system may get 10 hosts to work with quite fast, and later on the other 290 hosts will join... So this can actually be configurable to a 2-phase process; a short query and a longer one. The admin can choose the short query size based on his setup, and the longer query can pick up all the other hosts. What do you think?
Doron
----- Original Message -----
From: "Wei D Chen" <wei.d.chen@intel.com> To: "Doron Fediuck" <dfediuck@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, engine-devel@ovirt.org Sent: Saturday, April 27, 2013 9:36:44 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
Hi,
Our current consideration is add a new thread in engine's side to attest all of hosts (aggregated query from attestation sever) one time in case of engine's rebooting. There is still one potential issue under extreme condition, saying, hundreds of nodes in a datacenter, attest all of hosts still may take couple of mins, let's say, one hacked untrusted node before receiving the latest status may considered as a trusted host, so, the worst case in a datacenter with hundreds of nodes is, 1. engine is down for some reasons and boot up again, some trusted nodes may be hacked and rebooted during this period. 2. our thread is running to get all of node's status (trust /untrusted), may take couple of mins in large datacenter. 2. user create VMs on these hacked nodes and believe these VMs are trusted VMs launched on trusted nodes. 3. our thread get the correct status of these untrusted nodes, set these nodes as non-operational. 4. all of these "trusted" VMs running on these untrusted nodes are expected to migrate to other trusted node.
So, the question is in a trusted cluster with hundreds of nodes some VMs expected to create on trusted nodes may actually create on untrusted nodes instead, and this time may last for couple of mins. (worst case in my view is 10 mins with 1000 nodes). Does this acceptable from your point of view? Or any other suggestion?
Best Regards, Dave Chen
Doron Fediuck wrote on 2013-04-21:
integration with oVirt has been updated
----- Original Message -----
From: "Wei D Chen" <wei.d.chen@intel.com> To: "Ofri Masad" <omasad@redhat.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, engine-devel@ovirt.org Sent: Sunday, April 21, 2013 4:00:55 PM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
Ofri,
Absolutely right, aggregated query has a significantly time improve compared to separated queries. I agree a aggregated query on engine's starting. Is it possible to invoke attestation service in engine's initialization code block instead of "quartz job"? Is there any class similar with " InitVdsOnUpCommand " for engine's initialization?
Best Regards, Dave Chen
org.ovirt.engine.core.bll.Backend.Initialize()
Note you cannot block this method while waiting for results. Instead I suggest you fire a one-time background request from this method.
Ofri Masad wrote on 2013-04-21:
integration with oVirt has been updated
Dave,
If I'm not mistaking, there is a big difference between separated queries to the attestation server and aggregated one? Is it true?
Thanks, Ofri
----- Original Message ----- > From: "Itamar Heim" <iheim@redhat.com> > To: "Ofri Masad" <omasad@redhat.com> > Cc: "Oved Ourfalli" <ovedo@redhat.com>, "Wei D Chen" > <wei.d.chen@intel.com>, engine-devel@ovirt.org > Sent: Sunday, April 21, 2013 10:20:17 AM > Subject: Re: [Engine-devel] Design wiki page for trusted compute > pools integration with oVirt has been updated > > On 04/21/2013 10:13 AM, Ofri Masad wrote: >> Hi, >> One more thing we need to think about for the second approach - >> aggregated query. On engine start we need to determine the trust >> state of all the hosts. sending a separate query for each host >> will overload the attestation host and the network. an initial >> aggregated query needs to be send when the engine starts. >> Same thing can happen after management network fail and so on. >> Maybe we can run a quartz job every x minutes, checking if a large >> part of the hosts in the cluster (like 30%) are untrusted - in >> that case run the aggregated query. > > are we sure this optimization is needed? > how heavy/latent is the call to the attestation service? > _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Jimmy
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Oved Ourfalli" <ovedo@redhat.com> To: "Itamar Heim" <iheim@redhat.com>, "Wei D Chen" <wei.d.chen@intel.com> Cc: engine-devel@ovirt.org Sent: Sunday, April 21, 2013 8:41:50 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Wei D Chen" <wei.d.chen@intel.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, "engine-devel@ovirt.org" <engine-devel@ovirt.org> Sent: Saturday, April 20, 2013 5:49:47 PM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
On 04/19/2013 12:21 PM, Chen, Wei D wrote:
Hi All,
Our second approach for trusted compute pools integration with oVirt seems more simple and sensible than previous VM level approach. Welcome any comments on our latest design. Thanks in advance.
Link is here, http://www.ovirt.org/Trusted_compute_pools
a few nits: 1. last updated date isn't updated... 2. from reading it top to bottom, hard to understand the 2nd approach is the one to be used and not the first (vm level). 3. cluster dialog - the 'trusted' should be a checkbox, not radio button, and should only be enabled if virt service was chosen.
I'd also consider putting this property in a different side tab. Perhaps "Cluster policy" side tab would fit? (dividing it into two sections "scheduling policy" and "additional properties" or something similar.
What do you think about that?
I prefer such optimizations to wait for our scheduling work to be more advanced.
thanks, Itamar _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Our wiki page has changed according to your suggestion, that is, a checkbox is added for "Cluster policy" side tab. For details, pls check it from http://www.ovirt.org/Trusted_compute_pools. Best Regards, Dave Chen -----Original Message----- From: Doron Fediuck [mailto:dfediuck@redhat.com] Sent: Monday, April 22, 2013 8:53 PM To: Oved Ourfalli Cc: Itamar Heim; Chen, Wei D; engine-devel@ovirt.org Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated ----- Original Message -----
From: "Oved Ourfalli" <ovedo@redhat.com> To: "Itamar Heim" <iheim@redhat.com>, "Wei D Chen" <wei.d.chen@intel.com> Cc: engine-devel@ovirt.org Sent: Sunday, April 21, 2013 8:41:50 AM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Wei D Chen" <wei.d.chen@intel.com> Cc: "Oved Ourfalli" <ovedo@redhat.com>, "engine-devel@ovirt.org" <engine-devel@ovirt.org> Sent: Saturday, April 20, 2013 5:49:47 PM Subject: Re: [Engine-devel] Design wiki page for trusted compute pools integration with oVirt has been updated
On 04/19/2013 12:21 PM, Chen, Wei D wrote:
Hi All,
Our second approach for trusted compute pools integration with oVirt seems more simple and sensible than previous VM level approach. Welcome any comments on our latest design. Thanks in advance.
Link is here, http://www.ovirt.org/Trusted_compute_pools
a few nits: 1. last updated date isn't updated... 2. from reading it top to bottom, hard to understand the 2nd approach is the one to be used and not the first (vm level). 3. cluster dialog - the 'trusted' should be a checkbox, not radio button, and should only be enabled if virt service was chosen.
I'd also consider putting this property in a different side tab. Perhaps "Cluster policy" side tab would fit? (dividing it into two sections "scheduling policy" and "additional properties" or something similar.
What do you think about that?
I prefer such optimizations to wait for our scheduling work to be more advanced.
thanks, Itamar _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
participants (6)
-
Chen, Wei D -
Doron Fediuck -
Itamar Heim -
Ofri Masad -
Oved Ourfalli -
Wei, Gang