[Kimchi-devel] [PATCH] Use a pool of threads to valid all remote ISOs in parallel

[Aline Manera]

From: Aline Manera <alinefm at br.ibm.com>

That way we can reduce processing time in 8s.

Without threads the requests takes 15s to complete:
GET http://localhost:8000/config/distros    200 OK      10.64s

And with the pool of threads it takes 2s:
GET http://localhost:8000/config/distros    200 OK      2.01s

Signed-off-by: Aline Manera <alinefm at br.ibm.com>
---
 src/kimchi/model/config.py |   18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/src/kimchi/model/config.py b/src/kimchi/model/config.py
index d44ef90..3580ad3 100644
--- a/src/kimchi/model/config.py
+++ b/src/kimchi/model/config.py
@@ -17,6 +17,8 @@
 # License along with this library; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301 USA
 
+from multiprocessing.pool import ThreadPool
+
 import cherrypy
 
 from kimchi.basemodel import Singleton
@@ -91,12 +93,16 @@ class DistrosModel(object):
         self.distros = distroloader.get()
 
     def get_list(self):
-        res = []
-        # only return distro with valid URL
-        for distro, data in self.distros.iteritems():
-            url = data['path']
-            if check_url_path(url):
-                res.append(distro)
+        def validate_distro(distro):
+            if check_url_path(distro['path']):
+                return distro['name']
+
+        n_processes = len(self.distros.keys())
+        pool = ThreadPool(processes=n_processes)
+        map_res = pool.map_async(validate_distro, self.distros.values())
+        pool.close()
+        pool.join()
+        res = list(set(map_res.get()) - set([None]))
         return sorted(res)
 
 
-- 
1.7.10.4