Re: [ovirt-devel] Caching of data from the database done properly

Hi, some of information in mail are not exactly true. Namely MacPoolPerCluster *does not do caching*, it does not even have DB layer structures it could cache. So how it works is: pool has configuration upon which it initialize itself. After that, it looks into db for all used MACs, which currently happens to be querying all MAC addresses of all VmNics. So this is initialized from data in DB, but it does not cache them. Clients [of pool] asks pool for MAC address, which is then used somewhere without pool supervision. I don't want to question this design, and I'm not saying that it wouldn't be possible to move it's logic to db layer, but once, long, long time ago someone decided this to be done on bll, and so it is on bll layer. I understand, that these might come as a problem in arquillian testing, but that's to be resolved, since, not all singletons are for caching. And even if they are, testing framework should be able to cope with such common beans, we shouldn't limit ourselves not to use singletons. Therefore, I wouldn't invest into changing these 'caches', but into allowing more complex setups in our testing. If it's not possible, then 'reset' method is second best solution — we have to use write lock as suggested in cr, and then it should be fine. About drawbacks: ad 1) yes, this comes as an extra problem, one has to deal with tx on his own, that's true. This wasn't part of original solution, but it should be fixed already. ad 2) No. Caching done correctly is done closest to the consumer. I assume you can similarly ruin hibernate l2 cache via accessing data through different channel. But that's kinda common to all caches — if you bypass them, you'll corrupt them. So do not bypass them, or in this case, use them as they was designed. As it have been said, you ask pool for mac, or inform it, that you're going to use your own, and then use it. It means, that it's designed to actually bypass it on all writes. Therefore if someone writes a code using MAC without prior notification to the pool about such action, it would be a problem. To avoid this problem there has to be bigger refactor — pool would have to persist MAC addresses somehow and not vmNicDao, or if moved entirelly to db layer, there would have to be trigger on vmnic table or something like that... ad 3) It was requested, to have at least tens of millions of macs in pool. Forget about loop, initializing this structure for given clusterId is not acceptable even once per request. Loading that structure is quite cheap (now), but not that cheap to allow what you ask for. Moving whole stuff to db layer would be probably beneficial (when it was originally implemented), but it's worthless doing it now. About suggestions: neither of them applies to MacPoolPerCluster — point (3) for example: since pool is not a simple cache of db structure, and does not have corresponding data in db layer, it cannot cache writes and it even does not do any writes at all... —— Surely I can imagine better implementation of this, but it'd require bigger changes whose benefits aren't worth of cost of this change. (I hope that) we have to deal with it. This was original design, and since testing framework (with changed caches or without) should deal with singletons etc. anyways, it's not worthy to change it. If there aren't any better options(I don't know), reinitializing bean can be used (and I apologize for blocking that). I'd avoid bigger rewrites in this area. M. ----- Original Message -----