I agree a stable distributed storage solution is the way to go if we can
find one :)
Distributed storages usually suffer from a large overhead because:
1. They try to be resilient to node failure, which means keeping two
or more copies of the same file, which results in I/O overhead.
2. They need to coordinate metadata access for large amounts of files.
Bottlenecks in the metadata management system are a common issue for
distributes FS storages.
Since most of our data is ephemeral anyway I don't think we need to
pay this overhead.
--
Barak Korren
bkorren(a)redhat.com
RHEV-CI Team