<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Hi Sandro
<div class=""><br class="">
</div>
<div class="">I was was trying both</div>
<div class=""><br class="">
</div>
<div class="">Here is the debug version...</div>
<div class=""><br class="">
</div>
<div class="">
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">[root@powerhost1 ~]# engine-iso-uploader -v upload -i iso /tmp/rhel-server-6.6-x86_64-dvd.iso</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to abort): </div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: API Vendor(<a href="http://ovirt.org" class="">ovirt.org</a>)<span class="Apple-tab-span" style="white-space:pre">
</span>API Version(3.5.0)</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: id=1328bbfe-f05a-4779-ac34-560b213ece11 address=192.168.1.32 path=/Volumes/Raid1/iso</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">Uploading, please wait...</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: local NFS mount point is /tmp/tmpCjZhA_</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: NFS mount command (/bin/mount -t nfs -o rw,sync,soft 192.168.1.32:/Volumes/Raid1/iso /tmp/tmpCjZhA_)</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: /bin/mount -t nfs -o rw,sync,soft 192.168.1.32:/Volumes/Raid1/iso /tmp/tmpCjZhA_</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: _cmds(['/bin/mount', '-t', 'nfs', '-o', 'rw,sync,soft', '192.168.1.32:/Volumes/Raid1/iso', '/tmp/tmpCjZhA_'])</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: returncode(32)</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: STDOUT()</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: STDERR(mount.nfs: Connection timed out</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">)</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">ERROR: mount.nfs: Connection timed out</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo; min-height: 13px;" class="">
<br class="">
</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: /bin/umount -t nfs -f /tmp/tmpCjZhA_</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: /bin/umount -t nfs -f /tmp/tmpCjZhA_</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: _cmds(['/bin/umount', '-t', 'nfs', '-f', '/tmp/tmpCjZhA_'])</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: returncode(1)</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: STDOUT()</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: STDERR(umount2: Invalid argument</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">umount: /tmp/tmpCjZhA_: not mounted</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">)</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">DEBUG: umount2: Invalid argument</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">umount: /tmp/tmpCjZhA_: not mounted</div>
</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class=""><br class="">
</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class=""><br class="">
</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class=""><br class="">
</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class="">Could this be associated with the auto negotiation of the nfs version? Is there a way I can specify nfs3 on the mount or on the export(os x)</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;" class=""><br class="">
</div>
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Jan 28, 2015, at 6:11 AM, Sandro Bonazzola <<a href="mailto:sbonazzo@redhat.com" class="">sbonazzo@redhat.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Il
28/01/2015 16:05, Maor Lipchuk ha scritto:</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br class="">
<br class="">
<br class="">
<br class="">
----- Original Message -----<br class="">
<blockquote type="cite" class="">From: "Roy Golan" <<a href="mailto:rgolan@redhat.com" class="">rgolan@redhat.com</a>><br class="">
To: "Mikola Rose" <<a href="mailto:mrose@power-soft.com" class="">mrose@power-soft.com</a>>, "Maor Lipchuk" <<a href="mailto:mlipchuk@redhat.com" class="">mlipchuk@redhat.com</a>><br class="">
Cc: <a href="mailto:users@ovirt.org" class="">users@ovirt.org</a><br class="">
Sent: Wednesday, January 28, 2015 12:02:50 PM<br class="">
Subject: Re: [ovirt-users] oVirt 3.5.1 - VM "hostedengine" Failing to start<br class="">
<br class="">
On 01/28/2015 11:04 AM, Mikola Rose wrote:<br class="">
<blockquote type="cite" class="">-rwxr-xr-x 1 vdsm kvm 0 Jan 28 00:59 /Volumes/Raid1/__DIRECT_IO_TEST__<br class="">
-rwxr-xr-x 1 vdsm kvm 0 Jan 28 00:55<br class="">
/Volumes/Raid1/data/__DIRECT_IO_TEST__<br class="">
-rwxrwxrwx 1 vdsm kvm 0 Jan 28 00:55<br class="">
/Volumes/Raid1/iso/__DIRECT_IO_TEST__<br class="">
-rwxr-xr-x 1 vdsm kvm 0 Jan 26 09:43<br class="">
/Volumes/Raid1/vm/__DIRECT_IO_TEST__<br class="">
<br class="">
Note, since removing the file the hosted_engine1 is starting up and<br class="">
everything seems fine but then all of a sudden it restarts.<br class="">
<br class="">
</blockquote>
please attach some logs.<br class="">
<blockquote type="cite" class="">Also may be related...<br class="">
<br class="">
root@powerhost1 ~]# engine-iso-uploader --ssh-user=36 upload -i iso<br class="">
rhel-server-6.6-x86_64-dvd.iso<br class="">
Please provide the REST API password for the admin@internal oVirt<br class="">
Engine user (CTRL+D to abort):<br class="">
Uploading, please wait...<br class="">
INFO: Start uploading rhel-server-6.6-x86_64-dvd.iso<br class="">
ERROR: Unable to copy rhel-server-6.6-x86_64-dvd.iso to ISO storage<br class="">
domain on iso.<br class="">
ERROR: Error message is "unable to test the available space on<br class="">
/Volumes/Raid1/iso"<br class="">
</blockquote>
<br class="">
Maor?<br class="">
</blockquote>
<br class="">
<br class="">
Hi,<br class="">
<br class="">
I think this is related to the engine-iso-uploader tool,<br class="">
Sandro, do you have any insights about this?<br class="">
</blockquote>
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Please
don't use UIDs like 36 as user.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">a
command line like:</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">ovirt-iso-uploader
--ssh-user=root upload -i iso rhel-server-6.6-x86_64-dvd.iso</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">or
like</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">ovirt-iso-uploader
--ssh-user=your_user_in_kvm_group upload -i iso rhel-server-6.6-x86_64-dvd.iso</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">should
work.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br class="">
Regards,<br class="">
Maor<br class="">
<br class="">
<blockquote type="cite" class="">
<blockquote type="cite" class=""><br class="">
<br class="">
<br class="">
<br class="">
<br class="">
<br class="">
<blockquote type="cite" class="">On Jan 28, 2015, at 12:53 AM, Roy Golan <<a href="mailto:rgolan@redhat.com" class="">rgolan@redhat.com</a><br class="">
<<a href="mailto:rgolan@redhat.com" class="">mailto:rgolan@redhat.com</a>>> wrote:<br class="">
<br class="">
On 01/28/2015 03:10 AM, Mikola Rose wrote:<br class="">
<blockquote type="cite" class="">Hi there,<br class="">
<br class="">
<br class="">
I seem to have run into a problem. my hosted engine vm is failing<br class="">
to start;<br class="">
<br class="">
vdsm.log;<br class="">
<br class="">
<br class="">
Thread-20::DEBUG::2015-01-27<br class="">
16:53:37,999::fileSD::152::Storage.StorageDomain::(__init__) Reading<br class="">
domain in path<br class="">
/rhev/data-center/mnt/192.168.1.32:_Volumes_Raid1/443b4931-667f-441f-98d8-51384e67a0af<br class="">
Thread-20::ERROR::2015-01-27<br class="">
16:53:38,000::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain)<br class="">
Error while collecting domain 443b4931-667f-441f-98d8-51384e67a0af<br class="">
monitoring information<br class="">
Traceback (most recent call last):<br class="">
File "/usr/share/vdsm/storage/domainMonitor.py", line 221, in<br class="">
_monitorDomain<br class="">
self.domain = sdCache.produce(self.sdUUID)<br class="">
File "/usr/share/vdsm/storage/sdc.py", line 98, in produce<br class="">
domain.getRealDomain()<br class="">
File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain<br class="">
return self._cache._realProduce(self._sdUUID)<br class="">
File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce<br class="">
domain = self._findDomain(sdUUID)<br class="">
File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain<br class="">
dom = findMethod(sdUUID)<br class="">
File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain<br class="">
return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID))<br class="">
File "/usr/share/vdsm/storage/fileSD.py", line 159, in __init__<br class="">
validateFileSystemFeatures(sdUUID, self.mountpoint)<br class="">
File "/usr/share/vdsm/storage/fileSD.py", line 88, in<br class="">
validateFileSystemFeatures<br class="">
oop.getProcessPool(sdUUID).directTouch(testFilePath)<br class="">
File "/usr/share/vdsm/storage/outOfProcess.py", line 320, in<br class="">
directTouch<br class="">
ioproc.touch(path, flags, mode)<br class="">
File "/usr/lib/python2.6/site-packages/ioprocess/__init__.py",<br class="">
line 507, in touch<br class="">
self.timeout)<br class="">
File "/usr/lib/python2.6/site-packages/ioprocess/__init__.py",<br class="">
line 391, in _sendCommand<br class="">
raise OSError(errcode, errstr)<br class="">
OSError: [Errno 13] Permission denied<br class="">
<br class="">
I assume this is an nfs issue so I checked to see if I could create<br class="">
a file in the mounts from the host machine, which I could.<br class="">
<br class="">
My test bed is usign an old OS X NFS via xraid adn the export is ;<br class="">
/Volumes/Raid1 -maproot=root:wheel -network 192.168.1.0 -mask<br class="">
255.255.255.0<br class="">
<br class="">
drwxr-xr-x 6 vdsm kvm 204 Jan 27 16:30<br class="">
443b4931-667f-441f-98d8-51384e67a0af<br class="">
drwxr-xr-x 4 vdsm kvm 136 Jan 27 12:32 data<br class="">
drwxr-xr-x 4 vdsm kvm 136 Jan 27 00:18 iso<br class="">
drwxr-xr-x 3 vdsm kvm 102 Jan 26 09:43 vm<br class="">
<br class="">
the host machine seems to mount the shares fine....<br class="">
<br class="">
drwxr-xr-x 5 vdsm kvm 4096 Jan 27 12:28 .<br class="">
drwxr-xr-x 3 vdsm kvm 4096 Jan 27 10:04 ..<br class="">
drwxr-xr-x 13 vdsm kvm 544 Jan 27 00:12 192.168.1.32:_Volumes_Raid1<br class="">
drwxr-xr-x 2 vdsm kvm 4096 Jan 27 00:12<br class="">
192.168.1.32:_Volumes_Raid1_data<br class="">
drwxr-xr-x 2 vdsm kvm 4096 Jan 27 00:24 192.168.1.32:_Volumes_Raid1_iso<br class="">
<br class="">
and as I said above I can create files in any one of those mounts<br class="">
<br class="">
<br class="">
Is there a place I can look at to find the offending file? If that<br class="">
is the issue... Oddly enough everything worked until i rebooted, so<br class="">
I must have either changed something or something is buggered.<br class="">
<br class="">
</blockquote>
<br class="">
please echo the output of<br class="">
<br class="">
find /Volumes/Raid1 -name "__DIRECT_IO_TEST__" | xargs ls -la<br class="">
<br class="">
the failure is to create this file.<br class="">
<br class="">
<blockquote type="cite" class=""><br class="">
<br class="">
<br class="">
<br class="">
_______________________________________________<br class="">
Users mailing list<br class="">
<a href="mailto:Users@ovirt.org" class="">Users@ovirt.org</a><br class="">
http://lists.ovirt.org/mailman/listinfo/users<br class="">
</blockquote>
<br class="">
</blockquote>
<br class="">
Mik Rose | Manager, IT - Support Services | *PowerSoft Development Corp*<br class="">
1 (250) 642-0295 x23 <a href="http://www.power-soft.com" class="">http://www.power-soft.com</a><br class="">
<<a href="http://www.power-soft.com/" class="">http://www.power-soft.com/</a>> Live Support<br class="">
<<a href="https://secure.logmeinrescue.com/Customer/Download.aspx?EntryID=15095831" class="">https://secure.logmeinrescue.com/Customer/Download.aspx?EntryID=15095831</a>><br class="">
This e-mail may be privileged and/or confidential, and the sender does<br class="">
not waive<br class="">
any related rights and obligations. Any distribution, use or copying<br class="">
of this e-mail or the information<br class="">
it contains by other than an intended recipient is unauthorized.<br class="">
If you received this e-mail in error, please advise me (by return<br class="">
e-mail or otherwise) immediately.<br class="">
<span class="Apple-tab-span" style="white-space: pre;"></span><br class="">
<br class="">
<br class="">
<br class="">
<br class="">
<br class="">
</blockquote>
<br class="">
<br class="">
</blockquote>
</blockquote>
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">--<span class="Apple-converted-space"> </span></span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Sandro
Bonazzola</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Better
technology. Faster innovation. Powered by community collaboration.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">See
how it works at<span class="Apple-converted-space"> </span></span><a href="http://redhat.com/" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">redhat.com</a></div>
</blockquote>
</div>
<br class="">
<div class="">
<div class="">
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" style="margin-left: 1.65pt; border-collapse: collapse;">
<tbody class="">
<tr style="height: 26.25pt;" class="">
<td width="421" valign="top" style="orphans: 2; text-align: -webkit-auto; widows: 2; width: 315.75pt; padding: 0in 5.4pt; height: 26.25pt;" class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size: 9pt; font-family: Helvetica, sans-serif; color: rgb(31, 73, 125);" class="">Mik Rose | Manager, IT - Support Services | </span><b class=""><span lang="EN-CA" style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(0, 102, 0);" class="">PowerSoft
Development Corp<o:p class=""></o:p></span></b></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size: 8pt; font-family: Helvetica, sans-serif; color: rgb(31, 73, 125);" class="">1 (250) 642-0295 x23 <a href="http://www.power-soft.com/" style="color: purple;" class=""><span style="color: black;" class="">http://www.power-soft.com</span></a> <a href="https://secure.logmeinrescue.com/Customer/Download.aspx?EntryID=15095831" target="_blank" style="color: purple;" class=""><span style="color: black;" class="">Live
Support</span></a></span><span style="font-size: 9pt; font-family: Helvetica, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt 3.75pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size: 9pt; font-family: Helvetica, sans-serif; color: rgb(31, 73, 125);" class=""> </span></div>
<div style="margin: 0in 0in 0.0001pt 3.75pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<div style="font-size: 11pt; margin: 0in 0in 0.0001pt;" class=""><span style="font-size: 7pt; font-family: Arial, sans-serif; color: rgb(31, 73, 125);" class="">This e-mail may be privileged and/or confidential, and the sender does not waive<o:p class=""></o:p></span></div>
<div style="font-size: 11pt; margin: 0in 0in 0.0001pt;" class=""><span style="font-size: 7pt; font-family: Arial, sans-serif; color: rgb(31, 73, 125);" class="">any related rights and obligations. Any distribution, use or copying of this e-mail or the information<o:p class=""></o:p></span></div>
<div style="font-size: 11pt; margin: 0in 0in 0.0001pt;" class=""><span style="font-size: 7pt; font-family: Arial, sans-serif; color: rgb(31, 73, 125);" class="">it contains by other than an intended recipient is unauthorized.<o:p class=""></o:p></span></div>
<div style="font-size: 11pt; margin: 0in 0in 0.0001pt;" class=""><span style="font-size: 7pt; font-family: Arial, sans-serif; color: rgb(31, 73, 125);" class="">If you received this e-mail in error, please advise me (by return e-mail or otherwise) immediately.</span></div>
</div>
</td>
<td width="69" valign="top" style="orphans: 2; text-align: -webkit-auto; widows: 2; width: 51.75pt; padding: 0in 5.4pt; height: 26.25pt;" class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="color: rgb(31, 73, 125);" class=""><br class="Apple-interchange-newline" style="text-align: -webkit-auto;">
</span></div>
</td>
</tr>
</tbody>
</table>
<div class=""><br class="">
</div>
</div>
<div class=""><br class="">
</div>
<br class="Apple-interchange-newline">
</div>
<br class="">
</div>
</body>
</html>