- Users - oVirt List Archives

Direct Linux kernel/initrd boot
by Chris Adams 27 Jul '21

27 Jul '21

I was looking at using direct Linux kernel/initrd booting for a particular automation project, but I'm not sure how to use it. The docs talk about referencing files from an ISO domain, but those are deprecated (and I don't have one). Can the files be in a regular data domain? How would I specify which domain to look in? The docs also say the alternative is to specify a path "on the host", but I guess that requires the files to be copied to the same path on every host that could boot the VM? I guess I can instead make a one-off ISO and upload it to a data domain; it just would be easier to use direct boot. -- Chris Adams <cma(a)cmadams.net>

5 5

Requirements to put to cluster version 4.6
by Gianluca Cecchi 27 Jul '21

27 Jul '21

Hello, I updated all my 3 CentOS 4.4.5 hosts to 4.4.6. Or at least upgrading them from the GUI I see now they are on CentOS 8.4 and: [root@ov200 ~]# rpm -qa| grep 4.4.6 ovirt-release44-4.4.6.3-1.el8.noarch ovirt-host-4.4.6-1.el8.x86_64 ovirt-host-dependencies-4.4.6-1.el8.x86_64 [root@ov200 ~]# How to see from the webadmin GUI or from the Host Console that they are indeed 4.4.6? If I try to set cluster compatibility version to 4.6 I get: Error while executing action: Cannot change Cluster Compatibility Version to higher version when there are active Hosts with lower version. -Please move Host ov300, ov301, ov200 with lower version to maintenance first. I don't remember the need of giving downtime to get new cluster version... Or what are further requirements (and benefits....) to upgrade to 4.6? Thanks, Gianluca

3 8

Expiration date of certificates
by Demeter Tibor 26 Jul '21

26 Jul '21

Dear Listmembers, How can I exctract the expiration dates all certificates which ovirt are uses? Also, is there any command or tool in ovirt what I can use to create and deploy new certifications on hosted engine and hosts? Thanks in advance, Regards, Tibor

2 1

Install an old version of ovirt-engine
by Dominique D 26 Jul '21

26 Jul '21

I would like to test the update on lab my prod setup. How can I install an old version of ovirt engine. when i use the following repo, yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release44.rpm, i only have the choice of the latest version. how do install this version? ovirt-engine-4.4.4.1-1.el8.noarch.rpm thank you

2 1

ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(score) Penalizing score by 1600 due to network status
by Christoph Timm 26 Jul '21

26 Jul '21

Hi List, I'm trying to understand why my hosted engine is moved from one node to another from time to time. It is happening sometime multiple times a day. But there are also days without it. I can see the following in the ovirt-hosted-engine-ha/agent.log: ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(score) Penalizing score by 1600 due to network status After that the engine will be shutdown and started on another host. The oVirt Admin portal is showing the following around the same time: Invalid status on Data Center Default. Setting status to Non Responsive. But the whole cluster is working normally during that time. I believe that I have somehow a network issue on my side but I have no clue what kind of check is causing the network status to penalized. Does anyone have an idea how to investigate this further? Best regards Christoph

2 13

Gluster bricks error
by Patrick Lomakin 24 Jul '21

24 Jul '21

For a long time I have been seeing the same error, which cannot be corrected.After restarting the host which has a Volume of one or more Bricks, the Volume starts with the status "Online", but the Bricks remain "Offline". This leads to having to manually restart Volume, the ovirt-ha and ovirt-broker services, and run the hosted-engine --connect-storage command. And only after that I can start the hosted engine back to normal. I tried this on different server hardware and different operating systems for the host, but the result is the same. This is a very serious flaw that nullifies the high availability in HCI using GlusterFS.Regards!

3 6

Create gluster brick in oVirt engine, RAID-parameters
by Dominique D 23 Jul '21

23 Jul '21

Hi, When creating a gluster brick in oVirt (host->Storage Devices->Create Brick), I have to fill in the parameters of the RAID volume the brick : RAID-type, number of disks and stripe size. My setup is DELL RAID Controler PERC H710 4 SSD disks raid5 Default stripe for Dell RAID-controller defaults is 64KB. I have no choice of a Raid 5 and if I choose Raid6, 4 or 3 disk, 64KB or 128KB Stripe. It does not work, (Failed to create brick data_ssd on host xxx of cluster Default). The only option that works is RAID type = None tried to find information about this, but no luck yet.. I tried ovirt engine 4.4.1.10 and lastest 4.4.7.6 Can I leave this to None ? Performance ? Thank you

2 1

4.4.7 gluster quorum problem
by radchenko.anatoliy＠gmail.com 22 Jul '21

22 Jul '21

Hi guys, one strange thing happens, cannot understand it. Today i installed last version 4.4.7 on centos stream, replica 3, via cockpit, internal lan for sync. Seems all ok, if reboot 3 nodes together aslo ok. But if i reboot 1 node ( and declare node rebooted through web ui) the bricks (engine and data) remain down on that node. All is clear on logs without explicit indication of the situation except "Server quorum regained for volume data. Starting local bricks" on glusterd.log. After "systemctl restart glusterd" bricks gone down on another node. After "systemctl restart glusterd" on that node all is ok. Where should i look? some errors of log that i found: --------------------------------------------- bdocker.log: statusStorageThread::ERROR::2021-07-12 22:17:02,899::storage_broker::223::ovirt_hosted_engine_ha.broker.storage_broker.StorageBroker::(put_stats) Failed to write metadata for ho$ Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 215, in put_stats f = os.open(path, direct_flag | os.O_WRONLY | os.O_SYNC) FileNotFoundError: [Errno 2] No such file or directory: '/run/vdsm/storage/53b068c1-beb8-4048-a766-3a4e71ded624/d3df7eb6-d453-439a-8436-d3694d4b5179/de18b2cc-a4e1-4afc-9b5a-6063$ StatusStorageThread::ERROR::2021-07-12 22:17:02,899::status_broker::90::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(run) Failed to update state. Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 215, in put_stats f = os.open(path, direct_flag | os.O_WRONLY | os.O_SYNC) FileNotFoundError: [Errno 2] No such file or directory: '/run/vdsm/storage/53b068c1-beb8-4048-a766-3a4e71ded624/d3df7eb6-d453-439a-8436-d3694d4b5179/de18b2cc-a4e1-4afc-9b5a-6063$ During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 86, in run entry.data File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 225, in put_stats .format(str(e))) ovirt_hosted_engine_ha.lib.exceptions.RequestError: failed to write metadata: [Errno 2] No such file or directory: '/run/vdsm/storage/53b068c1-beb8-4048-a766-3a4e71ded624/d3df7e$ StatusStorageThread::ERROR::2021-07-12 22:17:02,899::storage_broker::223::ovirt_hosted_engine_ha.broker.storage_broker.StorageBroker::(put_stats) Failed to write metadata for ho$ Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 215, in put_stats f = os.open(path, direct_flag | os.O_WRONLY | os.O_SYNC) FileNotFoundError: [Errno 2] No such file or directory: '/run/vdsm/storage/53b068c1-beb8-4048-a766-3a4e71ded624/d3df7eb6-d453-439a-8436-d3694d4b5179/de18b2cc-a4e1-4afc-9b5a-6063$ StatusStorageThread::ERROR::2021-07-12 22:17:02,899::status_broker::90::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(run) Failed to update state. Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 215, in put_stats f = os.open(path, direct_flag | os.O_WRONLY | os.O_SYNC) FileNotFoundError: [Errno 2] No such file or directory: '/run/vdsm/storage/53b068c1-beb8-4048-a766-3a4e71ded624/d3df7eb6-d453-439a-8436-d3694d4b5179/de18b2cc-a4e1-4afc-9b5a-6063$ During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 86, in run entry.data File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 225, in put_stats .format(str(e))) ovirt_hosted_engine_ha.lib.exceptions.RequestError: failed to write metadata: [Errno 2] No such file or directory: '/run/vdsm/storage/53b068c1-beb8-4048-a766-3a4e71ded624/d3df7e$ StatusStorageThread::ERROR::2021-07-12 22:17:02,902::status_broker::70::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(trigger_restart) Trying to restart the $ StatusStorageThread::ERROR::2021-07-12 22:17:02,902::storage_broker::173::ovirt_hosted_engine_ha.broker.storage_broker.StorageBroker::(get_raw_stats) Failed to read metadata fro$ Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 155, in get_raw_stats f = os.open(path, direct_flag | os.O_RDONLY | os.O_SYNC) FileNotFoundError: [Errno 2] No such file or directory: '/run/vdsm/storage/53b068c1-beb8-4048-a766-3a4e71ded624/d3df7eb6-d453-439a-8436-d3694d4b5179/de18b2cc-a4e1-4afc-9b5a-6063$ StatusStorageThread::ERROR::2021-07-12 22:17:02,902::status_broker::98::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(run) Failed to read state. Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 155, in get_raw_stats f = os.open(path, direct_flag | os.O_RDONLY | os.O_SYNC) FileNotFoundError: [Errno 2] No such file or directory: '/run/vdsm/storage/53b068c1-beb8-4048-a766-3a4e71ded624/d3df7eb6-d453-439a-8436-d3694d4b5179/de18b2cc-a4e1-4afc-9b5a-6063$ During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 94, in run self._storage_broker.get_raw_stats() File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 175, in get_raw_stats .format(str(e))) -------------------------------------------------------supervdsm.log: ainProcess|jsonrpc/0::DEBUG::2021-07-12 22:22:13,264::commands::211::root::(execCmd) /usr/bin/taskset --cpu-list 0-19 /usr/bin/systemd-run --scope --slice=vdsm-glusterfs /usr/b$ MainProcess|jsonrpc/0::DEBUG::2021-07-12 22:22:15,083::commands::224::root::(execCmd) FAILED: <err> = b'Running scope as unit: run-r91d6411af8114090aa28933d562fa473.scope\nMount$ MainProcess|jsonrpc/0::ERROR::2021-07-12 22:22:15,083::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) Error in mount Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", line 97, in wrapper res = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", line 135, in mount cgroup=cgroup) File "/usr/lib/python3.6/site-packages/vdsm/storage/mount.py", line 280, in _mount _runcmd(cmd) File "/usr/lib/python3.6/site-packages/vdsm/storage/mount.py", line 308, in _runcmd raise MountError(cmd, rc, out, err) vdsm.storage.mount.MountError: Command ['/usr/bin/systemd-run', '--scope', '--slice=vdsm-glusterfs', '/usr/bin/mount', '-t', 'glusterfs', '-o', 'backup-volfile-servers=cluster2.$ netlink/events::DEBUG::2021-07-12 22:22:15,131::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 139867781396224)> MainProcess|jsonrpc/4::DEBUG::2021-07-12 22:22:15,134::supervdsm_server::102::SuperVdsm.ServerCallback::(wrapper) return network_caps with {'networks': {'ovirtmgmt': {'ports': [$ ---------------------------------------------------------vdsm.log: 2021-07-12 22:17:08,718+0200 INFO (jsonrpc/7) [api.host] FINISH getStats return={'status': {'code': 0, 'message': 'Done'}, 'info': (suppressed)} from=::1,34946 (api:54) 2021-07-12 22:17:09,491+0200 ERROR (monitor/53b068c) [storage.Monitor] Error checking domain 53b068c1-beb8-4048-a766-3a4e71ded624 (monitor:451) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/monitor.py", line 432, in _checkDomainStatus self.domain.selftest() File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 712, in selftest self.oop.os.statvfs(self.domaindir) File "/usr/lib/python3.6/site-packages/vdsm/storage/outOfProcess.py", line 244, in statvfs return self._iop.statvfs(path) File "/usr/lib/python3.6/site-packages/ioprocess/__init__.py", line 510, in statvfs resdict = self._sendCommand("statvfs", {"path": path}, self.timeout) File "/usr/lib/python3.6/site-packages/ioprocess/__init__.py", line 479, in _sendCommand raise OSError(errcode, errstr) FileNotFoundError: [Errno 2] No such file or directory 2021-07-12 22:17:09,619+0200 INFO (jsonrpc/0) [api.virt] START getStats() from=::1,34946, vmId=9167f682-3c82-4237-93bd-53f0ad32ffba (api:48) 2021-07-12 22:17:09,620+0200 INFO (jsonrpc/0) [api] FINISH getStats error=Virtual machine does not exist: {'vmId': '9167f682-3c82-4237-93bd-53f0ad32ffba'} (api:129) 2021-07-12 22:17:09,620+0200 INFO (jsonrpc/0) [api.virt] FINISH getStats return={'status': {'code': 1, 'message': "Virtual machine does not exist: {'vmId': '9167f682-3c82-4237-$ 2021-07-12 22:17:09,620+0200 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call VM.getStats failed (error 1) in 0.00 seconds (__init__:312) 2021-07-12 22:17:10,034+0200 INFO (jsonrpc/3) [vdsm.api] START repoStats(domains=['53b068c1-beb8-4048-a766-3a4e71ded624']) from=::1,34946, task_id=4e823c98-f95b-45f7-ad64-90f82$ 2021-07-12 22:17:10,034+0200 INFO (jsonrpc/3) [vdsm.api] FINISH repoStats return={'53b068c1-beb8-4048-a766-3a4e71ded624': {'code': 2001, 'lastCheck': '0.5', 'delay': '0', 'vali$ 2021-07-12 22:17:10,403+0200 INFO (health) [health] LVM cache hit ratio: 12.50% (hits: 1 misses: 7) (health:131) 2021-07-12 22:17:10,472+0200 INFO (MainThread) [vds] Received signal 15, shutting down (vdsmd:74) 2021-07-12 22:17:10,472+0200 INFO (MainThread) [root] Stopping DHCP monitor. (dhcp_monitor:106) 2021-07-12 22:17:10,473+0200 INFO (ioprocess/11056) [IOProcessClient] (53b068c1-beb8-4048-a766-3a4e71ded624) Poll error 16 on fd 74 (__init__:176) 2021-07-12 22:17:10,473+0200 INFO (ioprocess/11056) [IOProcessClient] (53b068c1-beb8-4048-a766-3a4e71ded624) ioprocess was terminated by signal 15 (__init__:200) 2021-07-12 22:17:10,476+0200 INFO (ioprocess/19109) [IOProcessClient] (e10cbd59-d32e-4b69-a4c1-d213e7bd8973) Poll error 16 on fd 75 (__init__:176) 2021-07-12 22:17:10,476+0200 INFO (ioprocess/19109) [IOProcessClient] (e10cbd59-d32e-4b69-a4c1-d213e7bd8973) ioprocess was terminated by signal 15 (__init__:200) 2021-07-12 22:17:10,513+0200 INFO (ioprocess/44046) [IOProcess] (e10cbd59-d32e-4b69-a4c1-d213e7bd8973) Starting ioprocess (__init__:465) 2021-07-12 22:17:10,513+0200 INFO (ioprocess/44045) [IOProcess] (53b068c1-beb8-4048-a766-3a4e71ded624) Starting ioprocess (__init__:465) 2021-07-12 22:17:10,519+0200 WARN (periodic/0) [root] Failed to retrieve Hosted Engine HA info: timed out (api:198) 2021-07-12 22:17:10,611+0200 ERROR (check/loop) [storage.Monitor] Error checking path /rhev/data-center/mnt/glusterSD/cluster1.int:_data/e10cbd59-d32e-4b69-a4c1-d213e7bd8973/dom$ Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/monitor.py", line 507, in _pathChecked delay = result.delay() File "/usr/lib/python3.6/site-packages/vdsm/storage/check.py", line 391, in delay raise exception.MiscFileReadException(self.path, self.rc, self.err) vdsm.storage.exception.MiscFileReadException: Internal file read failure: ('/rhev/data-center/mnt/glusterSD/cluster1.int:_data/e10cbd59-d32e-4b69-a4c1-d213e7bd8973/dom_md/metada$ 2021-07-12 22:17:10,860+0200 INFO (MainThread) [root] Stopping Bond monitor. (bond_monitor:53) Thanks in advance Best regards.

2 4

PostgreSQL password for ovirt 4.4.7
by louisb＠ameritech.net 22 Jul '21

22 Jul '21

I've finally installed successfully ovirt 4.4.7 using the default installation. I would like to change the passwords for the following userids: postgres, engine database and data warehouse database. Can someone provide me with information grading the files that contain the password to these items? I would also like to know the best way to change the passwords. Thanks

2 1

Cannot start hosted-engine
by Valerio Luccio 21 Jul '21

21 Jul '21

Hi list, I have a hosted engine running on a CentOS 8. The engine and all the VM's are stored on a 4-node gluster. I had some issues with the gluster and then the hosted-engine stopped working (even though the virtualization dashboard showed 4 virtual machines running). I tried to "systemctl restart" the hosted-engine, but it failed. I try to reboot the server and the hosted-engine still will not come up. Note that the server has no issue mounting the gluster: $ df hydra1:/MRIData 390664407040 20530130012 370134277028 6% /rhev/data-center/mnt/glusterSD/hydra1:_MRIData $ ls -l /rhev/data-center/mnt/glusterSD/hydra1\:_MRIData/6547dc22-b89e-4f14-8958-c9e8d27b29a4/ drwxr-xr-x. 2 vdsm kvm 4.0K Mar 29 12:24 dom_md drwxr-xr-x. 2 vdsm kvm 4.0K Jul 20 17:47 ha_agent drwxr-xr-x. 12 vdsm kvm 4.0K Apr 1 16:32 images drwxr-xr-x. 4 vdsm kvm 4.0K Mar 29 12:24 master Where "hydra1" is one of my gluster nodes and MRIData is the volume name. Here is the relevant snippet from /var/log/ovirt-hosted-engine-ha/agent.log MainThread::INFO::2021-07-20 17:29:07,584::agent::67::ovirt_hosted_engine_ha.agent.agent.Agent::(run) ovirt-hosted-engine-ha agent 2.4.6 started MainThread::INFO::2021-07-20 17:29:07,594::hosted_engine::242::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_hostname) Certificate common name not found, using hostname to identify host MainThread::INFO::2021-07-20 17:29:07,635::hosted_engine::548::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Initializing ha-broker connection MainThread::INFO::2021-07-20 17:29:07,636::brokerlink::82::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Starting monitor network, options {'addr': '192.168.39.65', 'network_test': 'dns', 'tcp_t_address': '', 'tcp_t_port': ''} MainThread::ERROR::2021-07-20 17:29:07,636::hosted_engine::564::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Failed to start necessary monitors MainThread::ERROR::2021-07-20 17:29:07,637::agent::143::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 85, in start_monitor response = self._proxy.start_monitor(type, options) File "/usr/lib64/python3.6/xmlrpc/client.py", line 1112, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python3.6/xmlrpc/client.py", line 1452, in __request verbose=self.__verbose File "/usr/lib64/python3.6/xmlrpc/client.py", line 1154, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib64/python3.6/xmlrpc/client.py", line 1166, in single_request http_conn = self.send_request(host, handler, request_body, verbose) File "/usr/lib64/python3.6/xmlrpc/client.py", line 1279, in send_request self.send_content(connection, request_body) File "/usr/lib64/python3.6/xmlrpc/client.py", line 1309, in send_content connection.endheaders(request_body) File "/usr/lib64/python3.6/http/client.py", line 1249, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib64/python3.6/http/client.py", line 1036, in _send_output self.send(msg) File "/usr/lib64/python3.6/http/client.py", line 974, in send self.connect() File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 74, in connect self.sock.connect(base64.b16decode(self.host)) FileNotFoundError: [Errno 2] No such file or directory During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent return action(he) File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper return he.start_monitoring() File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 437, in start_monitoring self._initialize_broker() File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 561, in _initialize_broker m.get('options', {})) File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 91, in start_monitor ).format(t=type, o=options, e=e) ovirt_hosted_engine_ha.lib.exceptions.RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'addr': '192.168.39.65', 'network_test': 'dns', 'tcp_t_address': '', 'tcp_t_port': ''}] MainThread::ERROR::2021-07-20 17:29:07,637::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to restart agent MainThread::INFO::2021-07-20 17:29:07,637::agent::89::ovirt_hosted_engine_ha.agent.agent.Agent::(run) Agent shutting down I'm puzzled by that "Certificate common name not found", which I had not seen before. The fqdn of the hosted engine resolves fine on the server, so does the fqdn of the server itself. The ip address it seems to try to use for the network is that of one of the university's gateways. Any ideas ? Any way to debug this further ? Thanks in advance, -- Valerio Luccio (212) 998-8736 Center for Brain Imaging 4 Washington Place, Room 157 New York University New York, NY 10003 "In an open world, who needs windows or gates ?"

3 3