I recently set up a Windows 2003 64-bit server, with one virtual host, also Windows 2003 64-bit.
The performance was terrible. Both servers would run, but when any load was put on them - copying files, clients connecting, they would both run extremely slowly.
Yesterday I read some messages in the forum about slow VM performance, and turned off debugging, which did help a bit.
Later on though, I discovered that for some reason, write caching was totally disabled on the C: drive of the physical server - the VM c: was also hosted on this C: drive. After turning on write cachine, performance of both hosts is excellent.
Just wanted to mention this. Check your write caching!