Can I make Virtual PC use multiple CPU’s?

Virtual Server can leverage multiple CPU’s (although it only allocates a single CPU per Guest operating system). Virtual PC can only use a single CPU split among all of the Guest operating systems.

First, let me state without question: This is an unsupported configuration. I am writing this up only for the few people out there who may have non-production environments where they do not want to leverage Virtual Server/are using Virtual PC/are using Windows Server 2003 SP1/have a multiple processor host that they would really like to be able to run more Guest operating systems but are bottlenecked due to the processor (not too restrictive eh?).

At one point in time, I was using Virtual PC to provide an environment to test a large number of operating systems (I was testing MOM 2005 agent deployment among other items). My configuration was a dual-processor system with two gigs of memory. What I found was that my processor was being maxed out running 2-3 guest operating systems but I still had plenty of memory and wasn’t bottlenecked on disk access either. So enough of the background, what’s the trick?

Windows Server 2003 supports up to three total remote connections: 1 to the console and 2 terminal server connections for remote administration. Connect to the console and run Virtual PC. Set the affinity for the process to CPU 0 (task manager). Next connect to the first terminal server connection, run Virtual PC and set the affinity for the process to CPU 1. In theory this would support up to three CPU’s (one for the console, and two for the virtual connections), leaving the fourth dedicated to non-Virtual PC processing.

Host SCSI Adapter Cache Configuration

If you are using a SCSI adapter which has cache available on it (on your Virtual Server or Virtual PC Host), the default configuration is often set to 50% of the cache used for read and 50% used for write. As identified in previous posts, the amount of time spent reading and writing within your virtuals can easily be identified (see “Compressing Virtuals” for details on this). To determine if your cache is configured correctly:

1. Identify the amount of data read and written from all virtuals.
2. Calculate the percentage of the time spent on read versus write.
3. Benchmark your current system configuration.
4. If the percentage of time spent doesn’t match the adapter controller’s cache configuration, update it.
5. Re-benchmark your system with the new configuration.
6. If it benchmarks higher, excellent! If not, reverse the changes.

As an example, our SCSI controller was set for 50% read and 50% write. The majority (about 75%) of the amount of data was being written not read. We updated the cache to use 75% read/25% write and re-benchmarked. Our benchmark results were positive so we have left the configuration in place to gather feedback from the users of the Guest operating systems and from further statistics that we gather.

Please note that cache ratio’s may only be able to be changed if the SCSI RAID controller has a battery-backed cache.

Parting Thoughts: If you are using SCSI and a controller with cache, don’t assume that the default configuration will be best for your environment!

The following are services I thought should be evaluated for disabling within the Guest Operating System:

• Application Experience Lookup Service: Auto-updates programs to run on the latest service packs and Windows Operating System. Details available at:  http://support.microsoft.com/?scid=kb;en-us;902196
• Automatic Updates: Download and install of Windows Critical updates. If you disable this you will need to patch through another method (such as visiting the Windows Update website, WSUS server, etc).
• Computer Browser: Provides the list of computers on the network for features such as My Network Places and the dos based Net View command.
• DHCP Client: Requests and registers dynamic IP addresses. Not required if you are hard-coding your IP addresses which is recommended in Virtual environments.
• Error Reporting Services: Used for error reporting to Microsoft.
• Help and Support: Used for the Help and Support icon on the start button.
• Print Spooler: Required for printing from the Guest operating system.
• Remote Registry: Stops remote systems from being able to access the system’s registry remotely.
• Secondary Logon: Provides the ability to use the “Run As” option.
• Shell Hardware Detection: Provides AutoPlay functionality for items such as cd-roms.
• Task Scheduler: Unless you are using scheduled tasks (control panel/scheduled tasks) in the Guest operating system.
• Telephony & Remote Access Connection Manager: Dial-up and other remote connections.
• Windows Time: Guest operating systems will auto-synchronize with the Host operating system by default.
• Wireless Configuration: Used for Wireless Configurations and 802.1x wired connections.

Each of these services may or may not be able to be disabled depending on the functionality required by the server; this is not a blanket statement that these services are not required.

Details on these services (as of Windows 2000 Server) can be found at: http://www.microsoft.com/technet/prodtechnol/windows2000serv/deploy/prodspecs/win2ksvc.mspx. Another good source for service information is:

http://www.theeldergeek.com/services_guide.htm

In testing I found that when all of the above services were disabled it resulted in an approximately 20 mb decrease in memory requirements on the guest Operating System.

Parting Thoughts: The decrease in the amount of memory required is not very much considering the amount of services which had to be disabled to generate it. Unless you are running a large number of Virtual’s this seems to be more trouble than it’s worth.

Does defragmentation really help performance in Virtual PC?

Another common recommendation for improving performance for Virtual PC or Virtual Server is to defragment both the Host and Guest systems. To test this theory, I did performance tests before and after defragmentation of the Host and then another round of tests after defragmenting both the Host and Guest operating systems. The summary of my tests are as follows:

Host Defragmentation Only:

6% Increase in Read IOps

3% Increase in Write IOps

4.5% Increase in Read/Write IOps (assuming 50/50 split of Read/Write)

Host and Guest Defragmentation:

17% Increase in Read IOps

8% Increase in Write IOps

12.5% Increase in Read/Write IOps (assuming 50/50 split of Read/Write)

IOps = IO's per second / the higher the better

MBps = MB's per second / the higher the better

Latency is the time required to access the disk, the lower the better

Diskeeper 10 was used to perform defragmentation on both the Host and the Guest Operating Systems. Both the Guest and Host were listed as critical when testing started, both were at Good or Warning when tests were performed after defragmentation.

VPC Testing Environment: Virtual PC tests were done on Virtual PC with SP1 32-bit platform running Windows XP SP2 for the Host operating system and Windows 2003 SP1 for the Guest operating system. The Host has a single drive connected via Firewire configuration running on a single processor Pentium 4 (1.7) with two gigs of memory. The Guest operating system was configured with 512 mb of memory and running SCOM2007 Beta 2 to simulate active applications. SQLIO was used to gather these metrics and per Microsoft: “Tests represented are based upon a synthetic workload generator and should not be depicted as representative of the performance of any current or future Microsoft product”.

Parting Thoughts: Defragmentation does help with performance/benefits can be gained from defragmentation of both the Host and Guest Operating Systems.

Performance: IDE vs. USB 2.0 vs. Firewire

For my laptop Virtual PC environment I use a total of three drives to spread out the disk access. My laptop’s primary drive (a 5400 IDE) provides the host operating system and if required also provides one lightly used Virtual. I have an externally connected Firewire drive which I use for most of my Virtuals, and when required I also attach a USB 2.0 drive for Virtuals. This allows me to spread the drive access across a total of three spindles. I did some testing to determine how my IDE drive performed versus the same drive attached as either USB 2.0 or Firewire. The results are as follows:

5400 IDE:        526 IOps / 17 MBps / 124 ms Latency

7200 Firewire: 560 IOps / 20 MBps / 92 ms Latency

7200 USB 2.0: 280 IOps / 16 MBps / 112 ms Latency

IOps = IO's per second / the higher the better

MBps = MB's per second / the higher the better

Latency is the time required to access the disk, the lower the better

The same drive connected via Firewire performed significantly better than when it was connected via USB 2.0. It achieved double the amount of IOps, 25% more MBps, and had 20 ms less average latency.

Local drive performance was slightly lower than Firewire in IOps, MBps and had an average of 32 ms higher average latency. This may be due to the fact that the drive tested was a 5400 rpm and was being compared to a 7200 rpm Firewire attached drive.

I have had issues when daily-chaining my Firewire drives together and have decided not to continue down that path. As a next step for my virtual performance tuning, I am considering purchasing a PCCard Firewire connector to connect the two additional Firewire drives that I have available and running all virtuals from only Firewire connected devices.

VPC Testing Environment: Testing computer running Windows XP SP2. The IDE drive tested was a 5400 RPM Hitachi. The USB 2.0 and Firewire drives were both the same 7200 Western Digital which has connections for both USB 2.0 and Firewire. The system tested is a single processor Pentium 4 (1.7) with two gigs of memory. SQLIO was used to gather these metrics and per Microsoft: “Tests represented are based upon a synthetic workload generator and should not be depicted as representative of the performance of any current or future Microsoft product”.

Parting Thoughts: Consider using Firewire instead of USB 2.0 for your Virtuals. You may have better performance as a result!

Virtual Server SCSI Guest Drive Performance

Its common knowledge to use high performance drives on the Host operating system for a Virtual Server when possible to speed up performance of the Guest operating systems. It even says in the documentation to use SCSI drives as a recommendation for performance enhancement of Virtual Server. Discussions on the web state that the SCSI adapter is slower when installing Operating Systems, but once the VMAdditions are installed the SCSI Guest is faster than an IDE Guest drive. Taking a similar approach used to tests done in a previous entry (Compressing Virtuals) I decided to run the SCSI Guest drive through its paces versus an IDE Guest drive. One question came up the same based upon my testing:

What percentage of the time that the drive is in use reading versus writing on the Guest system?

On Virtual Server disk writes to the SCSI were slower (49%) when compared with an IDE drive, and disk reads were also slower (20%).

As mentioned in a previous blog, the read/write can be determined through performance counters or on the configuration screen in the Virtual Server web interface.

On SCSI Guest disk writes averaged 49% less IO’s per second than the IDE Guest drive/disk reads on the SCSI Guest drives averaged 20% less IO’s per second than the IDE Guest drive. So on a drive that averaged 50% read/50% write the decrease in IO’s per second is an average of a 35%. On a drive that averaged 70% read/30% write the decrease is about 29%. On a system that is the reverse (30% read, 70% write) the decrease is about 40%.

VS Testing Environment: Virtual Server tests were done on Virtual Server 2005 R2 on a 32-bit platform running Windows 2003 SP1 for the Host operating system and Windows 2003 SP1 for the Guest operating system (latest version of VMAdditions available is installed as of 07/13/06 which is 13.552). The Host has a four drive SCSI RAID5 configuration running on a quad processor Xeon with six gigs of memory. The Guest operating system was configured with 128 mb of memory and was only running the host operating system. The Guest operating system was used to compare the performance of an IDE drive to a SCSI drive. SQLIO was used to gather these metrics and per Microsoft "Tests represented are based upon a synthetic workload generator and should not be depicted as representative of the performance of any current or future Microsoft product". 

Parting Thoughts: If you have a Virtual Server environment where you need SCSI devices (such as to cluster, or the need for drives larger than those supported by IDE) go ahead and use them. Otherwise, you are better off using a plain old IDE drive within your Guest operating system.

To Compress or not to Compress Virtual Server / Virtual PC Files

I have been using Virtual PC and Virtual Server since they were released. When I created more Virtuals than I could store on my drives I finally broke down and compressed the VHD files. I figured that as long as performance wasn’t too bad I could save the drive space and keep creating virtuals as needed. What I found over time was unexpected. It seemed to me that my compressed virtuals actually ran faster than my non-compressed ones. I decided to run some tests and see if this is a true or false statement. My initial tests included times to boot and shut down Guest Virtuals and copy files – the performance was very similar. I used IoMeter and SySoft Sandra and came up with results which indicated faster disk access on compressed versus uncompressed. Finally I installed SQLIO (per Microsoft "Tests represented are based upon a synthetic workload generator and should not be depicted as representative of the performance of any current or future Microsoft product") and ran full performance tests of the same Virtual both compressed and uncompressed. The results confirmed my gut feeling but were still very surprising!

Compressed Virtuals often perform better then Uncompressed Virtuals. 

There are two major factors which need to be determined when determining if you should or should not compress:

1)     How much processing power is available on the Host system? Using compressed VHD’s does increase the CPU utilization on the Host system. During drive stress testing 100% of the CPU was used on the Guest and one processor on the Host was also maxed. This does not occur during standard drive usage / only during stress testing of the drive. During standard drive usage the overhead on the processor was negligible on both VPC and Virtual Server. However, if your Host system is CPU bottlenecked compressing the Virtuals is not a good idea.

2)     What percentage of the time that the drive is in use reading versus writing on the Guest system?

On Virtual Server disk writes were slower (22%) on a compressed drive, but disk reads were faster (52%). The amount of time that the Guest operating system is reading or writing can be found through using perfmon with the physical disk\%Disk Write Time and %Disk Read Time counters. Add the two together and divide to finding the percentage. Example: 80% read time, 50% write time = 130 total. 80/130 is the percentage of time spend reading (in this case 62%). The amount read/written to the drive can also be seen on the configuration screen for the Guest OS on the Disk I/O line in Virtual Server web interface.
On Virtual PC disk writes were faster (57%) on a compressed drive, and disk reads were also faster (83%). The quickest way to find out how much of the disk access that a Virtual PC is using is to go to its properties, on the statistics page and developing the percentage based upon the amount read and written. As an example at 12 gb read, and 5 gb written, add the two together to 17 and divide to find the percentage. 12/17 which is approximately 71% read time. Perfmon counters can also be used as discussed above.

On Virtual Server disk writes on compressed drives averaged 22% less IO’s per second than on an uncompressed drive/disk reads on compressed drives averaged 52% more IO’s per second than on an uncompressed drive. So on a drive that averaged 50% read/50% write the increase in IO’s per second is an average of a 15%. On a system that averaged 70% read/30% write the increase is even more drastic at a 70%. On a system that is the reverse (30% read, 70% write) the increase is not significant (less than 1%).

My initial assessment of read/write usage on production Virtual Servers indicates that write usage is significantly higher than read usage. If read usage is not at least 30% of the total disk time on Virtual Server there will be no performance boost for compressing the system.

On Virtual PC disk writes on compressed drives averaged 57% more IO’s per second than on an uncompressed drive/disk reads on compressed drives averaged 83% more IO’s per second than on an uncompressed drive. So on a drive that averaged 50% read/50% write the increase in IO’s per second is an average of a 70%. On a system that averaged 70% read/30% write the increase is even more drastic at a 75%. On a system that is the reverse (30% read, 70% write) the increase is still significant at 65%.

Virtual PC shows a performance boost for both read and write. As long as there is available processing power on the host it appears logical to compress the virtuals.

VS Testing Environment: Virtual Server tests were done on Virtual Server 2005 R2 on a 32-bit platform running Windows 2003 SP1 for the Host operating system and Windows 2003 SP1 for the Guest operating system. The Host has a four drive SCSI RAID5 configuration running on a quad processor Xeon with six gigs of memory. The Guest operating system was configured with 128 mb of memory and was only running the host operating system. The Guest operating system used an ATA drive (not SCSI).

VPC Testing Environment: Virtual PC tests were done on Virtual PC with SP1 32-bit platform running Windows XP SP2 for the Host operating system and Windows 2003 SP1 for the Guest operating system. The Host has a single drive connected via Firewire configuration running on a single processor Pentium 4 (1.7) with two gigs of memory. The Guest operating system was configured with 512 mb of memory and running SCOM2007 Beta 2 to simulate active applications.

Parting Thoughts: If performance is an issue in your Virtual Server/Virtual PC environment (or if you are running out of disk space on your host system) do some tests to see how your Virtuals run compressed/you may be surprised!