Parallels vs Fusion - some Benchmarking

{edit:

as suggested the benchmark posting is not allowed - so I have removed it .. however I cannot see the issue with posting an informed opinion .. sooo:}

I found a nifty way to transfer a Parallels XP (SP2) Image to VMware's Fusion (Beta 3). use VMW's freeware Converter product (use the "live" image generation from within your running Parallels image).

In order to do the test I suggest you proceed to install SI Sandra Soft's benchmarking tool in each virtual machine and run the basic benchmarking. You should disable the debug code in Fusion. Seeing I can't post benchmarks I suggest you go and do your own tests. My opinion is below:

Parallels seemed the faster of the two, especially on the file intensive work, Fusion downloaded files a bit faster.

The whole benchmark thing might be a bit moot however if you're after a specific product for a specific reason (perhaps you love coherence or Fusions Direct X support). For the purposes of this test I had both virtual machine configured to be a close to each other as possible (so no 2nd cpu truend on is fusion for example, and obviously no Direct X tests).

 

Just to warn you... As is becoming popular with many software licenses lately, the license for VMWare Fusion specifically forbids the posting of benchmarks anywhere except their own forums...

 

I really doubt that such a restriction is legally enforceable.

 

I used passmark and found that after it was adjusted from default settings, Fusion benchmarked faster, except in 2D rendering, which may explain the feel of it being slower... I also used the exact same VM that I converted. I had to remove Parallels Tools and disable the Coherence service, which seemed to improve responsiveness in VMware. That, coupled with some tweaks on the VMware forums, netted a non-trivial speed advantage overall for VMware, at least in the benchmark. Close enough in usage to make the choice not as relevant to speed.

CD burning from XP also works, whereas Parallels was pretending to work and creating coasters for me - not that this was a high priority - but annoying.

Nice to have a real Apple look & feel. Parallels looks like a linux app, I think they are using QT or something for their GUI. VMware's is cleaner.

Vmware is also really being responsive in it's forums. Which is nice. They actually respond with technical information, and are pretty level headed about debating their product. It is refreshing.

In contrast to some guy like Ben writing FAQs no one cares about:

Little Johnny: Can I use parallels to run notepad in Windows XP?
Ben: Why yes little johnny, and here is how...

The hype-to-engineering ratio is way lower in the Fusion camp.

 

I am curious how one would enforce a EULA on a beta program. Do they send a swat team to your house to shoot you? By hitting a "yes" button, and using their software, this forms a legally binding agreement allowing them to sue you for untold monies?

I don't know, things seem to have gotten pretty silly with the EULAs. If they stand up in court, it must be kangaroo.

Forum Post EULA:
By reading this post you to agree to sign over all your wealth, and the wealth of all subsequent generations to Hoju Inc.

 

Can you kindly point me to the instruction on how to do the 'transfer'? I've found one on VMware forum and it has a large number of steps.

 

Unfortunately, in the U.S., EULAs have stood up in court. Any EULA that requires active confirmation has stood up. Basically, if they display a page that shows the EULA with "Accept" and "Decline" buttons, clicking "Accept" marks a legally valid contract.

Just because it's beta software doesn't mean it's not a contract. By your reasoning, I should be able to share the code to Windows Vista that I was using in beta. Or any other program I have used in beta. Sorry, 'beta' doesn't mean 'not licensed'. Beta programs, if anything, are *MORE* restrictively licensed, specifically to prevent beta programs from being abused. (Look at Apple's beta programs. Not only do you have to agree to a restrictive license, you also have to agree to a non-disclosure agreement.)

Sorry, but to be enforceable, it requires active confirmation. I decline your EULA. Now, if your EULA said that sending you an email or PM constituted acceptance, then someone would have to take action to confirm. Sending you that email or PM would be that confirmation. (Although this type of 'open' acceptance HAS been tested and found not binding in U.S. courts.)

For example, when you turn a new Dell computer on for the first time, the very first screen that comes up is a very short "By continuing to use this computer, you agree to the enclosed EULA." screen, with "Press spacebar to accept, turn the computer off to decline." The only problem is that the EULA is not enclosed on paper. The only way to read the EULA is to accept it, then read it later in the install process! This most likely would not hold up in court in the U.S., because you aren't given any opportunity to see what you are agreeing to. (The EULA is also for Windows, which, by accepting, you cannot return; yet Dell customers have been able to decline the Windows EULA and get refunds for the cost of Windows even after accepting this initial EULA. Probably because Dell knows this one isn't enforceable.)

Outside the U.S., though, some countries have found so-called "click-wrap" agreements to be invalid. i.e. It takes more than just clicking "I Accept" to form a legally enforceable contract.

 

Of course your points are valid. I was merely being cheeky.

There are a lot of stupid things that go on in the US courts. Patents come to mind.

But practically, I think if VMware starts taking legal action against everyday people who post benchmarks against a EULA, they a) have too much time on their hands and b) are employing to many lawyers.

More likely they would go after Parallels, as they are being posted on their forum. And if Parallels was smart, THEY would be the ones who were worried about such posts.

 

Guys, this forum is about Paralles -- not VMWare.

If I want to learn more about VMWare, I'll go to their site and/or forum.

 

So... what's the conclusion? Is Fusion slower even with debugging mode off or what?

 

Lets see..... Fusion specifically and intentionally codifies their objection to SHARING any benchmark of their product ???

Doesnt that just SAY everything about the results you'd need to know?

I mean, if I had a superior product to my competitior... I'd demand that you HAD to post the results of the benchmark!

Hell !!

There are whole sites dedicated to benchmarking software and hardware....

These TARDS want to send you to JAIL or SUE you for disclosing how well... or likely PISSPOOR they are doing ??

And to think I was gonna give them a try....

Not any more folks....

Bad enough that we need this program to still use M$... Im not about to help any other software tyranny!

Thanks for the post mate.. ya saved me the trouble...

PAX !

 

The reasonable conclusion is that, essentially and in most ways, Fusion is actually slower or -- one would assume -- they would want the results posted.

As I said before, I really don't want to hear about VMWare, etc. If I did, I would go to their site and forum(s).

 

You can conclude nothing because you don' t know if all debugging is off in Fusion, and if optimization is enabled in the compilation. There is not even a reasonable test defined that can measure existing environments. Further - Fusion has support for two processors and that needs to be considered. There are priorities in each camp: do you take it easy on the host system, or enhance Windows support at the expense of OS X? That isn't being tested. A full suite test will tell you what OS X and Parallels/Fusion VM's are doing, what the hardware is doing, and how many cpu clicks are fried chasing loose ends.

Functional tests: Drag and drop. Performance test: How long does that drag an drop take? Is this a Boot Camp or non-Boot Camp environment? Which Boot Camp version? Airport or CAT5? Shared networking? DNS? DHCP? What OS updates are installed? What firmware updates are in place? What services are running in each OS? What is the memory allocation? How is swap configured? Usabilty testing: Is coherence a factor? Is a virtual desktop a factor? How much free disk space is there? How much RAM is available? What is the disk speed? None of this is being considered. This entire performance discussion is premature, without value, subject to ravings along the lines of Chevy/Ford debates, pointless at this stage of development, and moronic.

 

As for the 'debugging mode' being off or not, I'm going to trust that the option to turn it off in beta 3 of Fusion really means it'll turn it off completely. Otherwise it'll be pointless.

That said, my brief brush with Fusion beta 3 led me to feel that the 2D graphics update rate is, in fact, slower than Parallels build 3188. I can definitely notice the flickers where Parallels has nearly none. This will mean that, even though Fusion may exhibit better performance in terms of stuff like I/O, the perceived performance will favour Parallels. This should be taken account of when testing. This kind of things are not necessarily easy to quantify.

 

you could post in a blog and link to it

you would be very much in breach of any beta test agreement you have made
those are normally confidential

at this stage the mature product will be better than a wet behind the ears beta

and thus you are not comparing like with like
which invalidates the test statistically speaking


Hugh W

 

For those of you critical of VMWare, or think they have something to hide, I wonder any of you actually read the Parallels Beta Testing Agreement, which states, under Restrictions on Grant:
"Except as otherwise specifically permitted in this Agreement, you may not: ... (h) publish any results of benchmark tests run on any Software to a third party without Parallels prior written consent;..."

This is similar to the Fusion Beta Test Agreement, which groups performance data under Confidential Information, and prevents disclosing "any Confidential Information to any third party."

In other words, standard beta software protection terms. VMWare places restrictions, so does Parallels. There is no need to draw wild conclusions here.

 

Science does not allow for what you can trust.

It's ok to say "I don't know what to conclude from the evidence because I haven't any" when it is true that you don't know what to conclude from the evidence you don't have. It's also ok to say "It seems to me that it appears to be improved..." when you lack supporting evidence. The rest of us might roll our eyes and wonder why you bothered, but that's just us.

Data is good. Share it if you have it.

 

You make being the next Ann Coulter look easy. I might give it a go myself. Everyone loves a good bout of hyperbole. Meanwhile, all vendors are within their rights to manage the disclosure of beta level code performance disclosures via on-line and other contracts. There are all manner of reasons for doing so that include SEC issues, controlling customer expectations, context of tests/performance control. For examples, examine the Apple IPhone project. You know for a fact there are beta/RC tests being run by folks out there and yet there's no data. Care to guess why, Madam Barrister?

 

Right - it's not all that complicated BUT I'll be brief because this *is* the Parallels Forum and well I don't think that the procedure of switching off parallels is a fair use of their forums (it might however be a potential way of creating a parallels image off a live NT4 machine via the additional step of running transporter against the vmware image ? - I haven't tested this...Transporter doesn't support NT4)

Step 1. Set up a shared folder and test it's working (I'll leave the details of this off as it has been discussed a million times on the forum.

Step 2. Remove Parallels tools

Step 3. Install vmware converter (link in original post at the begining) and use the live image creation option to create an image of your(save the image to the shared location)

Step 4. Uninstall vmware-converter and reinstall Parallels Tools

Step 5. Use Parallels =). (had to be put here)

Voila.

As for the Benchmark talk. Yep EULA's are legally being tested but that's probably not the point - the issue is that, perhaps rightly, the company could feel hard done by if a magazine published a performance report on a pre-release piece of software (especially /w debug code in place). I did the benchmark purely for information purposes. In hindsight it was probably a waste of time given the differences (but I did tell you it was tenuous =).)

Having said that - there should be NO objection to doing a bazillion benchmarks when the software has hit release point (and I would imagine it'd be hard to enforce at that point due to anti-competitiveness of the whole thing).

 

That kind of saying will get nowhere. We don't know if Parallels, or any other software, for that matter, has some sort of debug code cramped inside the current release. By your logic, such non disclosed ambiguity would invalidate any comparative testing. That's overly paranoid. We just need to test what is presented upon us as public builds. Whether or not 'debugging code' is really completely off or not is irrelevant as the VMware's build will still be quite different from the future release build.

Therefore the test results will obviously be inconclusive as far as what VMware's final code will be at, but it will be conclusive about where the code is right now, as far as VMware team wants to show to us. If VMware teams doesn't want to pull all the plugs when the user wants to run under debugging disabled mode, then so be it. We'll measure the performance under that condition and be at that.

I was not trying to conclude anything from the subjective experience I was presenting, actually. I was merely stressing the importance of having solid data. The lack of any evidence or conclusion within that paragraph I wrote is by design. You're free to roll your eyes at me. Heehee.

I may explore into having some sort of measurable metric developed for testing the 2D performance, be it using some old graphics benchmarks for some macro looping in the future, though. The flicker gets annoying sometimes. Not right now, though.