Beta Testing--or not?

[betatester]

As administrator of a large network of Macs, I've been asked by my company to evaluate Parallels for possible deployment company-wide. I have been doing so for some months now, upgrading my own machine (MacPro 3 ghz, 6 GB RAM, 1 TB disk, standard video) with each subsequent beta release. I have also been following these forums closely, and have picked up an enormous amount of valuable information, both from team members and from enthusiastic and expert members.

But I find myself increasingly confused about beta testing procedures and methods, an issue that was recently raised by another member in response to a posting elsewhere in this forum. This confusion perhaps arises from the fact that I have been a formal beta tester for an array of products on different platforms for over 20 years.

In these forums, I see users reporting a variety of replicable bugs, some minor, and others very serious. Sometimes, there is a response from the development team indicating that the issue is being worked on, sometimes not. Sometimes, other users write to suggest work-arounds. Yet other times, the users are chided for complaining about a problem "because it is a beta product."

But though I have searched through all the forums, I find no method for specifically reporting bugs, or providing development team members with the information that they might require to evaluate them. There is a "report a problem" capability in the software itself; but it is unclear if this is in fact a bug reporting mechanism, a request for user level support, etc. Moreover, there is no database of bugs reported, nor the status of the work on exterminating them.

So I would request direct clarification, not from users, however experienced they may be, but from a team member, so I can understand what the beta test procedures are for Parallels, and whether I or other users are in fact participating in a beta test in the usual sense of the term. This would be helpful to me in making appropriate recommendations to my company, as well as understanding the current state of play on the product development cycle.

In particular, I would ask:

1) Is there debugging code in Parallels?
2) Are there error logs or cores generated, and if so, where are they?
3) What is the formal procedure for reporting a bug, and what information should a tester provide and how should he or she provide it?
4) Is there a database of reported bugs, so we don't send duplicate reports?
5) If in general users are not participating in a formal beta test, but are being offered the opportunity to use and discuss beta *versions," do you have a formal beta test group, and if so, how does one apply to participate?

Thank you in advance for your kind consideration.

 

[don montalvo]

Sometimes, other users write to suggest work-arounds. Yet other times, the users are chided for complaining about a problem "because it is a beta product."

the only people who are chided are folks who ignore the warnings and disclaimers before participating in beta programs - then hurl condescending remarks at parallels staff. totally uncalled for, counter productive and downright immature.

i have several machines i'm testing parallels on. two are mac pro towers - i have a spare partitioned boot drive i use for parallels testing (so the normal boot drive isn't hosed by the beta). i also have full backups on my windows partition on my macbook pro so i can reimage my windows partition in a snap if things go haywire.

i wouldn't roll out parallels yet - build 1970 is stable but pointless now that we're so close to being able to run off our bootcamp partition. everyone's mileage may vary though so the decision rests on the sysadmin.

don

 

[betatester]

the only people who are chided are folks who ignore the warnings and disclaimers before participating in beta programs - then hurl condescending remarks at parallels staff. totally uncalled for, counter productive and downright immature.

i have several machines i'm testing parallels on. two are mac pro towers - i have a spare partitioned boot drive i use for parallels testing (so the normal boot drive isn't hosed by the beta). i also have full backups on my windows partition on my macbook pro so i can reimage my windows partition in a snap if things go haywire.

i wouldn't roll out parallels yet - build 1970 is stable but pointless now that we're so close to being able to run off our bootcamp partition. everyone's mileage may vary though so the decision rests on the sysadmin.

don

Points all well taken, Don. I think we agree that the product isn't quite ready for prime time, but shows enormous potential. And the procedures you are following to protect the integrity of your system are exactly those that one would normally follow in a beta test mode. Yet they are quite out of the ordinary for "normal" users, so they can easily get themselves into trouble--as we observe here. This is why beta testing programs frequently qualify their participants, to be sure that the costs of supporting them (or sucking up their frustration) do not exceed the benefits they provide to the development process.

The point is that there is something of an information vacuum, which experienced users/beta testers like you are valiantly attempting to fill. Others fill it with their desire and imagination. For better or for worse, I am trying to do so by reference to normal software testing and development procedures--hence my questions.

The underlying issue is--what is the basic development trajectory of the product, and how is it being pursued? What are the priorities and objectives? What is the time frame? Obviously, there are very serious bugs in the code, bugs that trash users' partitions in the worst case, and impair functionality in many cases. Yet before those bugs are ironed out, new, complex features are added, like Coherence, which contribute their own set of problems, even while they fire the imagination.

All this is quite confusing even to an IT professional, and to judge from the remarks of users, to them as well--especially in the absence of clarification from the company. I recognize that there are proprietary issues, and that no one wants to promise more than they can deliver--quite the opposite. But a better general sense of how things are likely to go would be reassuring and helpful, both to us professional geeks, and to ordinary users.

Meanwhile, in the background, we hear rumblings from good old VMWare, with its solid financing, excellent reputation in the corporate and business markets, and an established family of robust, proven products. Companies like mine are relatively price insensitive; but we cannot afford high support costs or lots of down time. At the end of the day, it isn't necessarily the most technically sophisticated product that wins--witness Microsoft.

That said, I want to back the plucky new kid on the block. But I also want to get the sense that the kid will back me as well, by providing the information I (and I venture to say others) need to make good decisions.

Anyway, sorry for blathering on off topic. I continue to hope that the developers would see fit to answer my previous questions, which are posed with all the best intent for the community.

 

[veggiedude]

The very first item in the very first topic of this board has this line:

As usual, any feedback is extremely appreciated at beta@parallels.com mailbox.

So I guess that (at least partially) answers one of your questions.

 

[don montalvo]

Points all well taken, Don. I think we agree that the product isn't quite ready for prime time, but shows enormous potential. And the procedures you are following to protect the integrity of your system are exactly those that one would normally follow in a beta test mode. Yet they are quite out of the ordinary for "normal" users, so they can easily get themselves into trouble--as we observe here. This is why beta testing programs frequently qualify their participants, to be sure that the costs of supporting them (or sucking up their frustration) do not exceed the benefits they provide to the development process.

The point is that there is something of an information vacuum, which experienced users/beta testers like you are valiantly attempting to fill. Others fill it with their desire and imagination. For better or for worse, I am trying to do so by reference to normal software testing and development procedures--hence my questions.

The underlying issue is--what is the basic development trajectory of the product, and how is it being pursued? What are the priorities and objectives? What is the time frame? Obviously, there are very serious bugs in the code, bugs that trash users' partitions in the worst case, and impair functionality in many cases. Yet before those bugs are ironed out, new, complex features are added, like Coherence, which contribute their own set of problems, even while they fire the imagination.

All this is quite confusing even to an IT professional, and to judge from the remarks of users, to them as well--especially in the absence of clarification from the company. I recognize that there are proprietary issues, and that no one wants to promise more than they can deliver--quite the opposite. But a better general sense of how things are likely to go would be reassuring and helpful, both to us professional geeks, and to ordinary users.

Meanwhile, in the background, we hear rumblings from good old VMWare, with its solid financing, excellent reputation in the corporate and business markets, and an established family of robust, proven products. Companies like mine are relatively price insensitive; but we cannot afford high support costs or lots of down time. At the end of the day, it isn't necessarily the most technically sophisticated product that wins--witness Microsoft.

That said, I want to back the plucky new kid on the block. But I also want to get the sense that the kid will back me as well, by providing the information I (and I venture to say others) need to make good decisions.

Anyway, sorry for blathering on off topic. I continue to hope that the developers would see fit to answer my previous questions, which are posed with all the best intent for the community.

now that virtualization on mac is becoming the hot thing, the floodgates are opening and this forum is bursting at the seams. parallels probablly needs to beef up its staff. it seems the forum gets more developer input after final versions are released...and less during the early/mid beta stages. the irony is that the most valuable input comes from the developers - and they're least accessible when they're the busiest. i'm sure they're monitoring the forum though. when there's a clear pattern of people have an issue, they hop on it. i'm hoping they fix the issue i'm having with removing 'parallels tools for bootcamp'. for some reason i wasn' t able to uninstall it when i updated the beta. i'll give it a few days then i'll wipe/rebuild. i can boot into windows and beta1/beta2 work so i'm in no hurry.

i see vmware staying a step or two behind parallels. they aren't going to innovate - they'll just follow parallels' lead. that could be a good thing...parallels will attract those who love riding the cutting/bleeding edge. vmware will attract those who stay behind as a matter of best practice.

if you have a spare mac pro, i'd toss a drive into it and begin to test. make sure you ghost your windows partition before you begin testing so you can wipe/reimage and begin testing again if things go south. you'll need to work out the whole active directory thing. the pc folks at the shops we support had an easy time with adding virtual machines to the domain, using login/batch scripts (automounting shares, autosetup of printers, policies, etc.) and vpn was a snap. an interesting idea was one shop's idea to create virtual machines within active directory. i'm waiting to hear back from tolisgroup to see if bru server can backup bootcamp partition without having to have bootcamp running.

don