Beta Testers

Don Jones · #1 11-24-2008, 09:25 PM

I am curious how some of you guys managed to be beta testers for some of these cool toys? I know having a flying airplane would be a major requirement, but I wondered what the process was like?
Inquiring minds want to know, if it's not top secret

DeltaRomeo · #2 11-25-2008, 04:54 AM

Morning, Don. I wish I could say it was based on my superior intelligence, but in the past they simply contacted me...not the other way around.

If I had to pick two attributes I would imagine they're web visibility and forum content.

b,
dr

Don Jones · #3 11-25-2008, 06:32 AM

Quote:

Originally Posted by DeltaRomeo

Morning, Don. I wish I could say it was based on my superior intelligence, but in the past they simply contacted me...not the other way around.

If I had to pick two attributes I would imagine they're web visibility and forum content.

b,
dr

Ahh, so you don't have to know the secret handshake

SteinAir · #4 11-25-2008, 11:50 PM

It's often a bit more involved than you may think. I participate in some beta testing of goodies, and pass on others. It's not like the new web browsers or latest Microflop software where you download it and sign up to become a "beta tester". In regards to a lot of our stuff that we play with, it can be quite involved. Not only do you have to test the product, but sometimes you need to be a guinea pig as well. Often you'll be required to do a specific set of tests, record LOTS of data points, and do it repeatedly.

For me, I rarely have the time it takes any more to do a good job participating in true beta testing of things. It's usually so much more than just having a gadget and playing with it. It takes a lot of time and diligence, and takes away from just pure fun flyin! Then, if it's a serious test with a big company, lawyers are involved, paperwork is involved and you have to keep secrets! In this business, it also usually requires that you either have a special background skill set relative to the item, or have a decent amount of time with similar things. This isn't always the case, but many times is. In the later rounds of beta test, mfgrs will use a scattered base of more general level testers to try and mimic a sampling of typical users/customers.

Anyway, in the end this can be one of those things where you can end up crying uncle...be careful what you wish for sometimes. That being said, it is really fun to play with some of the new goodies that are out there being developed. The next year promises to be a heck of a year for "gadgetiers"!

My 2 cents as usual.

Cheers,
Stein

Don Jones · #5 11-26-2008, 06:29 AM

Thanks for the detail of what is involved. I had imagined it wouldn't be all play and that some real work was involved, but working and flying is not a bad way to spend some time. Beats my day job

kentb · #6 11-26-2008, 09:44 AM

Quote:

Originally Posted by SteinAir

The next year promises to be a heck of a year for "gadgetiers"!

My 2 cents as usual.

Cheers,
Stein

There goes some more of my retirement money.

Kent

KPmarc · #7 11-26-2008, 10:12 AM

Beta testing isn't all fun. It can be, but it's also tedious and repetitive, and stuff happens over and over again.

I think what most people don't understand is that it takes a lot of discipline to get the data points, be confident in your findings and to appreciate the responsibility to not take the engineering team down a dead end. Some of the discussions are lively, to say the least.

In the case of testing the Dynon autopilot, for example, a good portion had to do with flying qualities and testing the features, but a substantial part of the beta-tester's back-and-forth centers on human factors. That's why a smart company like Dynon (and others) actually has pilots fly the product before release; and has more than one pilot do the honors. Among us you'll find pilots of all skill levels, ergonomic preferences, and tolerance for quirks. The truth of how the product works will be in there, even if it's sometimes a challenge for the engineers to pick it out of the noise.

Plus, you have to tamp down the occasional fear that "these guys just aren't going to get it" that sometimes turns into, "I'm ripping this thing out of the panel and sending it back with hammer marks." Particularly galling if you've been flying with a competing product that is, in the early stages at least, much better than what you're testing.

The reward is when the product hits the streets and you read someone's comment, "Man, that feature really works well," and you had some hand in making it work that well. Pretty cool.

Marc

N941WR · #8 11-26-2008, 10:41 AM

In addition to everything Marc said, you need to have a high risk tolerance.

Besides spending a lot of time in a hot or cold hangar, cutting up your airplane, its electrical system, etc. to install the test articles, things can and do go wrong in testing. Are you willing to take that risk?

You might also be asked to replace the installed test articles multiple times as changes to the hardware and software are made. Software upgrades are easy enough but sometimes replacing the hardware is tedious and doing it multiple times can be a real pain.

You also have to realize that at some point your airplane may just not be airworthy due to either mechanical or software problems associated with the product you are testing. Are you willing to give up that freedom to help some company?

I discovered a lot about myself and my risk tolerance while testing Dynon's autopilot. It was all enjoyable and I would be happy to help them out with any future products but I also fully understand the risks to me, the plane, my engine, etc. whenever I test a new product, as does my wife.

One thing that made working with Dynon so enjoyable was the people who work there. They are intelligent, eager, and motivated to bring out the best possible product they can a price that will make them a market leader. They were quick to grasp issues and concerns, make product improvements (both hardware and software), were very professional, and obviously enjoyed what they were doing.

Yes, I am very grateful for the opportunity to help them and would be more than willing to help them again. (BTW, the beta testers continue to test new software versions, even as Dynon releases their autopilot for sale. Continued improvement is what Dynon's all about!)

This is one of those, be careful what you ask for, you might just get it.

KPmarc · #9 11-26-2008, 12:34 PM

Bill made me thing of one more thing: During all the software development, it was only prudent to revert back to a working, standard build before doing any IFR flying. I do have backup instruments, but you have to be thinking about the status of your software before launching into IMC.

My serial/USB cord got quite a workout changing software back and forth.