Business Solutions Best Channel Vendors 2012 Survey Methodology & FAQ
How were the Best Channel Vendors survey and special report generated? We partnered for a fourth consecutive year with Penn State University to help ensure the survey's statistical accuracy. Penn State agreed with our philosophy of conducting a Web-based survey (as opposed to a phone-solicitation survey) of our subscribers to capture significant data from our most active resellers. The technology categories were determined by the Business Solutions magazine (BSM) editorial staff.
Who participated in the survey? How did they vote? From September 2011 to mid-October 2011, BSM VAR subscribers were asked to participate in the Best Channel Vendors survey. We asked subscribers to rate their vendor partners on a scale of 0 (worst) to 5 (best) in seven categories. We also provided some guiding questions to ensure consistency in how VARs interpreted each category:
- Service/Support: Do you have positive working relationships with their staff? Does their staff respond to your requests adequately? Fix your product issues quickly? Are they easy to do business with?
- Channel Friendly: Are they loyal to you and their other channel partners? Do they avoid competing with you, not unfairly selling direct? Do they push business toward you, instead of quoting prices to end users?
- Channel Program: Are you satisfied with the variety and quality of services they offer (e.g. reduced shipping, marketing funds, VAR rebates, lead generation, etc.)? Do they effectively and consistently communicate program changes?
- Product Features: Do their products meet your functionality needs? Is their product breadth adequate?
- Product Reliability: Do their products meet your quality needs? Do you frequently need to fix their products?
- Product Innovation: Do they update and adapt their products appropriately? Do they offer you new products to enhance your future growth opportunities?
- Adequate VAR Margins: Are their products priced fairly so you can make an adequate profit reselling and servicing them?
How many VARs participated? Did you throw away any votes? 4,300 people participated in this year's survey; we eliminated ballots that did not qualify as resellers. 10,833 votes were cast in the survey (slightly fewer than last year due to fewer companies in this year's survey).
How were the Best Channel Vendors determined? An overall average score for each vendor was determined by adding the cumulative scores of the seven categories and dividing that score by the number of VAR subscribers who rated that product. From that list, our editorial staff chose the Best Channel Vendors by applying a "Scree Test." We won't go into the full definition here — we'll leave that up to the Penn State statisticians to explain — but basically our editors applied reasonableness to the results and drew the line where there was a clear break between the highest scorers and the "rest of the pack." Our goal was to reflect the sense of the survey respondents and let our readership know who they felt were the Best Channel Vendors. Those familiar with our past Best Channel Vendors surveys might remember that our previous results included the top 15% of vendors in each technology category. After careful consideration of feedback we received on last year's survey, we tightened the grading criteria and chose the top 5% of channel vendors in each category for 2011.
How are the Best Channel Vendors listed? They are listed first by technology category (Bar Code Labeling through VoIP) and then by technology subcategory. Best Channel Vendors are listed in alphabetical order. We did not rank them according to their overall average score. One of the main purposes of this special report is to give our resellers a guide to who their peers think are Best Channel Vendors. Ranking vendors according to their overall average score would mislead readers and defeat that purpose.
Did you "fudge" any scores to make sure your advertisers were included on the list? Did any vendors buy their way onto the list? No. Other magazines have skewed their surveys to include current advertisers or entice new advertisers. We believe engaging in that behavior is not only a disservice to the industry, but it's dishonest. We would not compromise the reputation our magazine has built over the past 25 years for the extra revenue a few ads would generate.
Okay, so the survey is accurate. But is it perfect? Certainly not. Penn State's statistician stressed to us that all surveys have their limitations. So, this survey is not perfect, but it should be an excellent tool for the channel because it indicates products and vendors that have highly satisfied VARs.
What should I do if I have a comment about this special report? Please call BSM Publisher Sue Bresee at (814) 897-9000 x219 or or email her at email@example.com.