Why Is The Number Of Panelists Not The Best Criteria For Judging An Access Panel?

Research access panels are often the perfect way to gather opinions from hundreds or thousands of people from across the country. People have already consented to be contacted and share their opinions about a variety of topics, and they feel comfortable sharing their most personal experiences because they know and trust the company they’re helping. But a common concern often comes up when researchers are deciding which panel would best suit their needs.

Researchers might not seriously consider working with a particular panel because they believe it is too small. When some panels advertise their size in the millions, it can be hard to seriously consider a panel that advertises ‘only’ hundreds of thousands panelists. A decision to work with the larger panel seems obvious. Or is it?

To illustrate why panel size is often not relevant, let’s consider two hypothetical companies. MassivePanel says they have 1 million panelists. They launched a questionnaire to everyone on their panel and got 50 000 completes. This equates to a response rate of 5%. On the other hand, MaintainedPanel says they have 200 000 panelists. They launched the same questionnaire to everyone on their panel and also got 50 000 completes. This equates to a response rate of 25%.

Even though MassivePanel is 5 times as large as MaintainedPanel, it turns out that the effective sizes of the two panels are identical. There is no advantage to choosing the larger panel because the end result is the same. So what’s happening?

Panel rules are what’s happening. Most panels have rules about who they include when they define their panel size. MassivePanel might have decided that a panelist is ‘active’ if they clicked on a survey invitation in the past 12 months. A rule like that would result in a very large panel as many people would click on at least one email, even accidentally, over the course of a year. The survey completion rate would be low but the panel size would be large.

On the other hand, MaintainedPanel might have implemented a much stricter rule. They might have decided that an ‘active’ panelist is someone who has fully completed at least one survey in the last 3 months. A lot fewer people would meet this criterion but the panel would be full of people who are known to fully perform the research tasks asked of them. The survey completion rate of this panel would be high although the panel size would be small.

In other words, panel size is meaningless without an estimate of the research completion rate. The next time you’re considering which panel to work with, keep in mind that the starting panel size isn’t always the best indicator. The best indicator is whether you will receive the number of completed research studies you require.

You might like to read these:

Canadian Viewpoint is a one-stop market research data collection and fieldwork company. For over 40 years, we have been trusted by clients ranging from global Fortune 500 companies to local, boutique market, social, and academic research firms and offering top-quality solutions for offline, online, qualitative, and quantitative fieldwork. We specialize in providing high-quality solutions for offlineonline, qualitative, and quantitative fieldwork. As long-term members of the Insights Association, accredited members of the Canadian Research and Insights Council (CRIC), and corporate members of ESOMAR, we uphold the highest industry standards. Our diverse range of services includes sampleprogramming and hostingmall interceptscentral location recruitmentmystery shoppingin-home usage tests (IHUTS)sensory testingshelf testingcomputer-assisted telephone interviewing (CATI)Facial Coding, and other cutting-edge technologies. Explore our website to learn more about our offerings and access our demo site to experience our tools firsthand.

Follow us on LinkedInInstagram, & Twitter and sign up for our newsletter to stay updated with the latest industry insights and news.

This entry was posted in Market Research and tagged , , , , , . Bookmark the permalink.