While most omnibus services include their own demographic questions, some also include attitudinal and behavioral questions. Although all these free questions are positioned as a benefit to the client (and they are very useful), companies include them mostly because it makes good economic sense. Most clients usually will have some interest in analyzing their own data within a demographic framework, so it’s often simpler and less time-consuming for an omnibus service to have those questions pre-programmed into all their outgoing surveys.  Furthermore, most omnibus surveys use some kind algorithm to weight the final data, which in turn requires that they include special weighting questions. Vendors will often share the results of their demographic questions and most if not all of their behavioral and attitudinal questions. Since omnibus surveys charge by the question, you might want to consider asking which questions are included in the basic service as the research could end up costing you less than you expect and you could end up with far greater value than you realize.

Another item worth exploring is whether the vendor will allow you to tweak the response options to their standard questions. The way you or your company typically analyze data may not coincide with the way a response set is laid out for a given question. Since it’s impractical for an omnibus service to ask the same question in two different ways within the same survey, they might be willing to modify the question so it meets your needs while still allowing the company to provide its standard breaks to everyone else. For example, range breaks in a question about income might be altered from the customary $100K+ break to one that includes $100K – $120K and $120K+.   

Today’s post focuses only on elements that pertain to research sample.  

The data in any research study is only as good as the respondents who are willing to participate in the research. As the saying goes “garbage in, garbage out”. Making sure the sample comes from a reliable source is essential. Ask your vendor to explain where their sample comes from and what kind of quality controls they have in place to ensure the integrity of the output. If a vendor can’t provide reasonably good answers about the quality process in a timely fashion, you might want to look elsewhere.   

The next step is to determine how much sample you actually need to meet your objectives, keeping in mind future needs as well as current ones.  Shifting from vendor to vendor can be time consuming. If you plan to use omnibus research on a semi-regular basis, find an omnibus company that can anticipate your needs and that you believe will consistently meet or exceed your expectations.

You can find online omnibus companies that offer standard samples ranging from 500 completed interviews to 5,000, though most offer samples sizes in the 1,000 to 2,000 range. Choosing the vendor with the right sample size can be tricky as the size of the sample you’ll need almost always will depend upon whether you can get the right number of completes on the back end.  Irrespective of sample size, you’ll need to determine if  an online omnibus service is right (or any kind of omnibus service for that matter), or would you be better served with a targeted custom survey. A responsible vendor will likely help you work through the numbers. To illustrate the point, imagine you want to learn more about the behavior of widget buyers. Let’s say that widget buyers represent 10% of the population and you want at least 200 widget buyers. If the standard sample consists of 1,000 general population respondents, you might reasonably expect to obtain about 100 respondents who buy widgets. Since 100 is only about half of what you need, starting with only 1,000 respondents poses a problem. On the other hand, a sample company that starts with 2,000 respondents should be able to deliver what you need. 

Another important thing to consider regarding sample size is how you intend to analyze the data. If you plan to analyze data across multiple segments, the sample will have to be able to support those analyses.  Suppose that widgets have a 50% incidence rate, but you want to compare attitudes and usage between loyal widget buyers and buyers who switch between widget brands. Let’s also say that loyal buyers constitute about 30% of the widget buyer base while switchers constitute the remaining 70% and you need at least 200 respondents per buyer group. If we start with a sample of 1,000, multiply it by 50% for the incidence rate and then another 30% for the loyalty classification, we wind up with only 150 loyal widget buyers, short of the 200 needed.

So as we see from the widget examples, using a company that offers a starting sample of 1,000 respondents won’t meet the project’s objectives. That being said, you may still have options and should find out whether your omnibus company can:

• Field the study twice and have the data aggregated in a timely manner
• Increase the number of completes it typically offers

 As with any business decision, the choice of whom to use comes down to what is doable and cost-effective. If you continually get more sample than your analyses require, you could be paying for something you don’t need. The only way to know for sure is to check the price differences against competing services and weigh it against the other benefits you might get. As a general rule, it is wise to start with an online omnibus company that has the right basic sample “horsepower” for the majority of your omnibus needs, but also has the capability to reach back and get you a little something extra from time to time.

While size does matter, it’s not all that matters. Find out what your supplier can do with their sample. Can they split it into multiple cells? Being able to randomly assign respondents to different cells is critical if you plan to use an omnibus to test advertising or product concepts. Since sample size in an omnibus survey is usually fixed, the number of concepts that will get tested affects the sample size for any given concept. In other words, the more concepts there are, the smaller the base sizes will be per concept; so identifying what kind of sample options your supplier can offer will help you determine which provider is right for your business.   

Research that misses the mark on your objectives can result in missed business opportunities, and research that is inaccurate can lead to destructive business decisions and misdirected resources. Another requirement is a level of speed and cost-effectiveness that doesn’t compromise quality and reliability when research objectives are being converted into actionable information. This is especially important with online omnibus research, which can generally be done with less time and cost than custom research projects.

Effective survey research design and execution requires specialized skill and knowledge of a number of processes. An experienced survey consultant should have full command of each of the following and be able to confidently guide you through the particulars:

• Questionnaire design: Writing an effective questionnaire calls for the ability to translate business issues into a succinct set of questions that lets respondents cleanly and comfortably answer without creating bias. It matters that the survey has a logical flow, that question text and instructions are clear, and that the set of responses in each question align with the question text and provide all reasonable possibilities for response.

• Sample design: It’s important to understand where the sample for a survey comes from, that a sufficient amount is drawn, and that there’s enough representation in key groups to ensure that the required number and balance of completed interviews will result. It’s also important to be able to act when conditions aren’t optimal.

• Field: Key components of survey fielding that require special tools and expertise include: being able to verify quickly that a survey is going into the field correctly, with no content or programming errors; monitoring progress in field to be sure that completes are coming in sufficient numbers and balance in order to conclude on time, and understanding how to make adjustments on the fly if needed while the survey is in field.

• Deliverable design: Being able to organize the results into useful form is central to successful research. A knowledgeable survey consultant can design data deliverables that efficiently address the objectives of the research, and make sure all data are properly weighted, present and in place. Data tables should have correct analysis points, netted information and rank-ordering so the client can make the most effective use of the results. Data files should include all needed variables – including demographics and weighting – in a well-organized fashion. Analyses should be complete and concise, and provide insights that enhance the client’s understanding of their research objectives.

The series kicks off with five important checkpoints that you can use to help determine whether an online omnibus is right for your current research project. This straightforward review process comes naturally at the outset of every study, and you should feel comfortable that the survey company you’re working with is able to satisfy your concerns. As we discuss the checkpoints we’ll underline what an online omnibus can do so that you can better assess fit with your objectives.

#1 – Budget – Affordability is always a key concern when deciding whether and how to pursue a research project. Research should be undertaken to inform and support your business; as costs rise, the net value of the research diminishes. An online omnibus offers a strong way to help control costs while still providing valuable, robust and relevant results that speak to your research objectives.

• If you need to ask only a few research questions (or even just one), you can do that through an online omnibus without having to pay for a full custom research project, and gain a full set of demographic analysis points to help place those results in context.

• If you have a longer questionnaire (some clients have asked us to include up to 30 questions in a single field), you can still realize significant savings by using an online omnibus. Since they often don’t have to support a full custom research staff, online omnibus providers have lower overhead costs, and their base pricing should reflect that.

• Further cost savings may be realized when arrangements are made with the provider for ongoing work. Subscription discounts are usually progressive as well – the greater the commitment, the greater the savings.

#2 – Schedule – In research as in business overall, timing can mean everything. You need to receive research results in plenty of time for you to organize your resources to take best advantage of opportunities in the marketplace. When speed is of the essence, an online omnibus survey can harvest complete and reliable research results within just a few days after the survey has launched. Even when fast turnaround is not a necessity, the speedy return of quality research product from a reputable online omnibus service can allow you to act quickly and confidently on the results.

#3 – Design Support – Working with an experienced survey consultant when designing research is crucial. You will gain expert support in designing all aspects of your research project from someone with full knowledge of research practice who understands intimately how to make best use of the best tools, and will listen to you intently and ask the right questions. A quality online omnibus provider will offer strong consultative support at every stage of your project. We’ll talk more in the future about the role of the omnibus survey consultant.

#4 – Confidence – When you conduct research, you need to be able to trust completely that your provider, and the design and quality of the research results can all be relied upon. The sample needs to be robust and balanced, the questionnaire clean, clear and to the point, the deliverables complete and on-point, and deadlines will be reasonably set, then met. A well-run online omnibus service is focused on its craft, and will have its people, processes and tools aligned to provide you with an experience and a set of results in which you can place full confidence.

#5 – Repeatability – There’s a certain comfort in nurturing a productive and fruitful business relationship. Uniquely satisfying experiences that can be repeated often form the basis of your own ongoing success. In research, this can apply to a tracking study that needs to be executed in exactly the same way each time it goes out. It can also be about knowing you can go back to a company you’ve partnered well with before and expect a similarly positive result. Working with an online omnibus service that’s been set up and run by people who know what they’re doing, who remember you and value your business, can mean a smoother and richer research experience every time.