Surveys are an effective way of gathering information from your target audience to help you drive product development and marketing decisions, and make decisions about necessary changes in your product or service offering.
Surveys, though, are only as good as the quality of the design at the outset. The tips below will help guide you through the survey design process, to ensure that you are are gathering the kind of feedback that leads to actionable insights that is the most impactful for your business.
What’s the goal of the survey?
Are you interested in gathering feedback on a specific product feature or functionality? Do you want to increase your understanding of market trends? Or maybe find out how you stack up against your competitors? Regardless of your goal, know exactly what you want to find out so you’re asking the right questions to get the feedback you need. You will also have to be selective about who you’re sending the survey to in order to get the feedback you need.
What are different types of surveys?
Surveys come in different formats and choosing the right one will depend on your goals.
Do you want to get quantitative feedback from a large number of people? Do you want more detailed information from a smaller segment of the market? What you need to know will determine which of these four most common surveys would be most effective.
Pen and paper surveys are surveys sent via mail. This type of survey gives participants time to think through their responses and utilize any resources that they have in their home (if needed). It also is available for those in areas without reliable internet service. However, lists with addresses can be more difficult to find and the cost associated with mail can add to the overall cost.
Online surveys are very popular. They’re the least expensive way to get feedback from your target market and response time is quicker than with other types of data collection. They also allow for a larger sample size, which makes the results more statistically significant, but do not allow for the follow-up questions that are possible with in-depth interviews. There are a number of survey collection tools available (e.g. Qualtrics and SurveyMonkey) that allow you to quickly and easily create different question types. Online surveys can be posted on social media channels or on your website, or be embedded in blog posts. A very common way to distribute surveys is via email. In a future post we will share how to write invitations that get opened and create surveys that receive a higher response rate.
In-person interviews have the benefit of allowing you to build trust and rapport with the participant. It’s also easier to pick up on non-verbal cues, ask follow-up questions, and get clarification when needed. Due to the time individual interviews take and the logistics of getting the participants and survey conductor together, this type of data collection is typically more costly than the other approaches.
Over the phone (or web conferencing) interviews allow for some of the benefits of in-person interviews (e.g. being able to ask probing, clarifying questions and, in the case of web conferencing, to read facial expressions). Like in-person interviews, they take more time to conduct and thus cost more than online surveys, but based on the type of information you are looking to gather, they can be extremely effective in gathering information.
What kinds of questions do you ask in a survey?
While online survey tools offer many options for question construction (and we’ll devote another post to the most common types), at the highest level questions can be either open-ended or close-ended. Open-ended questions cannot be answered with a yes/no or a static response. For example:
- “What do you like most about this product?”
- “What changes would you suggest to our service?”
Close-ended questions include a yes/no response or a multiple-choice selection. An example is below:
A mix of open-ended and closed questions will allow you to quickly get the data points to easily see trends while also gathering detailed feedback in the voice of your market. A great way to combine open and close-ended questions is by utilizing a multiple-choice question with the option of text-entry. It would look like this:
Regardless of whether you’re conducting face-to-face interviews or collecting online responses, it is typically best to include both open and closed questions in your survey in order to achieve your survey goals.
How long should a survey be?
An online or pen-to-paper survey should include no more than 30 questions and take around 8 minutes (or less) to complete. Here’s a great article that references research that was conducted regarding survey length. Again, remember your goal! We find that companies sometimes see surveys as an opportunity to throw in all kinds of questions they might have (about just about everything). Please don’t do this. Survey participants will naturally hit a fatigue point, so focus on your overall goal and what you need to know. Consider whether you may need an additional survey or surveys if you have questions outside of the scope of the one you’re writing.
What kinds of questions can ruin the data?
So, you have a mix of open-ended and closed questions and you’re keeping the length in check. How you write the questions will have a big impact on the accuracy of the data you’re collecting. You need to ensure you’re leaving out any bias by not asking leading questions, not using absolutes, and not asking more than one question within a question. Examples of each are below.
Showing bias/leading question: Leading questions include language that may sway a respondent to answer in a particular way and may highlight the bias of the person making the survey. For example:
- How great of a job do you think our current CEO is doing?
- Change to: How would you rate our current CEO’s performance?
Absolutes: Absolutes in surveys should be avoided. Absolutes include language like “always, never, every, none.”
- Example: Do you always choose (brand)?
- Change to: How frequently do you choose (brand)?
Loaded questions: Sometimes in an effort to limit the number of questions in a survey, clients will want to combine multiple questions into one question. Don’t do this. Each question should include just one question or your data will be off.
- Example: How satisfied are you with the design and the price of this product?
- Change to:
- Question 1: How satisfied are you with the design of this product?
- Question 2: How satisfied are you with the price of this product?
If a participant selected “Very satisfied” to the first question above, you would not know if they were answering for the design, the price, or both.
By designing your survey with the overall goal in mind, keeping length in check, and removing language in questions that can skew the results, your data will be more accurate and you’ll be able to make better business decisions based on the feedback you’ve collected. So, start with the end in mind and be thoughtful about each and every question you ask.
Feel free to ask me questions in the comments below and I would be happy to answer them. Follow me on LinkedIn to know more about my work.