ONE DAY during his tenure as a professor, Albert Einstein was visited by a student. "The questions on this year's exam are the same as last year's!" the young man exclaimed. "Yes," Einstein answered, "but this year all the answers are different."
This anecdote captures an important truth - answers change frequently- but if we have the right questions, we will always have a guide to the insight we need. Questions and prompts are at the core of any survey - today we are going to focus on how we can make the most of these tools in the specific context of video surveys and questionnaires.
It’s an interesting challenge but very worthwhile since video makes it easier to be part of the participants’ world than any other medium. The suggestions in this blog are drawn from our experience and the abundant existing material out there on qualitative question design.
Ultimately, getting the most out of your videos is the goal. We want to capture the value in visual communication as well as spoken language; see them think, where they pause, facial expressions and body language. To start we’re going to discuss the value of incorporating tasks and immersive exercises into your question flow, then the best way to structure video surveys and finally the way questions themselves are constructed.
1. Get immersive where you can
Watching people speak or answer a question on video already provides a wealth of information that’s not available in text based media, but video can do so much more - particularly when it’s mobile. In addition to capturing tasks or exercises like product testing and retail experiences, video also lets you capture people’s behaviours, emotions and thoughts at the moment where they are experienced. This provides additional, potentially crucial contextual information about the environment in which people typically use a product or service
For better or worse almost everyone has a video camera in their pocket these days. You can leverage this to let your respondents really show you what they are talking about, in their own homes or out and about. As a result mobile video has some clear advantages over traditional ethnographic processes and adds real-world perspective to traditional in-person interviewing practices. It also delivers insights that are free of some of the biases inherent to focus groups and interviews like moderator, respondent, acceptance and recall bias.
See below for some of the great ways people have introduced tasks for added insight - rather than relying on spoken answers alone. If you’d like to read more from the UK Market Research Association on immersive market research see here
2. Structure your survey for the insight you need
Structure of a video task is one of the most important factors for drawing out authentic emotions; whether you are getting to know a potential new hire, testing ideas within your organisation or exploring how customers engage with your brand.
Short single day projects or surveys
Often the insight that’s needed can be targeted with a series of questions answered on a single day; like a preliminary job interview or a bit of team feedback within your organisation. With a single day task we face the challenge of overcoming the respondent’s nerves in front of a camera relatively quickly.
We’ve found that often even a single video is enough to lower the barriers to further responses. A test question - unrelated to and where possible asked separately from the principal task can be a great way to set participants more at ease and familiarise themselves with the process. This leaves them free to focus on their answers in subsequent questions, rather than worrying about being on video or the technology. This conscious effort to manage the discomfort most people initially feel in front of a camera can really help ensure you cut through to their genuine emotions, thoughts and feelings on your subject .
- Start easy: - ask them to talk about themselves or something fun related to the core topic if possible. Asking something fun but unrelated can also be valuable if it sets people at ease, and you will get to know your respondents better.
- Build up to the challenge: Place tasks or challenging questions closer to the end of the exercise. E.g. Please talk us through your wardrobe. What’s your newest purchase - why did you select it?
Multi-day diary and immersion projects
Sometimes the insight we need can only really be teased out over the course of a number of days, in the form of a video blog/diary or immersion where participants answer several questions per day over 3-7 days. You can really get to know your respondents and explore concepts and relationships with them more deeply than other formats.
This approach is closest to a face-to-face interview, however an in-depth interviewer will usually spend a portion of the available time up-front to build rapport with the subject. While we can’t do the same we can structure the tasks so that a significant portion of the first day is designed to put participants at ease; get them talking about things that motivate them and get comfortable with the camera and format.
We almost always see respondents relax and become more expressive and less inhibited over the course of these projects - ideally the structure should support and accelerate this trend.
- Start easy: As with single tasks move from questions focused on spoken answers in the early days of the project to more task-based questions towards the middle/end. With multi-day projects we have the luxury of more time to build up to more ivolved exercises. These can include interacting with a product or packaging on camera, taking us shopping or on a daily commute, even cooking and sharing a meal.
- Be adaptive: Review your questions as answers are received. With the multi-day approach you can easily see how your questions are performing, in terms of the insight they are delivering. Tweak them for subsequent days to fill gaps or dig deeper.
- Provide support: Encourage participants when you engage them for each day of the exercise - compliment their previous videos, make them feel valued and therefore more eager to complete the new exercise.
Breakout!
A common mistake is to make a single question too long, with too many components or sub questions. See the below for a real-life example of a single video prompt:
When this happens we usually see one of the two following outcomes:
- Most commonly respondents simply forget most of what was asked and respond to the first and last query.
- Participants record long rambling answers that can drift off point as they try and cover everything that was asked.
Not only do we lose out on valuable spoken information but many visual cues like facial expression and body language are confused as respondents struggle to remember the question. A better way is to set out two more concise and thematically linked queries that are easier to remember and so deliver more focused insight:
- Experiment: Test questions before you send them out - are they getting you the insight and content that you need? Are there gaps in the responses? Are the responses too long and rambling?
- Be a timecop: Aim for about 1.5 minutes at the most per response. Average talking speed is about 140 words per minute, so you can get a lot of information and insight in that time.
3. Nail your question design
"There are no right answers to wrong questions."
Ursula K. Le Guin
Once you’ve structured your interview it’s time to turn to the questions themselves - and make sure the tools are sharp and suited to the job - getting the best possible video content & insights! A great way to check if questions are being answered as intended is to test them on unsuspecting colleagues and friends! Also, be aware of and avoid the common question biases.
The yes/no trap
It’s always best to avoid closed questions with yes/no or single sentence answers. Video as a format delivers multiple channels of rich information - but only when you give respondents the right cues to really share their thoughts & emotions. We want to see where people pause, watch their expressions as they think about their answers and hear the emotive changes in their tone of voice. (For more tips on interpreting facial expressions - see our blog post
Use open questions that will leave the respondent free to voice their own, uninfluenced opinion. As a rule open questions tend to be descriptive - What, why, and how - that usually can’t be answered quickly and without contemplation and consideration.
- Anticipate the answer: E.g. “Who uses this product in your household” will result in a list of names more often than not. Instead ask “Please tell me who in the house uses this type of product and what are their reasons?” Who uses the product is valuable information, but the why is more important.
- Test motivations: Target respondents’ emotions, passions and perceptions to understand what drives their behaviour. This is true whether the respondent is a potential hire, a customer or a team member providing feedback.
Question funnels
A common and valuable questioning technique is to start with neutral open questions and move to more leading, but still open, queries that drill down into specific areas of interest. This approach remains absolutely relevant with video and can be tweaked to suit a wide range of situations.
This approach is also applicable for both single and multi day video projects. It can be particularly effective to separate general and more focused lines of questioning into different days on longer projects. It helps with thematic analysis and presentation and in addition allows for questions to be added or refined based on previous responses.
Emotive language and loaded questions
Video naturally lets you capture a wider range of emotive content than any other medium so the use of emotive language and loaded questions is very case specific. When it comes to research & feedback the standing rule to avoid emotive language and leading questions applies, to capture authentic & unbiased opinion. Neutral questions will capture all the complexities of the respondents emotions that traditional methods miss and in an uninfluenced way; through body language, tone of voice and facial expressions.
On the other hand there are some situations when you want to prime people’s responses. For testimonials or user generated content marketing it makes sense to ask what people love about a product - and video will help authentically and transparently communicate your participants’ passion.
- Be selective: Carefully match your questions to the desired content and insights. Neutral questions won't skew research results or feedback and carefully worded loaded questions don’t miss out on capturing emotion when it’s the goal for punchy, impactful user generated content.
Bringing it all together
There’s a general impression that participants are reluctant to participate in research tasks involving video (More on that in an upcoming blog). However, using careful survey and project design with the right incentives, we’ve seen amazing engagement across topics that range from the general to intensely personal as well as simple one-off to tasks through to complex multi-day diaries. The reward for overcoming participant reluctance is a wealth of visual, emotive and contextual insight that really is very difficult or expensive to collect in any medium other than video.