You have just completed the last of your expert interviews. You have hours of recordings and a mind filled with half-digested insights. Now comes the bit that all researchers secretly dislike the most: turning all this into something that actually makes sense.
Let’s face the facts. Transcription alone can take between 3 to 10 hours of work for every single hour of recording. Conduct 8 expert interviews and you’re looking at a whopping 80 hours of work just in transcription alone, before the actual analysis has even begun. This is the thing that all researchers get wrong, and this is why all expert interviews are essentially expensive conversations that never quite live up to their true potential.
In this guide, we’ll show you the entire process of analysis step by step, so nothing falls through the cracks between the conversation and the conclusion.
Why Analysis Is Where Expert Interviews Either Pay Off or Fall Apart
The interview was great. The expert was smart, the conversation was easy, and you leave with your head full of good stuff. And that is the first trap. The ‘good’ stuff doesn’t always translate to ‘good’ stuff on paper. The two most common errors in qualitative research are confirmation bias, where your own preconceptions skew your understanding of the data, and overgeneralizing, where what one or two experts say is assumed to apply universally.
So, what exactly are these common mistakes? Here’s what they look like in real life:
- You remember the most interesting quote and base your whole conclusion on it, while ignoring the three experts who said the opposite
- You code the data one way because it supports your hypothesis, when in fact, any objective coder would have coded it differently
- You finish the research with the same opinion you started with, before any expert interviews were conducted
- You simply present what people said, rather than what it all means
Step 1: Transcribe Fast and Transcribe Properly
Transcribe within 24 hours of each session because the more you wait, the more you lose in terms of context.
A one-hour interview can take 6 to 7 hours to transcribe correctly. This is a reality. And there is no way to do it faster that won’t cost you something. Whether you use some kind of automated transcription service that misses the nuances of language or you have someone else in the office who was not in the room and therefore cannot provide the context behind the words spoken.
The key elements that should be included in a proper transcript:
- The exact words spoken. This means including the “ums,” the pauses, and the corrections. These can also be telling in their own way.
- A time stamp system to make it easy to find key points later.
- A note about tone and body language. This can be important if the expert seemed hesitant, defensive, or overly confident about something.
- A memo to yourself that you write within a day or two. 3 to 5 things that caught your attention while the interview was still fresh in your mind.
Step 2: Code the Transcripts With a System
Coding is essentially going through your transcripts and coding different sections based on different themes, so that you can bring together all that different experts have to say on similar topics.
Even with a system in place, there are many judgment calls that lead to inconsistencies – especially if there is more than one person involved in coding. One person codes a comment on supplier risk under “operations.” Another person codes that same comment under “market uncertainty.” Do that ten times for ten different expert interviews and your conclusions start to go awry without you even realizing it.
There are two ways to go about coding:
- Start with your questions – design your themes based on your questions and then map your data to your questions. Good option if you know what to look for
- Let the data lead – read through your interviews without any preconceived notions and see what themes come up on their own. Good option if your expert interviews were exploratory and you didn’t know what to look for
Most of the time, the correct answer is in between.
The errors that ruin good coding:
- Trying to code sentences instead of ideas, as one block of text can and usually does have multiple themes
- Creating so many specific codes that nothing ties in with anything else
- Only coding the parts that back up what you already believe
- Not writing down why you coded something the way you did, because if you can’t explain the logic behind the code, the analysis is not defensible
Step 3: Find the Four Things That Actually Matter
This is the part of the process where the expert interviews actually earn their keep, as you’re no longer simply reading the transcripts but actively seeking out specific information.
- Consensus
Where are the experts agreeing with each other, without having talked to each other or even knowing each other? Four experts who don’t know each other and agree on the same topic without any prior discussion or research are unlikely to be doing so by accident.
- Disagreement
Where are the experts disagreeing with each other, and why? This is not a problem to be solved but a discovery to be made, and it is likely to be the most interesting part of the data you collected.
- Surprises
What were you surprised to hear, and what could you not have looked up in any other research? These are likely to be the most valuable results of the expert interviews, as you’d be unlikely to have come across the information elsewhere.
- Names that come up repeatedly
If you find that multiple experts are referring you to the same person, then it is likely to be worth investigating, as it is unlikely to be a coincidence.
Step 4: Work Out What to Trust
Not all the insights gathered in the expert interviews are of the same level of importance. A part of the process is understanding what to trust and what not to.
Be suspicious of:
- Strong opinions without any underlying reasoning
- Opinions that are advantageous to the expert in their professional or business capacity
- Information that is only provided by one expert and not others
- First reactions to difficult questions – people often give the easy answer before the correct one
Trust more:
- The things that experts say that don’t need to be asked
- The comments that cost the expert something to say – if a person is saying something that is contrary to their interests, they are more likely to be saying it
- The language that repeats itself across multiple interviews – if people who have not spoken to one another before are using the same language to describe something, that is a sign
- The moments that catch the expert in a contradiction – that is always where the truth is
Cross-checking and being transparent about how you have interpreted the results is non-negotiable in any credible form of research. The Business Research Company In terms of interviews with experts, that means checking the results you have come up with against other interviews and secondary research.
Step 5: Pick a Framework and Structure Your Findings
Before you even write a single word of your final product, you need to think about how you’re going to structure what you learned from your expert interviews.
The big options:
- By theme: Group your findings by what kept coming up in all of your expert interviews. Good for most expert interviews; easiest for your clients to understand
- By type of stakeholder: Show where your experts disagreed. Good when your interviews were with people who had significantly different roles
- By timeline: Show how something developed over time by what your experts said in your interviews. Good when you’re doing market or industry research
- By what we don’t know: Show where we don’t really know something. Sometimes the most interesting thing you can do with your findings is to precisely map what you don’t know.
Pick an option based on your findings.
Step 6: Write It Up in a Way People Can Actually Use
A few things that separate good write-ups from forgettable ones:
- Don’t lead with the methodology, lead with the finding – no one needs to read three paragraphs about how many expert interviews you conducted before you tell them what you actually learned from those interviews.
- Don’t over-quote from expert interviews – a good quote from an expert interview can help solidify a finding, but too many quotes can actually dilute the findings.
- Don’t pretend to have a consensus when you don’t – nothing loses credibility faster than pretending to have a consensus of experts when you don’t.
- Don’t conflate findings and interpretations – they’re different, and people can tell.
- Don’t forget to state the limitations of the research – expert interviews are qualitative research, and no one who actually understands research will think you’re claiming anything else, so you might as well state the obvious to prove you’re honest about what you’re doing.
Final Thoughts
The interview is just the beginning. What happens after that is where all the real work is done, and where all the real value is created. The value that determines whether all that effort interviewing experts was worth doing in the first place.
Did you get the analysis right? Then your research is useful. It influences decisions. It provides value to your client that they can’t find anywhere else. Did you get it wrong? Then all that effort is for nothing. It is just a stack of transcripts and reports that nobody ever acts on. Most people know how to conduct expert interviews. Not nearly so many know how to get all that value out of them once the recording stops. That’s where most of the value is and where most of it is lost.
At Katrium, analysis is what our market research services are all about. We make sure that nothing is lost in translation from conversation down to the conclusion. reach out for our help!