I’ve been seeing a lot of applications lately where the methods section starts something like this:

In this project, we adopt a mixed methods approach…

It is a statement that I’m coming to loathe, because the author is generally saying:

Warning: muddled thinking ahead.
In the following section, we are going to provide a grab-bag of methods. Their connection to the research question will be weak, at best. Hell, they probably won’t even be well linked to each other…

There are no mixed methods

In a grant application, the purpose of the methods section is to show how you’re going to answer your research question. Not explore the question. Not interrogate the space. Not flail about without a clue.

Your methods should present a road-map from not-knowing to knowing. If you are using more than one method (and, let’s face it, who isn’t?), you need to show that your methods will:

1. Work (get you to your goal); and
2. Link together (be greater than the sum of their parts).

You need to show me both of those things, not just one of them (or, as is sometimes the case, neither of them).

My methods will work

You must demonstrate feasibility. ‘Had we but world enough, and time’, anything is possible. But you don’t have unlimited time and resources. You need to be able to show that your team, with your data, using your methods, can solve your problem, within the time allowed.

You need to be able to make a very convincing argument of how that will happen. So, for example, it isn’t enough to say:

Our analysis of this data will advance our understanding of this problem.

You need to explicitly say:

1. We are using this particular analytic technique because…
2. This technique unlocks this particular aspect of the data…
3. Unlocking that is important to solving the problem because…
4. Once that part of the puzzle is solved, we can…

That last point – about part of the puzzle leading to the next stage – is really, really important. It isn’t enough to show that the thing is feasible. You need to be able to explain that each step leads to the next step.

Sometimes, this is straightforward. For example: The workshop informs the questionnaire, which forms the basis of the interviews. Even if it is straightforward, you still need explain it.

My methods link together

I see four main criticisms of methods sections:

1. Not enough detail (most common criticism eva!).
2. Don’t see how the different parts relate.
3. Don’t see why a particular method is being used.
4. Too complex (not often, but it’s a killer).

When thinking about mixed methods, you need to concentrate on #2 & #3. They are both failures to describe how the different methods link together. If the reviewer doesn’t see how the different parts relate to one another, that isn’t their failure to understand. It is your failure to explain.

You have failed at one of several key points. Either you haven’t thought this through carefully enough, or you haven’t planned it out in enough detail, or you haven’t explained it well enough.

Applicants can seem confident things will work out, but they aren’t sure how. Essentially, they are saying ‘trust us, we will muddle through’. You can imagine how well that goes down with assessors.

Sometimes, applicants know that their techniques will work (often because they have done it before), but they haven’t clearly mapped out the process. This generally comes through as a lack of detail (see point #1). The methods section is squishy, like jelly. It looks solid, but when you poke it, it turns out to be soft. It doesn’t stand up to a rigorous review. So, you get something like:

We’ll do method one.
[Hand waving]
Then we’ll do method two.

There is no clear link between the methods.

Then there are the applicants who know exactly what they are going to do, and why, but just don’t write it down clearly enough. This can sometimes read like a very confident version of the ‘trust us’ problem. The method section skates along, gliding from point to point, assuming that the reader has all the same experience as the writer. This works well when your application falls into the hands of other experienced researchers, but it can come unstuck when your reviewer is less experienced, or from outside your field, and can’t skate along with you. They’ll be looking for the detail, and it isn’t there.

A simple example

I often see applications where people are going to undertake a survey, and then they are going to do some interviews. This seems straightforward until I start to look for the details.

• Will you draw your interviewees from your survey respondents, or are they two completely different cohorts?
• If they are linked, how will you identify them and contact them?
• If they are independent cohorts, how will you link your two data-sets? Will they be combined, or will they be analysed separately?
• If you want to combine the analysis, how will that work since the data-points are so different?
• If you are going to analyse them separately, when and how will you bring it all together?

If you aren’t going to synthesise your findings, doesn’t that mean that you are doing two different projects concurrently?

To avoid this problem, give your draft to a critical friend, preferably in another discipline, or your local research whisperer. Ask them to read it, and read it hard. Then sit down with them and answer every single question they have about the methods section. Debate with them the merits of the actual methods that you are going to use (without descending into a flaming war). Be open to their suggestions, even the ones that make you feel pedestrian or inexperienced. Explore that discomfort. By the end, you should have a really clear idea of how another reader sees your project.

Now, go back and look at your methods section again. Be ruthless. If you are just using the same methods that you always use, think about how well they are working. Are you using them because they are the best methods to use, or because you are comfortable with them?

If you are using four different methods because there are four different people in the team, have a long discussion about what you are collectively planning to do and why. This will be a difficult discussion, but it is much better to have it before you submit your application or start your project, than when you are trying to answer hostile criticisms, or when you are in the middle of the research mess.

Finally, you will be ready to go back to your application and delete the statement, ‘We have adopted a mixed methods approach.’ There are some statements that are so general that they don’t mean anything (thanks, @ThomsonPat). This is one of those statements. You don’t need it. Your methods description will be detailed and thorough and complete. It won’t need this useless label because, in the end, all methods are mixed.