Research or not?: answer key
Introduction to Social Research Methodology
Task 1: research or not?
The criteria come from the lecture: social research follows an explicit, transparent and repeatable procedure, frames its questions with theory (existing concepts and explanations), and settles its claims with systematically gathered evidence rather than intuition, anecdote or authority. A verdict earns credit only if it names the criterion that decides it. Figure 1 summarises the model answers; the reasoning for each follows.
1. The regional manager’s store visits — not research. What is missing is an explicit procedure. The three stores and the staff on shift were picked haphazardly, nothing was asked in a standard way, and nobody else could repeat the visits and check the conclusion. “Basically fine” is a personal impression, not evidence. The visits might be good management; they are not research.
2. The annual engagement questionnaire — research. The same 20-item anonymous instrument goes to all 450 employees at the same time each year, and the procedure is documented: the method is explicit, transparent and repeatable, and the evidence is systematically gathered. Comparing results year on year gives it a question to answer (how is engagement changing?). Some pairs will call it routine monitoring; that position is defensible and leads straight into Task 3 (see Figure 3), so credit it if the pair says what the survey would need in order to count as research.
3. The CEO’s memo — not research. What is missing is evidence. The claim that hybrid working damages company culture is settled by two press articles and the CEO’s authority; no data were gathered, so there is no procedure to inspect or repeat.
4. The warehouse fieldwork — research. A predefined observation protocol (an explicit, repeatable procedure), a structured daily log (systematically gathered evidence) and a clear aim — to understand how safety rules are followed in practice — make this systematic observation in context: an ethnographic study.
5. The checkout A/B test — research. Visitors are assigned to version A or B at random and the outcome (completed purchases) is recorded for both groups: this is an experiment, and anyone could repeat it. Pairs sometimes dismiss it as “just business analytics”; the lecture’s criteria do not require a university or an academic purpose, only a systematic procedure that answers a question with evidence.
6. The consultancy’s four-day-week poll — research, but bad research. There is a procedure and there are real data, so it has the form of research. Its sample, however, is self-selected: newsletter subscribers who chose to click through. The 78% describes those readers, not “employees”, and nothing about the procedure allows the leap from one to the other. Accept “not research” if the pair argues that a procedure this unrepresentative fails the systematic evidence criterion, but the stronger answer — and the one Task 3 builds on — is that real data and real numbers can still make bad research.
7. The quarterly turnover figures — not research. The figures are compiled systematically and repeatably, but what is missing is theory: they fill a reporting template rather than answer a question framed by concepts. This is the hardest scenario to classify, and the reason is the point of Task 3: being systematic is necessary for research, but not sufficient.
8. The team leader’s interviews — research. The same five open questions for every team member (a standardised procedure), answers recorded with permission (consent, so also ethical), and coding for recurring themes (systematic analysis): a small qualitative interview study. Eight participants is not too few; qualitative research works with small samples by design.
Marking guidance for Task 1
- The justification must name a criterion. “It feels unscientific” earns nothing; “no explicit procedure: the stores and staff were picked haphazardly” earns full credit.
- Numbers are not the test. The commonest error is ticking scenario 6 as good research because it has a percentage, and scenario 8 as not research because it has no numbers. Use both in the plenary.
- Systematic is not enough on its own. Pairs who call scenario 7 research because it is orderly and repeatable have spotted half the definition; ask them what question the figures are designed to answer.
- Research is not confined to universities. Scenarios 2, 5 and 8 are run inside organisations for managerial purposes and still count, as the lecture’s “organisations run on it too” point says.
Task 2: what kind of research?
The five scenarios classified as research are placed on the lecture’s two axes in Figure 2: the kind of data and reasoning (qualitative or quantitative) and whether the researcher intervenes (experimental or descriptive).
| Scenario | Data and reasoning | Intervention | Why |
|---|---|---|---|
| 2. Engagement survey | Quantitative | Descriptive | Standardised items, aggregated and compared year on year; nothing is manipulated. |
| 4. Warehouse log | Does not fit neatly | Descriptive | Immersion in a setting to understand practice is qualitative; a predefined protocol and structured log produce entries that can be counted. |
| 5. Checkout A/B test | Quantitative | Experimental | Visitors are randomly assigned to two conditions and an outcome is counted. |
| 6. Four-day-week poll | Quantitative | Descriptive | A percentage from a poll; the bad sample does not change the type of study. |
| 8. Team interviews | Qualitative | Descriptive | Open questions, coded for themes; the team leader asks but does not change anything. |
The odd one out is scenario 4. It combines the qualitative logic of ethnography (immersion, behaviour in context, the aim of understanding how rules are followed) with the quantitative logic of a structured protocol, whose entries could be counted and compared. Accept “qualitative” or “mixed” for its first-axis placement as long as the pair names the tension. Some pairs will nominate scenario 8, on the grounds that coded themes can be counted; accept the argument if it is made explicitly, but point out that coding for themes is the qualitative analysis the lecture described.
Scenario 2 is descriptive, even though it compares one year with another: comparing over time observes change; it does not create it. Pairs who call it experimental have confused comparison with intervention.
The qualitative–experimental cell is empty. Use that in the plenary: what would a study in that cell look like here? For example, the company pilots a new rota in some stores and interviews staff in those stores about how they experienced it — an intervention studied qualitatively (session 9 calls this an embedded design).
Task 3: plenary discussion
Where is the line between good record-keeping and research?
The line is a research question the data are designed to answer (Figure 3). Record-keeping can be just as systematic, documented and repeatable as research — scenario 7 is all three — but it serves a template or a routine, not a question framed by concepts. The same figures cross the line the moment they are used to answer one: why is turnover higher in some stores than others? Scenario 2 sits close to the line, which is why it is defensible either way. Designed to measure a concept (engagement) and track how it changes, it is research; filed and forgotten each March, it slides back into monitoring.
Expected discussion points:
- Systematic is necessary but not sufficient. This restates the definition from the start of the lecture: explicit procedure and theory and evidence.
- The difference lies in purpose, not in the data. Identical turnover figures can be record-keeping in one report and evidence in a study; what changes is the question they are asked to answer.
- Organisations hold vast amounts of systematic data (HR records, sales, customer feedback) that are not yet research. Session 6 (secondary data analysis) returns to exactly this: turning existing records into evidence for a question.
Scenario 6: why is it still bad research, and how would you fix it?
The main problem is who answered (Figure 4). The poll reached only the consultancy’s own newsletter subscribers, and only those who chose to click through: a self-selected, non-probability sample. The 78% describes those readers, and nothing allows the claim about “employees” in general; people interested in a four-day week are probably more likely both to read such a newsletter and to click on a poll about it. Two further problems are worth naming but are secondary: who asked (a consultancy may gain from a headline result) and what was asked (the question wording is not published, so it may have led respondents).
To fix it, using the lecture’s stages:
- Define the population the claim is about — for example, all employees in Poland.
- Draw a probability sample from a list of that population (a sampling frame), so that every member has a known, non-zero chance of selection and the result can be generalised.
- Ask a neutral question and publish its exact wording.
- Report the method and its limits: sample size, response rate and who commissioned the study, so others can judge (and repeat) it.
Credit any answer that identifies self-selection as the central flaw and proposes a probability sample of a defined population. An answer that only says “ask more people” misses the point: a larger self-selected sample is still self-selected.
Before you leave
There is no model answer: each student writes about a decision from their own experience. A strong pair of sentences looks like this:
- My employer moved every team to hot-desking because the managers believed it would increase collaboration across teams.
- Did employees in teams moved to hot-desking communicate with colleagues in other teams more often in the six months after the move than in the six months before?
Check that (a) names a concrete decision and the intuition behind it, and that (b) is answerable with evidence rather than a statement of the intuition in question form (“Is hot-desking good?”). Session 2 picks these sentences up when it turns topics into research questions.