Weighing the evidence: answer key
Introduction to Social Research Methodology
Task 1: A defensible ranking
There is no single correct ordering, and the point of the task is the justification, not the sequence. The ranking below is defensible; the notes indicate where reasonable groups will diverge. Mark for whether students name the feature that raises or lowers credibility (peer review, disclosed methods, independence, quality control, currency), not for matching this order.
Figure 1 summarises the ranking below as it appears on the model-answer slide; the dashed brackets mark the three places where reasonable groups diverge.
1. Source B — peer-reviewed journal article (Kowalczyk & Ferreira)
The only source vetted by independent experts before publication. Beyond peer review itself, the abstract signals methodological strength students should spot: a large sample (1,842), a three-wave panel design tracking the same people over time, measurement of actual quitting rather than intention, and statistical controls for rival explanations (pay, contract type, labour market). This is the anchor source for an academic review.
2. Source F — scholarly book chapter (Novak)
Published by an academic press with editorial review — real quality control, though generally less adversarial than journal peer review. Its distinctive value is synthesis: it maps four decades of research and the competing models, making it an ideal orientation source and a rich mine of references for citation searching. Some groups may rank it first for exactly that reason; that is defensible if argued — but note that a 2019 review is not current, and a review chapter reports others’ evidence rather than new findings.
3. Source G — industry-association report (Retail Workforce Barometer)
Grey literature, but comparatively transparent grey literature: the method (annual survey of 240 member firms) and breakdowns are disclosed, and the descriptive statistics (turnover by format, region, role) are exactly the sectoral context an academic review can legitimately cite as context. Two credibility deductions: no independent review, and a clear lobbying interest — the report ends by calling for lower employment costs, so its framing of the problem is motivated. Membership surveys also over-represent firms willing to respond.
4. Source A — consultancy white paper (Hartwell & Grey)
Also grey literature, ranked below G because the commercial motive is more direct: the paper exists to sell the consultancy’s trademarked retention framework (“deployed with over 40 retail clients”). The headline statistic (68% “considering leaving”) measures loosely defined intention, not behaviour, and the proprietary survey’s sampling and questions are undisclosed. Its strengths are currency and reach (4,000 employees, 12 markets — data academics rarely have). G vs A is the classic dispute: groups who rank A above G on grounds of scale and multi-country coverage are making a reasonable argument, provided they acknowledge the sales motive. Either order earns full credit with that acknowledgement.
5. Source E — newspaper article (The Business Courier)
A quality broadsheet applies journalistic standards — fact-checking, editing, named sources including official statistics and an economist. But it is written for news value on a deadline, not as cumulative knowledge: three executives and one economist is anecdote plus authority, not a study, and the causal claim (wage competition from warehouses) is the journalist’s synthesis, not tested evidence. Useful as a signpost to the labour-ministry statistics it cites — chase the primary source, cite that instead.
6. Source C — practitioner blog post (Lem)
No quality control beyond the author; evidence is fifteen years of personal experience — unsystematic, unverifiable, and possibly shaped by the author’s consulting interests. Yet it is not worthless: the claim that exit interviews mislead is a genuine hypothesis-generator a research team could go on to test, and practitioner insight into mechanisms can be sharp. C vs E is the other legitimate dispute: a group ranking C above E because it at least proposes a testable mechanism, while the newspaper recycles others’ claims, has understood the exercise. Either order with reasoning is acceptable.
7. Source D — Wikipedia entry
Anyone can edit it, content changes without notice, and this entry itself flags that it needs additional citations — an unstable, uncitable source for academic work. Its proper use is as a gateway: the definitions (voluntary/involuntary, functional/dysfunctional turnover) orient a newcomer, and the reference list leads to citable sources. Students should conclude “use it, mine its references, never cite it”.
Managerial-context question
Expect (and credit) answers along these lines: A and G are the sources a board would find most immediately persuasive — current sector benchmarks (G) and multi-market scale plus actionable framing (A); Meridian’s 27% only means something against a sector baseline. E establishes salience and points to the wage-competition threat from e-commerce warehouses — directly relevant to Meridian’s own e-commerce expansion. C suggests a cheap practice (stay interviews) worth piloting even before research concludes. The key insight to reward: credibility for an academic review and usefulness to a decision-maker are different tests — grey and popular sources supply context, currency, and hypotheses, but the review’s arguments must rest on B and F.
Figure 2 sets the two tests side by side: B and F carry the review’s arguments; A and G, then E and C, give the Meridian board its context; E, C and D keep other legitimate uses in the review.
Task 2: Example search strings with commentary
Any coherent iteration deserves credit; what matters is that each refinement is motivated by inspection of results and the change is correctly explained. Approximate hit counts will vary — do not mark them against a target. A model log:
| Version | Search string | Approx. hits | Commentary |
|---|---|---|---|
| 1 | employee turnover retail |
Hundreds of thousands | Unusable breadth. Words matched separately, so results include finance papers on asset turnover and anything mentioning retail in passing. |
| 2 | "employee turnover" AND retail |
Tens of thousands | Quotation marks force the exact phrase — the single highest-value refinement. Results now on-topic but dominated by descriptive and non-European studies, and still missing work that says “attrition” or “quit”. |
| 3 | ("employee turnover" OR "staff attrition" OR "turnover intention") AND retail AND (antecedents OR causes OR "supervisory support") |
A few thousand | OR bundles the field’s synonyms; the third AND block tilts results from describing turnover to explaining it. First page now dominated by studies that would survive abstract triage. |
Figure 3 is the same log as it appears on the model-answer slide, with the highest-value refinement — quotation marks — picked out in version 2.
Further refinements worth crediting: a date filter (e.g. since 2015) for currency; -financial or -asset to remove residual finance hits; restricting a term to titles (intitle:turnover); adding frontline OR "service sector" to sharpen the population.
Common faults to correct: treating hit counts as the goal (the test is first-page quality, not smallness); stacking AND terms until nothing survives; writing OR between concepts instead of within them (turnover OR retail broadens disastrously); refining without saying why — the log’s final column is where the learning is.
Figure 4 gathers the rule (OR within a concept, AND between concepts), the further refinements worth crediting and the common faults to correct.
Timing and discussion guidance
- If groups stall on Task 1, push them past “it’s biased” to how the bias operates — who paid, who benefits, what was left undisclosed.
- In the wrap-up, stage the G-vs-A and C-vs-E disputes deliberately: they show credibility is a judgement with criteria, not a lookup table.
- Close by connecting to the session: the ranking exercise is the hierarchy-of-credibility slide made concrete; the search log is the documented search a systematic review demands — and exactly what their project reports should contain.