Foundations of secondary analysis
Primary and secondary data
Primary and secondary data
Two columns. Primary data are collected by you, for your own question, with an instrument you designed: a survey you commission, interviews you conduct, an experiment you run. Secondary data already exist: collected by someone else, usually for some other purpose: government statistics, a rival consultancy's survey, your own company's HR records. Under both, the distinction is about origin and purpose, not quality: excellent studies are built on each. The key consequence: with secondary data, every design decision, who was sampled, how questions were worded, how concepts were defined, was made by someone else, before you arrived. Everything attractive, and everything risky, about secondary analysis flows from that fact.
PRIMARY DATA
Collected by you
for your own question, with an instrument you designed
A survey you commission
Interviews you conduct
An experiment you run
SECONDARY DATA
Already exist
collected by someone else , usually for some other
purpose
Government statistics
A rival consultancy's survey
Your company's HR records
The distinction is about origin and purpose — not quality: excellent studies are built on each
THE KEY CONSEQUENCE
With secondary data, every design decision was made by someone else , before you arrived:
who was sampled
how questions were worded
how concepts were defined
Everything attractive — and everything risky — about secondary analysis flows from this one fact.
What is secondary analysis?
What secondary analysis is
At the top, Babbie's definition: secondary analysis is research in which data collected and processed by one researcher are reanalysed, often for a different purpose, by another. Below, two lanes. The original study runs from its question, to the data collected, to its analysis, to its findings. Your study starts from your own question, reuses the same data, and runs a new analysis to new findings: the data are not new; the analysis is. Bottom left: collection, here, means locating, obtaining and preparing existing data, which makes secondary analysis a genuine data collection method. Bottom right: for a manager it is usually the first method to consider, because the answer may already be sitting in a filing cabinet or a public database; the method has grown enormously as governments, organisations and academic projects have opened their datasets.
SECONDARY ANALYSIS
Data collected and processed by one researcher are reanalysed — often for a different purpose — by another
Babbie
THE ORIGINAL STUDY
Their question
Data collected
Their analysis
Their findings
YOUR STUDY
Your question
The same data
A new analysis
New findings
the data are not new
the analysis is new
COLLECTION, HERE, MEANS
locating
obtaining
preparing
existing data — not generating fresh observations:
a genuine data collection method
FOR A MANAGER
Usually the first method to consider: the answer may already
be sitting in a filing cabinet or a public database.
It has grown enormously as governments, organisations and
academic projects open their data.
Not the same as a literature review
Secondary analysis is not a literature review
A comparison in four rows. What you reuse: a literature review reuses other researchers' findings and conclusions; secondary analysis reuses their raw data. What you do: read, evaluate and synthesise, against reanalyse, with new questions, new comparisons and new statistics. Output: a synthesis of what is known, against a new empirical result. You depend on: the authors' interpretations, against the quality of the original data collection. In short: a review tells you what others concluded; secondary analysis lets you check, extend or contradict them with the evidence itself.
Literature review
session 5: other researchers' conclusions
Secondary analysis
today: other researchers' data
What you reuse
What you do
Output
You depend on
Others' findings and conclusions
Read, evaluate, synthesise
A synthesis of what is known
The authors' interpretations
Others' raw data
Reanalyse: new questions, new comparisons,
new statistics
A new empirical result
The quality of the original data collection
IN SHORT
A review tells you what others concluded ;
secondary analysis lets you check, extend or contradict them with the evidence itself.
Why managers should care
Why managers should care about existing data
Organisational questions rarely arrive with a budget for a bespoke study, so before commissioning anything a competent research team asks two questions: what do we already have, inside the organisation, and what already exists out there? Existing data establish context. Two small charts show Meridian's turnover rising from 14% to 27% in two years. In the first, the sector's turnover stays flat: Meridian is an outlier, and the problem is Meridian's. In the second, the sector rises too: Meridian is riding a sector-wide wave, which calls for a different diagnosis and a different remedy. At Meridian: before surveying a single employee, ask what the company's own records, and Poland's official statistics, already reveal.
No budget for a bespoke study comes attached — so before commissioning anything, ask:
INSIDE THE ORGANISATION
What do we already have?
OUTSIDE IT
What already exists out there?
CONTEXT: IS OUR PROBLEM UNUSUAL, OR SECTOR-WIDE?
Meridian is an outlier
Meridian
sector
two years ago
now
14%
27%
a Meridian problem: look inside
A sector-wide wave
Meridian
sector
two years ago
now
14%
27%
a sector problem: a different remedy
AT MERIDIAN
Before surveying a single employee: what do Meridian's own records — and Poland's official statistics — already
reveal?
Sources of secondary data
Internal sources: the organisation’s own records
Internal sources: the organisation's own records
Most organisations sit on more data than they ever analyse. Four internal sources, one at a time. HR records: headcount, contracts, tenure, absence, pay grades, and exits by store and month. Sales and CRM data: transactions, customer complaints, satisfaction scores, loyalty-card behaviour. Exit interviews: routinely conducted, rarely systematically analysed. Past employee surveys: engagement or satisfaction studies from previous years, often run online. At the bottom: all of them were collected for administration, not research. That is their weakness, because they record only what the process needed, and their innocence, because nobody shaped them to please a researcher.
Most organisations sit on more data than they ever analyse
HR records
headcount
contracts
tenure
absence
pay grades
exits by store and month
Sales and CRM data
transactions
complaints
satisfaction scores
loyalty-card behaviour
Exit interviews
Routinely conducted — rarely systematically analysed : a
qualitative dataset waiting for an analyst
Past employee surveys
Engagement or satisfaction studies from previous years,
often run online (CAWI) — results on a shared drive
COLLECTED FOR ADMINISTRATION, NOT RESEARCH
THEIR WEAKNESS
They record only what the process needed — your variable
may not be there
THEIR INNOCENCE
Nobody shaped them to please a researcher: no social
desirability in a rota log
Internal sources at Meridian
What Meridian's own records could tell the board
At Meridian, before any new data are collected, four internal sources each answer a different question, with a caveat. HR records: who leaves, from which stores, roles, tenure bands and managers. Exit interviews: what reasons leavers give, but only if the interviews were conducted consistently enough to compare. Past engagement surveys: did warning signs appear before the turnover rise, but only if the same questions were asked each year. Sales and rota data: do high-turnover stores also show falling sales or heavier weekend scheduling? Such correlations suggest a mechanism but do not prove it. Every one of these is analysis of existing data: the marginal cost is analyst time, not fieldwork.
AT MERIDIAN: WHAT COULD ITS OWN RECORDS TELL THE BOARD — BEFORE ANY NEW DATA?
HR records
Who leaves — which stores, roles, tenure bands,
managers?
Exit interviews
What reasons do leavers give?
only if conducted consistently enough
to compare
Past engagement
surveys
Did warning signs appear before the rise?
only if the same questions were asked
each year
Sales and rota data
Do high-turnover stores show falling sales or
heavier weekend rotas ?
a correlation suggests a mechanism; it
does not prove one
NOTICE
Every one of these is analysis of existing data. The marginal cost is analyst time , not fieldwork.
External sources: official statistics
External sources: official statistics
Three official sources, from national to cross-national. Statistics Poland, GUS, the national statistical office: employment, wages and labour demand by sector and region, much of it in the Local Data Bank. Eurostat, the EU's statistical office: harmonised labour-market data, including the EU Labour Force Survey, comparable across member states. The OECD: cross-national indicators on employment, job tenure and earnings across rich democracies. What unites them: a statutory mandate, a published methodology, and large, carefully designed samples, which give them credibility and comparability. Their limit: they describe sectors and countries, not your firm. A bar of Polish retail made up of thousands of firms contains Meridian as one sliver: GUS can tell the board about Polish retail, and nothing about store 17.
national
European
cross-national
Statistics Poland (GUS)
Employment, wages and labour
demand by sector and region
much of it in the Local Data Bank
Eurostat
Harmonised labour-market data, incl.
the EU Labour Force Survey
comparable across member states
OECD
Cross-national indicators:
employment, job tenure, earnings
across rich democracies
WHAT UNITES THEM
a statutory mandate
a published methodology
large, carefully designed samples
credibility, comparability
THEIR LIMIT
They describe sectors and countries — not your firm
Polish retail: thousands of firms in one figure
Meridian
GUS can tell the board about
Polish retail — and nothing
about store 17
External sources: reports and academic surveys
External sources: reports and academic surveys
Two more external sources. Industry-association and consultancy reports, from retail federations, HR consultancies and job portals, are often timely and sector-specific, but you must ask who paid for them and what they are selling. Large academic surveys are designed for reuse from the outset: the European Social Survey, on attitudes and behaviour across Europe every two years, and the European Values Study, a long-running study of values including attitudes to work. Underneath, a scale of documentation runs from a headline and a chart to the questionnaire, sampling design, response rates and data files. Industry and consultancy reports spread across the low and middle of the scale; official statistics sit high, with published methodologies; academic surveys such as the ESS and EVS sit at the top: the gold standard of documentation.
INDUSTRY AND CONSULTANCY REPORTS
retail federations
HR consultancies
job portals
+ often timely and sector-specific
− ask who paid for them —
and what they are selling
A job portal's report on how hard it is to keep staff is also an
advert for a job portal.
ACADEMIC SURVEYS, DESIGNED FOR REUSE
European Social Survey (ESS)
attitudes and behaviour across Europe, every two years
European Values Study (EVS)
a long-running study of values, incl. attitudes to work
+ questionnaires, sampling designs and data files
published openly
HOW WELL DOCUMENTED?
a headline and a chart
questionnaire, sampling design,
response rates, data files
industry and consultancy reports: it varies
official statistics
published methodology
ESS, EVS
the gold
standard
Digital trace and big data
Digital trace and big data
Digital trace data are found, not designed: behaviour leaves traces as a by-product, and nobody is asked anything. Examples: website clickstreams, till and loyalty-card transactions, staff rota and badge-swipe logs, social media posts, online job-board activity. Their attractions: enormous volume, fine grain, continuous collection, and behaviour recorded as it happens rather than recalled afterwards. But, being found rather than designed, they have no sampling frame, no consent for research, and they cover only those who leave traces, so they need more methodological caution, not less. The line to remember: big data tells you what people did, at scale; it is usually silent on why.
FOUND DATA, NOT DESIGNED
Behaviour leaves traces as a by-product —
nobody is asked anything
website clickstreams
till and loyalty-card transactions
staff rota and badge-swipe logs
social media posts
online job-board activity
ATTRACTIONS
enormous volume
fine-grained
collected continuously
Behaviour recorded as it happens — not recalled afterwards, so no
memory slips and no effort to look good
BUT: FOUND, NOT DESIGNED
no sampling frame
no consent for research
covers only those who leave traces
needs more methodological caution, not less
WHAT people did
at scale, in fine detail
big data tells you this
WHY they did it
motives, meanings, intentions
it is usually silent on this
Assessing fitness for purpose
The fitness-for-purpose checklist
The fitness-for-purpose checklist
Interrogate any secondary dataset like a witness, with seven questions in three groups that follow the anatomy of a study. Provenance: one, who collected it, and are they competent and independent? Two, why, for what purpose and with what interest in the result? Sampling: three, when, is it current enough and consistent over time? Four, from whom, what population, sample and response rate? Measurement: five, how, by what mode and procedure: PAPI, CAWI, CATI, administrative records or digital trace? Six, with what instrument, what exact questions or measures? Seven, under what definitions, how are the key concepts defined, and have the definitions changed? At the bottom: if the documentation cannot answer these questions, that silence is itself a finding.
Interrogate any dataset like a witness:
PROVENANCE
1
Who
collected it — competent and independent?
2
Why
for what purpose — with what interest in the result?
SAMPLING
3
When
current enough — and consistent over time?
4
From whom
what population, what sample, what response rate?
MEASUREMENT
5
How
mode and procedure: PAPI, CAWI, CATI, records, digital trace
6
With what instrument
the exact questions or measures
7
Under what definitions
how key concepts are defined — and whether that changed
If the documentation cannot answer, that silence is itself a finding
Who collected it, and why
Who collected the data, and why
Who: collectors range from a national statistical office and a university consortium, through an industry body and a consultancy, to a vendor with a product to sell; each has different competence and different incentives. Why: purpose shapes design. Data collected to administer, such as payroll and till records, record what the process needed and no more. Data collected to persuade, as in some consultancy reports, may be sampled and worded conveniently. Data collected to research, such as GUS, Eurostat or the ESS, are designed for inference, but for someone else's inference. The quick test: would the collector be embarrassed by any particular result? If yes, read the methodology section twice.
WHO COLLECTED IT?
national statistical office
university consortium
industry body
consultancy
vendor with a product to sell
no stake in the result
a stake in the result
different competence, different incentives
WHY WAS IT COLLECTED? PURPOSE SHAPES DESIGN
To administer
payroll, tills, rotas
Records what the process needed —
no more
To persuade
some consultancy reports
May be sampled and worded
conveniently
To research
GUS, Eurostat, ESS
Designed for inference — but for
someone else's
A QUICK TEST
Would the collector be embarrassed by any particular result?
If yes, read the methodology section twice.
When, and from whom
When the data were collected, and from whom
When: vintage is substantive, not a footnote: a 2019 study of retail employment describes a pre-pandemic labour market. For trends, you need consistent measurement across waves, the same question, the same definition, the same population, every time; a series that changed its question in the middle is two series wearing one label. From whom: identify the population, the sampling method and the achieved response rate. Two traps. A national survey of employees recruited from one job portal's users is a survey of that portal's users. And aggregates hide composition: an average retail wage mixes hypermarkets, boutiques and e-commerce warehouses, and may describe none of them.
WHEN?
2019
2020
now
a 2019 retail study
describes a pre-pandemic labour market:
vintage is substantive, not a footnote
For trends: consistent measurement
across waves
wave 1
wave 2
wave 3
same question
✓
✓
✗
same definition
✓
✓
✓
same population
✓
✓
✓
A question changed in wave 3? Then it is two series
wearing one label .
FROM WHOM?
population
sample
responses
response rate = responses ÷ sample
Trap 1: a self-selected sample dressed as a general one
employees
A “national survey of employees” recruited
from one job portal is a survey of that
portal's users
Trap 2: aggregates hide composition
hypermarkets
boutiques
e-commerce warehouses
one “average retail wage” — which may describe none of them
How, with what instrument, under what definitions
How, with what instrument, under what definitions
Three measurement questions, one row each. How: the mode shapes the answers: CAWI panels, CATI interviews and administrative registers each have characteristic biases; face-to-face and telephone modes invite socially desirable answers, more in session 8. With what instrument: get the exact wording, because two items reported under the same label, are you satisfied with your job, and would you recommend working here, are different variables wearing similar labels. Under what definitions, the decisive question for comparability: does turnover mean all separations, voluntary quits, or replacements hired? Does retail include e-commerce fulfilment? Does employee include agency staff? Only when definitions match can you compare Meridian's 27 per cent with anyone else's number.
How?
the mode
CAWI panels
CATI interviews
administrative registers
Each mode has characteristic biases: face-to-face and telephone modes invite
socially desirable answers — more in session 8.
With what
instrument?
exact wording
“Are you satisfied with your job?”
reported as “employee satisfaction”
“Would you recommend working here?”
also reported as “employee satisfaction”
≠
Different variables wearing similar labels — no exact wording, no comparison
Under what
definitions?
the decisive question for
comparability
“turnover”
all separations?
voluntary quits?
replacements hired?
“retail”
incl. e-commerce fulfilment?
“employee”
incl. agency staff?
incl. civil-law contracts?
Only when definitions match can Meridian's 27% be compared with anyone else's number
Applying the checklist at Meridian
Applying the checklist at Meridian
An industry report says retail staff turnover in Poland averages 31 per cent. Set beside Meridian's 27 per cent, it looks reassuring: the board's relief is that the problem must be normal. Then the checklist, one question at a time. Who and why: the report comes from a recruitment agency, which profits from portraying turnover as universal. From whom: an online panel of 400 HR managers, self-selected, with no reported response rate. Definitions: the report counts planned seasonal contract expiries, which Meridian's figure excludes. The comparison collapses: the 31 and the 27 per cent are not measuring the same thing. The lesson: a number without its methodology is not evidence; fitness for purpose is established, never assumed.
THE COMPARISON
31%
industry report
27%
Meridian
The board, relieved:
“We're below average — the
problem is normal!”
Who and why
✗
A recruitment agency — it profits when turnover looks universal
From whom
✗
An online panel of 400 HR managers , self-selected, response
rate unreported
Definitions
✗
Counts seasonal contract expiries — Meridian's 27% excludes
them
The comparison collapses:
31% and 27% are not measuring the same thing
THE LESSON
A number without its methodology is not evidence: fitness for purpose is
established, never assumed
Combining secondary and primary research
Secondary first, primary second
Secondary first, primary second
In organisational diagnosis the efficient sequence is to exhaust the existing data before collecting new data. Secondary analysis does three preparatory jobs for the primary research that follows. It establishes context: is the problem unusual, or sector-wide? It sharpens the question: internal records show where and among whom the problem sits, so primary research can ask why. It provides benchmarks: the yardstick against which your own results will be judged, as when a survey finds 40 per cent of staff dissatisfied with scheduling and the board asks whether that is a lot. All three feed a primary study that is smaller, sharper and cheaper. And it disciplines spending: never pay to collect what already exists.
1 Exhaust the existing data
2 then collect new data
Establishes context
Is the problem unusual — or
sector-wide?
national retail figures vs Meridian's 27%
Sharpens the question
Records show where and among
whom — so primary research can ask
why
the rise sits in six stores with one regional
manager
Provides benchmarks
The yardstick for your own results
“40% dissatisfied with scheduling — is
that a lot?”
A primary study that is smaller, sharper and cheaper than it would otherwise be
Never pay to collect what already exists
What secondary data cannot do
What secondary data cannot do
Four hard limits, each pointing at what the primary study must still do. Secondary data cannot answer questions nobody's data address: no public dataset contains Meridian employees' reasons for leaving, so ask Meridian's own people. They cannot rescue a mismatched unit of analysis: sector aggregates never resolve into stores, teams or managers, so collect data at the level of the question. They cannot reach meanings and motives: records show that people left, not what leaving meant to them, which calls for qualitative methods. And they inherit every flaw of the original collection, permanently: no reanalysis repairs a biased sample or a leading question, so design your own sample and instrument. The question is never secondary or primary, but what must the primary study still do, once the secondary work is done?
Secondary data cannot…
…so the primary study must
answer questions nobody's data address
no dataset contains Meridian employees' reasons for leaving
ask Meridian's own people
rescue a mismatched unit of analysis
sector aggregates never resolve into stores, teams, managers
collect at the level of the question
reach meanings and motives
records show that people left — not what leaving meant
use qualitative methods
(session 7)
escape inherited flaws , permanently
no reanalysis repairs a biased sample or a leading question
design its own sample and
instrument
Never secondary or primary? but
what must the primary study still do, once the secondary work is done?
A combined design for Meridian
A combined design for Meridian
A four-stage research plan for Meridian's board, drawn as a staircase in which each stage costs more than the one before. Stage one, internal secondary analysis: HR records give turnover by store, role, tenure and manager, and first of all show whether the definition of a leaver changed. Stage two, external secondary analysis: a benchmark against GUS and Eurostat retail employment and wage data shows whether Meridian is an outlier or riding a sector-wide wave. Stage three, primary qualitative research: interviews with recent leavers from the worst-affected stores, targeting the why that the records cannot reach. Stage four, primary quantitative research: a survey of current staff, designed around what stages one to three revealed. The records tell the interviews whom to talk to; the interviews tell the survey what to ask. Each stage is cheaper than the next, and each makes the next one smarter.
1 Internal secondary
HR records: turnover by
store, role, tenure,
manager — first, was
“leaver” redefined?
2 External secondary
Benchmark against GUS /
Eurostat retail data: an
outlier, or a sector-wide
wave?
3 Primary qualitative
Interviews with recent
leavers from the worst-hit
stores: the why
4 Primary quantitative
A survey of current staff ,
designed around what
stages 1–3 revealed
the records tell the interviews whom to ask; the interviews tell the survey what to ask
Each stage is cheaper than the next — and makes the next one smarter