The operationalisation ladder
Introduction to Social Research Methodology
Scenario
You are the junior research team advising the board of Meridian, a mid-sized Polish retail-and-services company with around 450 employees, 32 stores, and a growing e-commerce arm. Annual staff turnover has risen from 14% to 27% in two years, customer satisfaction scores are falling, and the board wants evidence, not hunches.
The board has approved a measurement programme. Before a single questionnaire is printed, the team must do the work this session was about: climb down the ladder from abstract concept to concrete survey item, and make sure every number that will eventually land on the board’s dashboard was made well.
Work in pairs for Task 1, individually for Tasks 2 and 3.
Task 1: Build the ladder (25 minutes)
With your partner, choose one of the two concepts the board most wants measured:
- Employee engagement, or
- Customer loyalty.
Build a complete operationalisation ladder for your chosen concept, writing down each rung:
- Conceptual (nominal) definition. In one or two sentences, stipulate what the concept means for this study. Remember: concepts have no true definitions, only more and less useful ones — so choose deliberately, and make sure your definition would let a reader distinguish your concept from its neighbours (engagement is not mere attendance; loyalty is not mere satisfaction). (5 minutes)
- Dimensions. Identify 2–3 dimensions — specifiable aspects or facets of the concept that could, in principle, move independently of one another. Give each a name and a one-line description. (5 minutes)
- Indicators. For each dimension, propose 2 concrete indicators — observable signs of the presence, absence, or degree of that dimension. Indicators may come from a questionnaire or from Meridian’s records (HR system, tills, loyalty-card data); at least one record-based indicator somewhere in the ladder is encouraged. (5 minutes)
- Draft survey items. For each questionnaire-based indicator, draft one survey item exactly as a respondent would see it — the question or statement and its response options. For record-based indicators, state precisely what would be extracted from which system. Next to every item or extract, note its level of measurement (nominal, ordinal, interval, or ratio) — and where an ordinal rating would in practice be treated as interval, say so. (10 minutes)
Set the ladder out so it can be read in both directions: from any single survey item, it should be possible to climb back up through indicator and dimension to the concept it serves.
Does every dimension have indicators, and every indicator an item? Could another team, handed only your ladder, produce the same measurements? What has your operational definition deliberately ruled out — and are you comfortable with that?
Task 2: Sort the variables (10 minutes)
Meridian’s data warehouse and draft questionnaire between them contain the twelve variables below. Working alone, assign each variable its level of measurement: nominal, ordinal, interval, or ratio. Be ready to justify each answer in one sentence — and if you think a variable is genuinely arguable, note why.
| # | Variable | Level of measurement |
|---|---|---|
| 1 | Store ID (stores numbered 1–32) | |
| 2 | Department (sales floor / e-commerce fulfilment / head office) | |
| 3 | Contract type (permanent / fixed-term / B2B) | |
| 4 | Tenure in months | |
| 5 | Monthly sales per store, in PLN | |
| 6 | Number of absences in the last quarter | |
| 7 | Satisfaction rating (1 = very dissatisfied … 5 = very satisfied) | |
| 8 | NPS category (detractor / passive / promoter) | |
| 9 | Age bracket (18–24 / 25–34 / 35–44 / 45 and over) | |
| 10 | Seniority band (junior / mid / senior) | |
| 11 | Year the store opened | |
| 12 | Score on the standardised aptitude test used in recruitment |
Is zero real — does it mean the complete absence of the thing? Then ratios are meaningful. Are the numerals just labels of convenience? Then no arithmetic is meaningful at all, however numeric the column looks.
Task 3: Diagnose the defective measures (10 minutes)
A colleague on the team has drafted two operational definitions. Both are defective — but defective in different ways. Working alone, identify for each one whether the central problem is a validity problem (the measure does not capture the concept it claims to) or a reliability problem (the measure captures it inconsistently), and explain the fault in two or three sentences. One diagnosis per measure: name the main disease, not every symptom.
Draft measure A. “Employee engagement will be measured as the number of hours each employee worked last month, taken from the payroll system. Payroll data are checked monthly and error rates are near zero, so the measure is fully trustworthy.”
Draft measure B. “Customer satisfaction will be measured by asking customers leaving the store: ‘How do you feel about things these days?’ Whichever staff member is on shift will listen to the answer and record the customer as satisfied or dissatisfied, as they judge best.”
Wrap-up discussion (10 minutes)
Pairs briefly report: which rung of the ladder was hardest to build, and why? Which of the twelve variables divided the room? For the two defective measures, agree as a class on the diagnosis — and on what the archer’s target from the lecture says about draft measure A’s claim to be “fully trustworthy”.