Skip to content

Research

How we write our questions, and how we read the answers

Nearly four in five of the questions in Polzy’s bank are informed by published research, and each of those names the research it rests on. Most are in our own words; the few we adapt from public surveys say where they come from. Every result is worked out in the open: how many people it rests on, how far it could still move, and what it keeps back to protect them.

  • 487 questions in six languages
  • 380 informed by published research
  • About 110 sources, cited question by question

The bank

Questions that keep their words

A result means something only beside another one asked the same way. So every question in the bank has a permanent identity and fixed wording, and its answers compare across surveys and over time.

Fixed wording

A bank question keeps its words. Its answers count as that question’s in every survey that asks it, so a trend compares like with like.

Six languages

Each question is written in English, Dutch, French, German, Spanish and Arabic. Switching a survey’s language changes the words a person reads, not the question the answer counts towards.

Organised by what it measures

Every scored question files under one topic: manager support, workload, ease of use, value for money. A topic’s score is the average of the questions that measure it.

And by the moment it is asked

Each question is written for a moment: a regular pulse, a yearly survey, a new starter’s first weeks, someone leaving, a support request closed, a training finished, an event.

Where they come from

Informed by published research

438 of the bank’s 487 questions are in our own words. 48 adapt a question from a published survey, 46 of those from surveys the US and UK governments run, and one is asked as published. For 380 of them, what the question measures and how it is worded is informed by published research and by the public-sector surveys that publish their questionnaires and results. This is that research, by area.

Work and wellbeing

Research on job demands and resources, self-determination theory (autonomy, competence and belonging), psychological safety in teams, and organisational justice: whether decisions and treatment are fair.

Public employee surveys

The surveys governments run of their own staff and publish openly: the US Federal Employee Viewpoint Survey and the UK Civil Service People Survey, with the Australian Public Service Employee Census for one topic.

Customers and service

Research on service quality (being reliable, responsive and caring), on the effort it takes a customer to get something done, and on satisfaction and loyalty.

Products

Usability research on how easy a product is to learn and use, and research on whether it is useful for the job people bring to it.

Training and events

The four levels of training evaluation (reaction, learning, behaviour and results) and research on how what people learn carries over into their work.

How to ask

Survey methodology: how wording, answer scales, translation and the order of questions change the answers people give.

Questions by audience, and what they rest on

Employees170
Customers114
Product55
Events70
Learning53
Any survey10
Forms15
20015010050
EmployeesCustomersProductEventsLearningAny surveyForms
Forms are written from practice: a name, an email and a consent box need no theory.

“Informed by” means this research shaped what a question measures and how it is phrased, not that its words were taken from it. The 49 questions that do take their words from a published survey are recorded with that source and its licence. None of the organisations named here has reviewed or endorsed Polzy. We do not use questions from commercial questionnaires sold under licence, or from other survey companies.

By topic

The research behind each topic

Every topic the bank measures, with the research its questions are written against. Most of it shaped what a question asks, not its words; a question that takes its words from a published survey says so on its line in the survey builder.

How a question is written

Rules every question follows

Each question is drafted from a written definition of what it measures (what the topic covers, and what it does not) and checked against these rules.

  • One idea per question

    No “and” joining two things that could get different answers. “My manager is fair and easy to reach” is two questions, so it is written as two.

  • Plain words

    Short sentences (fourteen words at most in English) at an everyday reading level, with no jargon and no idioms, which do not translate.

  • In the first person

    A statement you agree or disagree with is about your own experience: “My manager is clear about what our team should focus on.” Satisfaction is asked as a question: “How satisfied are you with…?”

  • A scale that fits

    Statements are answered on five points from Strongly disagree to Strongly agree, each point labelled. Satisfaction, effort and how often something happens each have a labelled scale of their own.

  • Neutral, not leading

    No question assumes an answer or an experience: never “How great was…”, never “Since the training helped you…”.

  • Reversed questions: few, and planned

    A few questions say the opposite (“My workload is more than I can handle”) so that agreeing with everything stands out. At most one per topic, only in full surveys, never in a short pulse, and turned round before it is scored. Otherwise questions avoid “not”, which people misread.

How a question is checked

From draft to the bank

Every question in the bank was researched and drafted this way. The checks with experts, with readers and with professional translators come next, and the last one runs on real answers as they come in.

  1. 1

    Research and definitions

    Done

    Each topic gets a written definition and the research behind it, before any question is drafted.

  2. 2

    Drafting

    Done

    Several wordings per topic, written against the rules above, so the weaker ones can be dropped.

  3. 3

    Expert review

    Next

    An organisational psychologist for workplace questions, customer and user researchers for customer and product questions, and a learning specialist for training and events rate how well each question fits its topic.

  4. 4

    Think-aloud testing

    Next

    People answer the questions aloud as they read them, in each language, so a word that is misread or a question that is hard to answer shows up before anyone relies on it.

  5. 5

    Professional translation

    Next

    Translators work from a shared glossary of words like manager, team and workload; an independent back-translation checks the meaning survived; Arabic is checked right to left on the survey page. Until this step is done, the bank’s translations are drafted by machine.

  6. 6

    Checking with real answers

    As answers come in

    Once enough people have answered, figures across many answers (never one person’s) will show the questions that are often skipped, answered the same way by almost everyone, or that do not move with the rest of their topic. Those are reworded or retired, and a translation that is answered differently from the others is checked again.

What we may use

Our own words, and the few we adapt

438 of our 487 questions are entirely our own words. 46 adapt questions that the US and UK governments publish for reuse, and 3 come from a published product-market fit survey. Each one says where it comes from.

  • Our own words

    438
  • Adapted from US public-domain surveys

    The Federal Employee Viewpoint Survey 2024, and the federal customer experience standard in OMB Circular A-11 §280.

    42
  • Adapted from the UK Civil Service People Survey

    Under the Open Government Licence v3.0, with changes made.

    4
  • Product-market fit questions, with attribution

    Ellis (2010) and Vohra (2018).

    3

We use no question from a questionnaire sold under licence or from another survey company, and a test stops our build when a question comes too close to one.

In the builder

What each question rests on

In the survey builder, every bank question carries one of these labels, and the research-informed ones name their sources.

  • Research-informed

    380

    Written to a definition from published research, which the question names: 336 in our own words, 44 adapted from public surveys.

  • From practice

    107

    Written from running feedback programmes: the first questions Polzy shipped, open questions and form fields. 5 of them adapt or quote a credited source: 2 questions from the US federal customer experience standard, and the 3 product-market fit questions.

  • Used as published

    A published questionnaire’s own question, asked word for word. None of our questions carries this label today.

A question we write is not called validated because it was written with care. That takes evidence from real answers, in every language, and those checks begin as answers come in.

Scores

From an answer to a score

Each answer becomes a figure from 0 to 100: a rating’s point over its top point, so 4 out of 5 is 80. A question worded the other way round is turned round first. A survey’s score is the average of its answers, unless its author weights some questions more, and screens show it out of 10.

  • My manager takes my ideas seriously.4 of 580
  • My workload is more than I can handle.2 of 540turned round60
  • I trust my manager to treat me fairly.5 of 5100
  • Average80Shown as8.0

The recommend question (eNPS) is the share who answer 9 or 10, minus the share who answer 0 to 6.

Themes

Theme scores you can trace

A theme is scored only from the questions tagged with it that a person answered, and each response counts once. A theme nobody answered about has no score, not a zero.

  • Where the AI reading is on, a written answer can count towards its theme, and a theme no question asks about is scored only when the reading points to what was written.
  • A bank question asked in its own words keeps its score under a permanent id, so it compares with every survey that asks it. Reword it, or change its type or scale, and it becomes your own; put the words back and it counts again.
Questions from the question bank

Reading the results

How sure each number is

The dot beside a score comes from a 95% margin: how far the score could move if everyone had answered, worked out from how many people answered and how much their answers differ.

  • People count once, however often they answer.
  • The margin narrows when most of the people asked have answered.
  • Under ten people, or twice your privacy floor, no margin is stated, and the dot says how sure the number can be from the count alone.
How sure is this number?

Comments

Topics in what people write

A comment is filed under the theme of the question it answers. Where the AI reading is on, each response is also tagged with up to six topics from what was written.

  • A topic is recurring when it comes up in at least two campaigns, and new or rising against the same number of days before.
  • A topic fewer people raised than your privacy floor is not listed.
  • A summary of comments names nobody and gives no figure it was not given, and a quote must be the respondent’s own words.

Insights

Insights that show their working

Polzy’s insights are found by code, not by the AI. Seven checks compare the last 30 days with the 30 before, and each finding carries the figures that produced it and how sure it is.

The seven checks

  • A group moving 8 points or more against the rest of the workspace
  • A group whose own score rose 8 points or more
  • A topic that is new or rising in two or more campaigns
  • Negative answers concentrated, 40% or more, in one group
  • A group that has gone quiet
  • A topic whose mention comes with a score 8 points or more lower
  • More people showing a risk sign

The AI then writes the sentence and a next step from those figures. A topic that comes with a lower score is said to come with it, never to cause it.

AI analysis and Insights

Privacy

A privacy floor on every result

A group’s result appears only when enough people answered: three by default, or a higher number up to 50 that your workspace sets. It counts people, not answers.

  • When one group is hidden, Polzy hides the smallest of the groups shown too, to protect anonymity.
About the privacy floor

What we do not claim, yet

  • We do not say a question measures what it should until real answers show it does.
  • We do not compare you with other organisations: Polzy has no benchmarks.
  • We do not say what causes a score, only what comes with it.

Read more

Close the feedback gap.

Free for your first 1,000 responses a month. Team and Business start with a free trial.

Data stored in the EUGDPRCancel any time