Skip to content
Closer

Data & Methods

Rigorous Methods. Radical Transparency.

Decision-makers deserve to know where every number comes from, who was reached, how it was weighted, and how personal data is protected. This page sets it out in full.

Observed vs. surveyed

Every chart says where its data came from.

Closer separates three kinds of data and labels each one on every chart, table and export. We never blend unverified estimates into primary research without clear disclosure.

Surveyed

Direct survey data

Answers collected from people your study set out to reach, using a questionnaire we design with you.

How it is gathered
Web panel, WhatsApp, SMS, USSD and field enumerators, all feeding one dataset.
What we show
Sample size (n), margin of error, fieldwork dates, channel mix and the exact question wording.
Best used for
Intent, awareness, satisfaction, priorities, willingness to pay, voting or service intent.
Watch out for
Only as representative as the sample. Weighting corrects known gaps, not unknown ones.
Reference

Public reference data

Official or published baselines, such as census population and demographic indicators, used for context and weighting.

How it is gathered
Loaded from published sources and cited with source name and year.
What we show
Source, publication year and geography level. No margin of error is added by us.
Best used for
Population by county, age and sex structure, urban and rural split, weighting targets, map overlays.
Watch out for
Reference data ages. Where a figure is older than your fieldwork, the report says so.
Observed

Observed public sentiment

What is being said in open channels: news sites, public social posts and comments, and open-ended survey answers.

How it is gathered
Monitored public sources are collected, de-duplicated, clustered into themes and scored for tone.
What we show
Mention volume, reach, theme and tone. Never a margin of error, because it is not a probability sample.
Best used for
Early warning, share of voice, why sentiment moved, themes to test in a follow-up survey.
Watch out for
Skews to people who post online. We never present it as what the whole population thinks.

What a label looks like on a chart

Surveyed · n=412 · ±4.8 pts · 3 to 14 MayReference · 2019 censusObserved · 1,840 mentions · no margin of error

Method

How a study runs, from question to report.

The same eight steps apply whether you run a 200-person pulse check or a multi-county baseline. Scale changes; the discipline does not.

  1. 1

    Scope the decision

    We start from the decision the study must support, then fix the audience, geography (national, county or local area), timeline and the number of reportable subgroups. This sets the sample size you actually need.

  2. 2

    Design the questionnaire

    Questions are written once and rendered for each channel. We keep wording neutral, randomise answer order where order could bias, and shorten instruments for SMS and USSD where screens and sessions are small.

  3. 3

    Build the sampling plan

    Quotas are set by county, urban or rural setting, age band and sex, using reference data. Each quota cell has a target and a minimum before it can be reported on its own.

  4. 4

    Pilot and soft launch

    A small pilot checks comprehension, timing, translation and routing on every channel before full fieldwork starts. Problem questions are fixed, not patched later in analysis.

  5. 5

    Field the study

    Channels run side by side. Quota progress is tracked live by area and group, and under-filled cells are topped up through another channel or an assisted field visit.

  6. 6

    Validate and clean

    Automated checks flag duplicates and implausible responses. Flagged records are reviewed, kept, or removed with the reason logged, so a cleaning decision can always be traced.

  7. 7

    Weight and analyse

    Weights bring the sample back toward known population shares. Small samples are shrunk toward a pooled value so a thin base cannot produce a dramatic headline.

  8. 8

    Report with its labels

    Every chart carries its data type, n, margin of error where it applies, channels and dates. Exports and PDFs keep those labels, so a number never travels without its context.

Representation

Reaching beyond the smartphone bubble.

Digital-only research in Kenya systematically under-samples rural residents, older people and feature-phone users. Our multi-modal design pairs low-cost digital channels with USSD, SMS and assisted field enumeration, so one questionnaire reaches balanced representation across income brackets and geographies. Channel is recorded on every response, so we can test whether channels disagree.

ChannelWho it reachesStrengthsWatch out forTypical use
Web panelSmartphone and desktop users, mostly urban and peri-urbanFast, cheap, supports long questionnaires, images and concept testingUnder-represents rural, older and feature-phone users on its ownConcept tests, brand tracking, professional audiences
WhatsAppSmartphone users with data, across income groupsHigh response, conversational flow, supports voice notes and imagesNeeds a lawful contact list and opt-in; skews younger and urbanMember and customer pulse checks, follow-ups, urban cohorts
Two-way SMSAny handset, including feature phonesWide coverage, works with weak data, simple to explainShort questionnaires only; STOP opt-out is always honouredShort polls, reminders, recontact for panel waves
USSDFeature-phone users, including many rural and older respondentsNo data or app needed; session is fast and anonymous by designVery short menus; session timeouts; shortcode set-up neededRural and low-connectivity coverage, service feedback, farmer polls
Field enumeratorsPeople who are hard to reach remotely: remote wards, older residents, low literacyAssisted interviews in local language; offline-capable forms; richest dataHighest cost and slowest; interviewer effects need back-checksBaselines, public participation, health and household studies

Languages

Questionnaires run in English and Kiswahili, with local-language wording added for field and WhatsApp studies where the audience needs it. Translations are back-translated on request.

Accessibility

Assisted interviews and short-menu USSD flows keep low-literacy and older respondents in the sample rather than quietly dropping them.

Channel effects

Because channel is recorded, we compare answers across channels and adjust or flag where the same question reads differently on SMS than in the field.

Sampling and weighting

A balanced sample first, a corrected one second.

Quota design

We set targets by county, urban or rural setting, age band and sex from census reference data. For national studies, counties are grouped into regions so every region has a reportable base. Small or hard-to-reach counties can be over-sampled and weighted back down.

Post-stratification weights

After fieldwork we compare the achieved sample with reference shares and calculate weights so over-represented groups count less. Extreme weights are trimmed so a handful of respondents cannot swing a result.

Small-base shrinkage

Area and subgroup scores based on few responses are pulled toward the pooled value in proportion to how thin the base is. A county with 14 respondents cannot outshout one with 140.

Minimum bases for reporting

  • n under 30

    Not reported. The cell is suppressed.

  • n of 30 to 99

    Shown as indicative, with a visible caution and wide margin.

  • n of 100 or more

    Reported normally, with margin of error.

Margins of error

How big is the uncertainty?

For surveyed percentages we report a 95% margin of error that includes a design effect, because weighted and quota samples carry less information than a simple random sample of the same size.

The formula we use

MOE = 1.96 × √( p × (1 − p) ÷ (n ÷ deff) )

p is the observed share (0.5 gives the widest margin), n is the valid sample and deff is the design effect, taken as 1.5 unless a study measures its own.

Doubling the sample does not halve the margin. Going from 400 to 1,200 completes narrows it from about 6 points to about 3.5. Subgroups carry their own, wider margins, so decide up front which groups you need to read separately.

Worst-case margin of error at 95% confidence (percentage points)
Valid responses (n)With design effect 1.5Simple random sample
100±12.0±9.8
200±8.5±6.9
400±6.0±4.9
800±4.2±3.5
1,200±3.5±2.8
2,400±2.5±2.0

Observed sentiment is not a probability sample, so it is shown without a margin of error and labelled as observed.

Data quality

Checks that run before a number reaches a report.

Duplicate detection

The same person or device answering twice is flagged by phone hash, device signature and answer pattern, and the later record is held for review.

Quota tracking

Completes are counted live against targets by county, setting, age and sex, so gaps are found during fieldwork when they can still be fixed.

Speeders and straight-liners

Interviews completed far faster than the pilot median, or with identical answers down a grid, are reviewed before they reach the dataset.

Location consistency

Self-reported county is compared with channel signals such as field-agent assignment or shortcode region. Mismatches are queried.

Enumerator back-checks

A share of field interviews is re-contacted or audited to confirm the interview took place and key answers match.

Open-text review

Free-text answers are screened for nonsense, abuse and personal data, then clustered into themes and checked by a person.

Audit trail

Every removal, recode and weight change is logged with who, when and why, so a number can be traced to its source records.

Observed sentiment

How public conversation is measured.

Observed data tells you what is being said and how fast it is moving. It does not tell you what a representative Kenyan thinks, and Closer never presents it that way.

  1. 1

    Collect

    Open news sites, public social posts and public comments in monitored channels. No private groups, no private messages.

  2. 2

    Clean

    Remove duplicates, reposts and obvious spam, then tag language (English, Kiswahili or mixed) and county where the source gives it.

  3. 3

    Cluster

    Group mentions into themes such as pricing, water access, feeder roads or service reliability, using topic clustering that a researcher reviews and renames.

  4. 4

    Score

    Score tone as positive, neutral or negative. Mentions are weighted by reach so one viral post does not distort the picture. Sarcasm and code-switching are spot-checked by a person.

  5. 5

    Report

    Show volume, theme, tone and the top sources, labelled observed, with the date window and the number of mentions analysed.

Reading the numbers

The terms you will see in a report.

Sample size (n)
How many valid responses a figure is based on. Always shown beside the figure. Subgroups have their own, smaller n.
Margin of error
The range around a survey percentage that likely contains the true value, at 95% confidence. Smaller n means a wider range.
Confidence band
The shaded range around a score or trend line. If two bands overlap, treat the difference as unproven.
Design effect
How much weighting and clustering reduce the information in your sample. We assume 1.5 unless a study measures its own.
Quota
A target number of completes for a group, such as women aged 18 to 24 in Kisumu, used to keep the sample balanced.
Weight
A multiplier that makes over-represented groups count a little less and under-represented groups a little more.
Shrinkage
Pulling a score from a small base toward the wider average so thin samples do not look more extreme than the evidence allows.
Observed vs. surveyed
Surveyed numbers come from questions we asked. Observed numbers come from what people said in public. They answer different questions.

Compliance

Kenya regulatory and privacy compliance.

Closer is designed around Kenya's Data Protection Act and the guidance of the Office of the Data Protection Commissioner (ODPC), and around Communications Authority of Kenya (CAK) standards for messaging.

Data protection (ODPC)

  • Personal and sensitive data is encrypted in transit and at rest.
  • We collect the minimum needed for the study and keep it no longer than the engagement requires.
  • Role-based access control means a coordinator sees only the territory and studies they are assigned.
  • Reports and exports are aggregated by default. Respondent-level data is shared only where the client has a lawful basis and a data agreement is in place.

Consent and outreach (CAK)

  • Every outreach workflow records timestamped opt-in consent before a respondent is contacted again.
  • Automated STOP opt-out requests are honoured across SMS, WhatsApp and USSD, in line with Communications Authority of Kenya standards.
  • Respondents are told who is asking, why, and how their answers will be used before they begin.
  • Participation is voluntary and respondents can skip any question.

Where contacts come from

  • We never sell, rent or bundle electoral registers or scraped personal phone lists.
  • Clients upload or collect only their own lawfully obtained contacts, or recruit through our consented panel.
  • Public sentiment is gathered from open sources only. We do not access private groups or private messages.
  • Observed data is analysed in aggregate and is not used to profile named individuals.

Honest limits

What our data cannot tell you.

Good methods include saying where they stop. These are the caveats we put in front of every client.

  • Quota samples are not random samples. Margins of error are an honest guide to sampling noise, not a guarantee against every bias.
  • Observed sentiment reflects people who post or comment publicly. Treat it as a signal to investigate, then confirm with a survey.
  • Small areas often have small bases. Where n is under the reporting minimum, the figure is shown as indicative or suppressed.
  • Stated intent is not behaviour. Purchase intent and voting intent run higher than eventual action, and reports say so.
  • Self-reported answers on sensitive topics are shaped by who is asking. Channel and enumerator effects are checked, not assumed away.

Take a closer look

Want to see the methods applied to your question?

Tell us your audience and geography. We will share a draft sampling plan, channel mix and sample-size options before you commit.