For research agencies & corporate teams
African Statistics Day: spotting sampling bias in business surveys
Use African Statistics Day to review coverage, self-selection and nonresponse in a business survey, with a practical disclosure template.
TL;DR
- Ask who could enter the study and who actually responded.
- Quotas and larger totals do not automatically remove selection bias.
- Report the sample source, exclusions and limits before generalising.
African Statistics Day is observed annually on November 18. For business research teams, it is a useful occasion to examine a familiar slide: a percentage labelled as “what customers think” with little explanation of who answered.
This guide focuses on sampling bias in agency and corporate surveys. The examples are original hypothetical situations, not national statistics or findings from a DatumForms study.
Separate the intended audience from the reached audience
Begin with the population your brief describes. Then identify who the recruitment approach can actually reach. A mobile-web link, an email list and an interviewer visit each create different opportunities to participate.
Suppose a hypothetical Ghana retailer asks about all recent shoppers but shares the survey only with loyalty-programme members. Customers outside that programme are missing from the source. Increasing responses from members cannot reveal the experiences of nonmembers who never had the opportunity to answer.
Coverage is a property of the route into the study, not a criticism of a particular respondent. Documenting it helps the team decide what the evidence can support.
Identify self-selection and nonresponse
Self-selection occurs when willingness to opt in is part of the process determining who appears in the data. An open complaint-page link may attract people with a particular reason to speak. Nonresponse concerns the selected or approached people who do not provide the relevant answers.
A high completion rate among starters says little about people who never clicked the invitation. Similarly, an invitation response rate does not reveal every reason participants and nonparticipants differ. Review the entire path from the source to the analysed records.
| Stage | Question for the research team |
|---|---|
| Source | Who is absent from the contact list or recruitment route? |
| Invitation | Who was approached, and how? |
| Participation | Who chose to start or declined? |
| Analysis | Who was excluded, and on what recorded basis? |
Treat quotas as composition controls
Quotas can help a team obtain responses within specified categories. They do not establish that everyone within each category has the same chance of inclusion, or that the people who join resemble those who do not.
For example, matching age-group totals does not necessarily balance internet access, purchasing behaviour or willingness to answer. Avoid calling a quota-balanced opt-in sample a probability sample. If weights are used, disclose the variables, reference data, method and remaining assumptions rather than presenting weighting as a universal repair.
Decide what comparisons are defensible
Compare achieved sample characteristics with relevant, trustworthy information when it exists and is appropriate to the study population. A national population distribution may not be the right target for a study of a narrowly defined business customer group.
Differences can be diagnostic without proving the size or direction of bias in every finding. Record what is known, what is plausible and what remains unmeasured. Consider complementary methods or a revised sample source when an important group is missing.
Use a short disclosure template
Include the following in the method section of a client or internal report:
- Intended population and geographic coverage.
- Source, selection approach, eligibility and recruitment route.
- Field dates, languages and data-collection mode.
- Invitations or approaches where known, achieved responses and exclusions.
- Quota or weighting decisions and the claims they do not justify.
- The population to which the findings can reasonably speak.
A suitable statement might begin, “These results describe eligible respondents recruited through this source during these dates.” Complete the statement with the actual project details rather than implying a representative view of Ghana.
Change the wording of the findings when needed
“Among respondents” is often more accurate than “Ghanaians believe” for an opt-in business study. Preserve useful observations while narrowing the claim to match the design.
The purpose of a bias review is to improve the next decision: change recruitment, add a missing source, narrow a comparison or communicate uncertainty more clearly. It should happen before collection and again before the findings leave the research team.
Continue the research workflow
- Sample size business survey
- Survey response rate denominator
- Survey observation and explanation
Explore current plans , discuss your study or create a business account . For business inquiries: info@datumforms.com .