Digital SAT Math · Problem-Solving & Data Analysis
Evaluating statistical claims
This skill is vocabulary wearing a data costume. A short study description lands on the screen — who was measured, how they were chosen, whether anyone was put into a treatment group — and the choices are four claims. The arithmetic is almost never the point. The point is matching the strength of the claim to the strength of the design: random selection decides whom you may generalise to; random assignment decides whether you may say “caused”.
On the test
| Domain | Problem-Solving and Data Analysis (score report) |
| CB skill | Evaluating statistical claims: observational studies and experiments |
| What it looks like | A short paragraph describing a survey or experiment, then “Which conclusion is best supported?” (or close) with four full claim sentences as choices |
| Often asked | “Which conclusion is best supported…?”, “To which population can the results be generalised?”, “Which statement must be true?” |
| Format | Multiple choice and student-produced response (design counts: how many were randomly selected / assigned) |
| Calculator | Allowed throughout; almost never useful here — the work is reading, not multiplying |
Recognition cues: randomly selected, randomly assigned, survey, observational, experiment, treatment, control, generalise, association, causes, based on the study, best supported.
Pattern recognition
Two design switches, and they are independent. Read the stem once for each.
- Random selection (sampling) — who was chosen to be measured, and out of what list? If every member of a named population had a chance to be picked, results may generalise to that population. If the sample was volunteers, a restaurant crowd, or “students who walked by”, the claim stops at the people actually measured.
- Random assignment (experiment) — were participants placed into treatment groups by a random process? If yes, a causal claim about the treatment is on the table. If researchers only watched people who already differed (gym members vs not, music-studiers vs not), the design is observational and the safe verb is associated, not causes.
A study can have both switches on, either one, or neither. The correct choice matches both.
Method
- Who was randomly selected? Underline the selection sentence. “Randomly selected from all registered voters in Oregon” is a different design from “asked every customer who entered the café on Saturday”.
- Out of what population? Write the population in five words or fewer: Oregon registered voters, employees at Company R, students at Lakewood High. That phrase is the outer wall of every legal claim. Crossing it is an overgeneralisation distractor.
- Was a treatment randomly assigned? Look for assigned, placed into groups, given treatment X or a placebo. Measuring people who already had the habit (exercise, coffee, standing desks they chose themselves) is not assignment. No assignment → no causal verb.
- Pick the narrowest claim the design supports. Among the remaining options, the correct one uses the right verb (causes only with assignment; associated otherwise) and the right population (the one in step 2, not “all adults nationwide”). Stronger-sounding claims are almost always the traps.
| Words in the stem | What they unlock |
|---|---|
| randomly selected from [population] | generalise to that population |
| volunteers / responded to a post / customers at one store | generalise only to the people measured (often: do not generalise) |
| randomly assigned / placed at random into | causal language about the treatment is allowed |
| asked / observed / compared people who already… | association only — no “causes” |
| large sample with no random selection | sample size does not repair a biased design |
Worked example 1 — random sample, no assignment
Stem. A city parks department randomly selected 400 residents from the full list of adult residents of Brookhaven and asked each whether they visit a city park at least once a month. Of those surveyed, 58% said yes. Which conclusion is best supported by the survey?
Step 1 — who was selected? 400 residents, randomly, from all adult residents of Brookhaven.
Step 2 — population wall. Adult residents of Brookhaven. Not “all Americans”, not “all park users”.
Step 3 — assignment? None. People were only asked about a habit they already had. Observational.
Step 4 — narrowest supported claim. An estimated 58% of adult Brookhaven residents visit a city park at least once a month. Association language about habits is fine; causal language (“visiting parks causes…”) is not on the table, and neither is any claim about adults outside Brookhaven.
Check. The selection sentence names Brookhaven adults and the word randomly; the assignment sentence does not exist. Answer: about 58% of adult Brookhaven residents visit a city park monthly — generalise to that city, no causal add-on.
Trap watch. “About 58% of American adults visit a city park monthly” crosses the population wall. “Visiting parks causes residents to report higher life satisfaction” invents both a causal verb and an outcome the stem never measured. “Exactly 232 Brookhaven adults visit parks monthly” treats the sample count as a census of the city.
Worked example 2 — random assignment, limited population
Stem. A company with 1,200 employees wants to test a new break-reminder app. Researchers randomly assign 150 of the company’s volunteer employees to use the app for six weeks and 150 to continue without it. At the end, the app group reports fewer afternoon headaches on average. Which conclusion is best supported?
Step 1 — who was selected? Volunteers from one company — not a random sample of all workers, not even a random sample of the company’s full staff.
Step 2 — population wall. At most, these 300 volunteers (or, carefully, “volunteer employees at this company”). Not “all office workers nationwide”.
Step 3 — assignment? Yes — randomly assigned to app or no app. Causal language about the app is allowed for this group.
Step 4 — narrowest supported claim. For these volunteer employees, using the break-reminder app caused a reduction in afternoon headaches (on average).
Check. Assignment is present → cause is legal. Selection is not a random sample of a broad population → do not generalise to all workers. Answer: the app caused fewer afternoon headaches among the study’s volunteer employees at this company.
Trap watch. “The app causes fewer headaches for all office workers” overshoots the population. “The app is associated with fewer headaches, but no causal claim is possible” understates the design — random assignment is exactly what unlocks cause. “Volunteering for a wellness study causes fewer headaches” misidentifies the treatment.
Worked example 3 — both switches on
Stem. From the full list of patients at Riverside Clinic, 240 patients with mild insomnia were randomly selected. Each selected patient was then randomly assigned to receive either a six-week sleep-coaching program or standard written advice. Patients in the coaching group reported significantly better sleep scores. Which conclusion is best supported?
Step 1 — selection. Random sample from Riverside Clinic patients with mild insomnia.
Step 2 — population wall. Patients at Riverside Clinic with mild insomnia — not every insomnia patient in the country.
Step 3 — assignment. Yes — coaching vs written advice, at random. Cause is allowed.
Step 4 — full-strength claim the design actually supports. The sleep-coaching program caused better sleep scores among patients with mild insomnia at Riverside Clinic.
Check. Both switches are on, so the correct claim uses caused and stops at Riverside Clinic’s mild-insomnia patients. Answer: coaching caused better sleep scores for mild-insomnia patients at Riverside Clinic.
Trap watch. “Coaching is associated with better scores at Riverside Clinic” is true but weaker than the design allows — the under-claim distractor. “Coaching caused better scores for all people with mild insomnia” crosses the population wall. “Selecting patients at random caused better sleep scores” confuses selection with the treatment.
Practice
Answer before you open the explanation. Ten items are multiple choice whose options are full claim sentences — each wrong claim breaks exactly one design rule. Two items are student-produced response that ask for a design number (how many people were randomly selected, or how many were assigned to a treatment). Every wrong choice below has a named trap.
Question 1 Warm-up
A counselor randomly selected 80 students from the full enrollment list at Lakewood High School and asked each how many nights per week they sleep fewer than seven hours. Of those surveyed, 45% reported fewer than seven hours on at least four nights. Which conclusion is best supported by the survey?
Show the answer Choice B
Why it is right
The sample was randomly selected from Lakewood High School's full enrollment, so the 45% result may be generalised to students at that school and no further. Nothing was assigned as a treatment — students only reported a sleep habit — so the claim must stay descriptive. Choice B matches both: the right population wall (Lakewood) and no causal verb.
Why each other choice fails
- Choice A
- Overgeneralisation past the sampled population. The sampling list was Lakewood High School only, so the result does not extend to all U.S. high school students.
- Choice C
- Causal language from an observational survey. No treatment was randomly assigned; students were only asked about a habit they already had, so 'causes' is not supported — and grades were never measured.
- Choice D
- Treats the sample count as a census. 45% of 80 is 36 surveyed students, not a proven headcount of every Lakewood student with that sleep pattern.
Question 2 Standard
Researchers asked 500 randomly selected adult residents of a city whether they hold a gym membership and measured each resident's resting blood pressure. On average, residents with a gym membership had lower blood pressure than residents without one. Which conclusion is best supported by the study?
Show the answer Choice C
Why it is right
Residents were randomly selected from the city, so generalisation to adult city residents is allowed. But nobody was randomly assigned to join a gym — researchers only recorded an existing membership and a blood-pressure reading. That is an observational study, so the supported claim is an association for the city, not a causal claim. Choice C is exactly that sentence.
Why each other choice fails
- Choice A
- Causal language from an observational study. Membership was not randomly assigned, so other differences between members and non-members could explain the blood-pressure gap.
- Choice B
- Stacks two design errors: a causal verb without random assignment, and a population wall that extends past the city to the whole country.
- Choice D
- Strongest-sounding option: 'every' and 'will' promise a certain individual effect the study never measured and could not guarantee even with random assignment.
Question 3 Standard
A manufacturing firm with 900 employees wants to test whether a new shift-stretching routine reduces reported muscle strain. Researchers randomly assign 120 volunteer employees to follow the new routine for four weeks and 120 volunteer employees to keep their usual warm-up. The new-routine group reports significantly less muscle strain. Which conclusion is best supported by the study?
Show the answer Choice A
Why it is right
Random assignment to routine vs usual warm-up unlocks a causal claim about the routine. Selection was from volunteers at one firm, not a random sample of all manufacturing workers, so the causal claim stops at the study's volunteer employees. Choice A uses 'caused' for exactly that group and does not overshoot the population wall.
Why each other choice fails
- Choice B
- Overgeneralisation past the sampled population. Volunteers at one firm are not a random sample of all manufacturing workers nationwide.
- Choice C
- Under-claim after random assignment. Association is true but weaker than the design supports — random assignment is what makes a causal claim about the routine legitimate for these volunteers.
- Choice D
- Misidentifies the treatment. Volunteers were in both groups; the difference tracked the routine assignment, not the act of volunteering.
Question 4 Standard
A student newspaper posts an optional online poll on its social-media page asking, 'Should the school day start after 9 a.m.?' Of the 340 people who chose to respond, 72% answered yes. Which conclusion is best supported by the poll?
Show the answer Choice D
Why it is right
Respondents self-selected by choosing to open a social-media poll. That is not a random sample of the school, the city, or any larger group, so the only population the 72% describes is the people who actually answered. Choice D stays inside that wall and makes no causal claim.
Why each other choice fails
- Choice A
- Generalises a voluntary-response sample to all students at the school. Students who follow the paper and choose to click may differ systematically from the full enrollment.
- Choice B
- Pushes the same voluntary sample even further past the population wall, to all teenagers in the city.
- Choice C
- Invented causal claim. Nothing was randomly assigned; the poll only recorded an opinion.
Question 5 Standard
A polling agency randomly selected 1,200 registered voters from the full list of registered voters in Oregon and asked each whether they support Measure Q. Of those surveyed, 54% said yes. Which conclusion is best supported by the survey?
Show the answer Choice B
Why it is right
The sampling frame was registered voters in Oregon, drawn at random, so the 54% estimate generalises to that population. It does not automatically extend to all Oregon adults (some adults are not registered) or to registered voters in other states. No treatment was assigned, so causal language about turnout is unsupported. Choice B matches the population wall exactly.
Why each other choice fails
- Choice A
- Widens the population from registered voters to all adults in Oregon. Unregistered adults were not on the sampling list.
- Choice C
- Overgeneralisation past the sampled population. Oregon registered voters are not a random sample of U.S. registered voters.
- Choice D
- Causal language the study never tested — turnout was not measured, and nothing was randomly assigned.
Question 6 Harder
From the full list of patients at Harborview Clinic who had been diagnosed with mild seasonal allergies, 180 patients were randomly selected. Each selected patient was then randomly assigned to receive either a new antihistamine nasal spray or a saline placebo spray for two weeks. Patients in the antihistamine group reported significantly fewer allergy symptoms. Which conclusion is best supported by the study?
Show the answer Choice C
Why it is right
Both design switches are on. Random selection from Harborview's mild seasonal-allergy patients lets the result generalise to that clinic population. Random assignment to antihistamine vs placebo unlocks a causal claim about the spray. Choice C uses 'caused' and stops at Harborview's mild seasonal-allergy patients — the full strength the design supports and no more.
Why each other choice fails
- Choice A
- Under-claim after random assignment. Association is true, but random assignment to spray vs placebo is exactly what supports a causal claim for this population.
- Choice B
- Overgeneralisation past the sampled population. The sampling list was Harborview Clinic patients, not every person with mild seasonal allergies.
- Choice D
- Confuses random selection with the treatment. Selection decides whom the result describes; the spray is what was assigned and compared.
Question 7 Harder Student-produced response
A transit agency maintains a roster of 4,800 monthly pass holders. For a satisfaction study, 360 pass holders were randomly selected from that roster and asked to rate the agency's on-time performance. Of those surveyed, 63% rated performance as good or excellent. How many pass holders were randomly selected for the study?
Show the answer 360
Why it is right
The stem states the design count directly: 360 pass holders were randomly selected from the roster of 4,800. That number — not the roster size, not 63% of either total — is the count of people who entered the sample. Random selection of 360 is what allows generalisation to the full roster; the question asks only for that selection count.
Answers students type instead
- 227
- Computes 63% of 360 (about 227), the estimated count of positive ratings in the sample, which is not the selection count asked for.
- 3024
- Computes 63% of the full roster (0.63 × 4,800 = 3,024), treating the sample percent as a census count of the population.
- 4800
- Reports the full roster size — the population the sample was drawn from — instead of the number randomly selected into the study.
Question 8 Harder
A national lifestyle magazine emails a product survey to its subscriber list. Of the 52,000 subscribers who choose to respond, 81% say they prefer Brand X coffee. Which conclusion is best supported by the survey?
Show the answer Choice A
Why it is right
Respondents self-selected by choosing to answer an email survey. A large voluntary sample is still voluntary: the 81% describes the people who responded, not all subscribers and not all adults. Sample size improves precision when the design is unbiased; it does not repair selection bias. Choice A is the only claim that stays inside the actual respondent group and avoids causal language.
Why each other choice fails
- Choice B
- Large-sample-fixes-bias trap. Fifty-two thousand self-selected responses are not a random sample of all adults in the country, so the result does not generalise nationally no matter how large n is.
- Choice C
- Overgeneralises from respondents to all subscribers. People who open and complete a product survey may differ from subscribers who ignore the email.
- Choice D
- Causal language from an observational, self-selected survey. Direction and cause are both unsupported.
Question 9 Harder
A researcher studied 200 students at one university who already studied either with background music or in silence (students chose their own habit). End-of-term exam scores were higher, on average, among the students who studied with music. Which conclusion is best supported by the study?
Show the answer Choice D
Why it is right
Students self-selected their study habit; the researcher only observed an existing difference. That is observational, so the supported claim is an association, not a cause. The 200 students were also not described as a random sample of the university (or of all universities), so the safest population wall is the students who were actually studied. Choice D uses association language and stays inside that group.
Why each other choice fails
- Choice A
- Causal language from an observational study. Students who choose music may differ in motivation, major, or prior skill — confounders random assignment would have balanced.
- Choice B
- Stacks a causal verb without assignment on top of overgeneralisation past one university to all university students.
- Choice C
- Strongest-sounding option: 'every' and 'will' promise a certain individual effect no observational study can deliver.
Question 10 Hardest
A city health department randomly selected 600 adult residents from the full list of adult residents and asked each how many hours they spend in parks per week and how they rate their overall life satisfaction. Residents who reported more park hours also reported higher life satisfaction, on average. Which conclusion is best supported by the study?
Show the answer Choice B
Why it is right
Random selection from all adult residents of the city allows generalisation to that population. No treatment was assigned — people reported habits and ratings they already had — so the claim must be association, not cause. Choice B hits both switches correctly: association language, city-wide adult population. Choice D is the under-generalisation trap that forgets random sampling was used.
Why each other choice fails
- Choice A
- Causal language from an observational study. Park hours were not randomly assigned, so other factors (income, free time, health) could drive both park use and satisfaction.
- Choice C
- Stacks a causal verb without assignment on top of overgeneralisation from one city to all adults in the country.
- Choice D
- Under-generalises after random selection. Because residents were randomly selected from the full adult list, the association may be extended to adult residents of the city, not only to the 600 people measured.
Question 11 Hardest Student-produced response
A hospital recruited 280 adult volunteers with mild lower-back pain for a device trial. Each volunteer was randomly assigned to one of two groups: 140 were assigned to use a new lumbar-support device during workdays for eight weeks, and 140 were assigned to continue their usual care without the device. How many volunteers were randomly assigned to use the new lumbar-support device?
Show the answer 140
Why it is right
The stem states that 140 of the 280 volunteers were randomly assigned to the new device arm and 140 to usual care. The question asks for the treatment-arm count, which is 140. Total volunteers (280) is the pool that was randomised; it is not the number assigned to the device.
Answers students type instead
- 8
- Reads the eight-week duration of the trial as if it were a headcount.
- 70
- Halves 140 again, as if only half of the device arm received the device.
- 280
- Reports the full volunteer pool — everyone who entered randomisation — instead of the number assigned to the device arm.
Question 12 Hardest
From the full enrollment of first-year students at Northfield College, 250 students were randomly selected. Researchers then recorded, for each selected student, average weekly hours spent in the campus makerspace and end-of-semester GPA. Students who spent more hours in the makerspace had higher GPAs, on average. Which conclusion is best supported by the study?
Show the answer Choice C
Why it is right
Random selection from Northfield's first-year enrollment allows generalisation to first-year students at that college. Hours in the makerspace were only observed, not randomly assigned, so the claim must stay at association. Choice C is the narrowest claim that uses both facts correctly. Choice D starts from the right association and then adds an absolute individual guarantee the design cannot support.
Why each other choice fails
- Choice A
- Causal language from an observational study. Students who choose more makerspace time may differ in motivation, major, or prior skill — the study did not assign makerspace hours.
- Choice B
- Stacks a causal verb without assignment on top of overgeneralisation from one college's first-years to first-year students nationwide.
- Choice D
- Strongest-sounding add-on: 'any' and 'guaranteed' turn a group-level association into a certain individual promise no observational study delivers.
Common mistakes
- Causal language from an observational study — “gym members have lower blood pressure, so membership causes lower blood pressure.” No random assignment means no “causes”.
- Generalising past the sampled population — a random sample of one high school’s students becomes “all high school students nationwide”.
- Picking the strongest-sounding option — absolute words (all, proves, anyone, will always) usually overshoot what the design supports.
- Treating a large sample as a fix for bias — 50,000 self-selected responses are still self-selected.
- Confusing random selection with random assignment — selection unlocks generalisation; assignment unlocks cause; neither unlocks the other.
- Under-claiming after random assignment — defaulting to “associated” when the stem clearly assigned treatments and a causal claim for the right population is available.
- Treating the sample count as a population census — “exactly 232 of the city’s adults…” when 232 is 58% of a sample of 400.
- Generalising from volunteers as if they were a random sample — people who signed up differ systematically from people who did not.
- Inventing a treatment — reading “students who chose to study with music” as if researchers assigned the music.
- Ignoring who could have been selected — if only soccer players were on the sampling list, the result does not extend to all students.
FAQ
Do I ever compute a margin of error on this skill? No. Margin of error and confidence intervals are a different CB skill (inference from sample data). Here the question is only whether the claim matches the design — selection, assignment, population wall.
What if the stem never says “randomly”? Then neither switch is on. You may describe the people who were actually measured; you may not generalise to a larger population, and you may not say the treatment caused the outcome.
Is “associated with” always safe? It is almost never false, but it is sometimes too weak. After a clean randomised experiment, the best-supported claim is allowed to say caused for the right population. If a softer association option and a correctly scoped causal option both appear, take the causal one.
How do SPR items work here? They ask for a design count already stated in the stem — how many people were randomly selected from the population, or how many were randomly assigned to one treatment arm. Type the number only; no units, no percent sign.
Desmos? Almost never. See the box above.