Clicky

The Global Ministerial Mental Health Summit

GlobalMHSummit.com

Before-and-After Scores in Brain-Supplement Ads: What a Missing Control Group Means

by

Short answer: a before-and-after score shows that a number changed in a group of people between two points in time. It cannot show that the supplement caused the change, because nothing in that design tells you what the number would have done without the product. A control group provides that missing comparison. This guide explains why, and gives you a five-question study card to run on any brain-supplement ad that shows a score.

What a before-and-after score can honestly show

A before-and-after figure, such as “average memory score rose from 52 to 71 in eight weeks,” is a real kind of data. It can tell you that the scores moved, by how much, and in which direction. It can help researchers decide whether a question is worth testing properly.

What it cannot do is separate the product from everything else that happened over those eight weeks. The design has no “what if they had not taken it” arm, so the change could come from the product, other factors, or a mix.

Why the score alone cannot establish cause

The National Center for Complementary and Integrative Health (NCCIH) describes the placebo effect as a beneficial outcome that follows a person's expectation that an intervention will help. Someone who starts a brain supplement expecting to feel sharper may score or rate themselves differently for that reason alone, regardless of what is in the capsule.

NCCIH also notes that when patients or investigators know who received which treatment, it can shape their impression of who improved, no matter how hard they try to avoid it. A before-and-after design without blinding leaves that door open.

Two more explanations are worth keeping in mind. These are general research-methods points rather than findings from the sources below. Taking the same test twice can change how you do on it, and people's scores and moods naturally move around over weeks. None of this means the product did nothing. It means the design cannot tell the explanations apart.

What a control group adds

NCCIH explains that in a controlled clinical trial, researchers compare groups that are as identical as possible in other respects: an experimental group that receives the treatment and a control group that receives standard care or an inactive substance. In a double-blind version, neither participants nor researchers know which group they are in.

MedlinePlus, from the National Library of Medicine, adds that randomized controlled trials give the clearest information about whether a treatment is effective and safe in humans. Random assignment helps prevent bias in the results.

The practical effect is simple. If both groups improve, the product's added benefit is whatever remains after subtracting the comparison group's change. In a published randomized, placebo-controlled supplement trial in people with migraine, for example, average migraine days fell in the supplement group (6.2 to 4.4) and also in the placebo group (6.2 to 5.2). That is a different supplement and a different outcome, so it is not evidence about any brain product. It simply shows that a comparison group can move too. A before-and-after number reported for the supplement group alone would have missed that.

The five-question study card

When an ad shows a score, look for the study behind it and ask these five questions. Where an answer is missing, write “not stated” and treat it as unknown. A missing answer is a gap in what you can conclude, not proof of anything else.

  1. Comparator: compared with what?
    • Look for: a control or placebo group, and how people were assigned to groups.
    • Why it matters: without a comparison group, you cannot tell how much of the change is the product's.
    • If it is missing: the result can still be a starting point for a better study. It cannot support “this caused the improvement.”
  2. Blinding: who knew who got what?
    • Look for: the words “blinded,” “masked,” or “double-blind,” and a placebo that looked the same as the product.
    • Why it matters: NCCIH advises checking whether participants and investigators were kept unaware of group assignment, because knowing can shape impressions of improvement.
    • If it is missing: expectation could explain some or all of the change.
  3. Attrition: what happened to everyone who started?
    • Look for: how many people enrolled, how many finished, and whether the reported average includes those who left.
    • Why it matters: the CONSORT 2025 reporting guideline for randomized trials includes a flow diagram showing the progress of all participants through the trial. An average that counts only finishers describes finishers.
    • If it is missing: you cannot tell whether the group behind the number is the group that began.
  4. Outcome: what exactly was the score?
    • Look for: whether it was a performance test or a questionnaire people filled out about themselves, whether the measure was named in advance, and whether the ad reports only the result that looks best.
    • Why it matters: a self-rated feeling and a timed memory task answer different questions.
    • If it is missing: you can't check a number without a named measure. Our article on which measure a well-being study used covers this in more depth.
  5. Funding: who paid and who is reporting?
    • Look for: a funding or conflict-of-interest statement, and whether the report comes from the researchers or from the seller.
    • Why it matters: MedlinePlus points out that people paying for research can sometimes profit if the study gets certain results. This is a reason to look harder, not a verdict on the study.
    • If it is missing: note it and weigh the claim more carefully.

The card in use: an invented example

This is a made-up ad, not a real study. It says: “Memory score up from 52 to 71 in 8 weeks.” Run the card.

  • Comparator: the ad mentions only one group. Not stated.
  • Blinding: nothing about it. Not stated.
  • Attrition: it says “participants” but not how many began or finished. Not stated.
  • Outcome: “memory score” doesn't name a test. Not stated.
  • Funding: no source given. Not stated.

Five blanks do not tell you the claim is false. They tell you the ad has not given you what you need to judge it. Now suppose a full report showed a comparison group that took an identical-looking placebo and also rose, say, 15 points. The supplement group's added gain would be about 4 points, a very different story from “52 to 71.” The card is built to surface that gap.

What a missing control group does not tell you?

A before-and-after design without a control group does not show that a product is ineffective, and it does not show that anyone was careless or dishonest. It shows that the design cannot answer the cause question. Some products have been tested properly, and some have not. The card helps you tell which situation you are looking at, and an honest “not stated” is a fair place to leave it.

For a worked example of setting one product page's claims against what is documented, see this site's CogniHoney claims-versus-evidence review. It illustrates the kind of checking described here, not a reason to buy or avoid anything.

When a memory or focus concern needs a person, not a score

If you notice lasting changes in memory, attention, or thinking that are getting worse or interfering with daily life, a health professional can evaluate them in a way no ad or score can. Bring a note about what you are experiencing, how long it's been going on, and what you already take, including supplements. If you are on prescribed treatment, talk with your prescriber before changing it because of something you read.

Where to go next

Sources checked

This article is for general educational purposes and is not medical advice. It does not diagnose, treat, or recommend any product. See our Medical Disclaimer and Editorial Standards.

Related guides