Blog 

Start with the exact claim, not “clinically proven”

Write the claim as a testable sentence: This device improves this outcome in this population over this period, compared with this alternative. Then locate each element in the paper. If the advertisement promises fewer visible wrinkles but the study measures satisfaction with facial appearance, the outcomes do not automatically match.

For U.S. health-related advertising, the Federal Trade Commission emphasizes that evidence must fit the specific product and claimed benefit, and that the whole body of relevant research matters. One favorable paper cannot resolve conflicting evidence by itself. These are U.S. substantiation principles, not a statement about an individual device’s regulatory status. [1]

This worksheet applies the FTC’s relevance questions to LED research; it is not a validated scoring tool. A shared color name or wavelength alone does not demonstrate a complete protocol match. The American Academy of Dermatology also notes that differences in devices, study duration and research questions complicate interpretation of at-home red-light evidence. [1] [7]

Keep the technology boundary explicit. A paper on ultraviolet treatment, intense pulsed light, a laser, light used with a photosensitizing drug, a bright-light lamp or infrared heating is not automatically evidence for an ordinary LED skin-care mask.

Follow the people from enrollment to analysis

The headline sample size may describe enrollment rather than the people with usable measurements. Find the participant flow diagram and the denominator for the outcome you care about. CONSORT, the Consolidated Standards of Reporting Trials, asks authors to report group-specific allocation, treatment receipt, analysis counts, losses and reasons. [2]

• Screened: assessed for eligibility; some never enter the trial.
• Randomized: assigned to groups; this is the starting point for comparing assigned treatments.
• Followed up: supplied measurements at a particular visit.
• Analyzed: included in a particular result; this count can differ by outcome.

Ask whether losses were balanced and why people left. Missing observations can distort results when absence is related to the outcome—for example, when people with disappointing results are harder to follow. There is no universal dropout percentage below which bias disappears. Cochrane’s guidance considers the reasons and likely relationship to the outcome, not just the fraction missing. [3]

An intention-to-treat analysis aims to compare people in their randomized groups regardless of adherence. A result limited to people who completed the schedule answers a different question. Neither label tells you how missing measurements were handled: read the actual inclusion rules and sensitivity analyses. [3]

Read primary, secondary and satisfaction endpoints separately

The primary outcome is the main planned test. Secondary outcomes answer additional questions; exploratory analyses generate leads for further investigation. Find the measurement, analysis method and time point, then compare the publication with the protocol and registry. Do not promote a favorable secondary finding into an unqualified claim that the main test succeeded. [8]

A real example: different measures gave different answers

Bragato and colleagues’ 2025 trial abstract describes 95 women aged 45–60, a red LED mask at 660 ± 10 nm, two active application-frequency groups and a sham group over four weeks. These details identify the research setting; they are not instructions for another device. [4]

These statements come from the journal abstract and the current NCT04911140 registry record. Full trial methods and the historical registry versions were not verified for this article. We therefore do not assign a risk-of-bias rating, establish the original prespecified hierarchy or judge whether reporting differences were justified. [4] [5]

Ask how large the difference was

A small p-value does not measure effect size or importance, and it does not give the probability that a claim is true. Look for the actual difference between groups, its uncertainty interval and the scale’s meaning. An instrument-detected change and a noticeable everyday improvement are separate questions. [6]

For any “improved by X%” headline, identify the denominator: percentage change in a score, percentage of participants meeting a threshold, or percentage reporting satisfaction? Those statements answer different questions. Also ask whether the percentage describes change from baseline or an advantage over the control. [8]

Check the sham, the masking and the funding

What did the comparison group actually receive?

A sham aims to resemble the treatment experience without delivering the intended active intervention. Read its light output, appearance, sounds, visit schedule and associated skin care. A no-treatment group answers a different comparison from a convincing sham. If treatment is combined with another procedure, ask whether the control received that procedure too before attributing the whole result to LED.

Random allocation and blinding serve different purposes. Allocation concealment protects assignment before enrollment decisions can be influenced; blinding limits the influence of knowing the assignment afterward. “Double-blind” is incomplete unless the report identifies who was blinded and how. With visible-light interventions, similarity of the active and sham experience deserves particular attention. [8]

Who paid, and what could they influence?

Read funding, device-supply and conflict-of-interest statements. Ask whether a sponsor helped design the comparison, controlled the data, performed the analysis or influenced publication. Funding alone does not settle reliability; the disclosed role and safeguards are the useful questions. An academic affiliation is not a substitute for these details. [8]

Check registration, corrections and what happened afterward

A registration number lets you compare the planned question with the reported one. ICMJE recommends public registration at or before first participant consent for enrollment, to help prevent selective reporting. Check the registration submission date, enrollment timing and earlier versions; first public posting and first submission are different dates. A current record alone cannot establish what was declared before the trial began. [9]

Compare the registered outcomes, measurement times and planned enrollment with the paper. Changes are questions to investigate, not automatic evidence of misconduct. Look for explanations in the methods, protocol or supplements. Separately check the journal’s current article page for corrections or retractions. “No notice seen” is more limited than a comprehensive publication-integrity audit.

Finally, distinguish the end of treatment from later follow-up. A measurement taken while treatment continues cannot, by itself, show how long a benefit lasts after stopping. Look for the dates and number assessed at every follow-up point, as well as adverse-event collection. CONSORT asks for these details because timing and available data shape interpretation. [2]

Common questions

Does appearing in PubMed mean a study is endorsed?

No. Treat a database record as a route to the publication, not a quality verdict. Read the methods and result-specific limitations. Formal bias assessment requires supported judgments across relevant domains; this article’s worksheet does not replace that process. [3]

Can a study be informative when its primary result is not significant?

Yes. It can describe uncertainty, secondary findings and questions for future trials. A non-significant result does not prove that the treatments are equivalent, and a significant secondary result does not make the primary result positive. Interpret the estimates and planned analyses together. [6] [8]

What if I can access only the abstract?

Record the population, intervention, comparison and outcome statements it actually contains. Leave analysis denominators, missing-data methods, sponsor roles and unreported harms unresolved. Request the full paper or protocol before making a detailed appraisal; do not fill the gaps with assumptions.

Is one rigorous trial enough to settle the claim?

Read it alongside other relevant research. Replication, conflicting findings and differences between protocols affect how confidently the claim can be stated. Counting papers is less useful than checking their relevance and methods. [1]

Finish with a bounded conclusion

Use this sentence to summarize your reading: In this population, this intervention showed this result versus this control at this time; these methods or applicability questions remain unresolved. If the headline drops those limits, return to the evidence before accepting it.