775,260 people hold a “Professional Scrum Master I” certificate (grade E, vendor counter, as of October 2026). There is not a single independent study that tests whether any one of these certificates has ever made a project better.
That is the pattern.
What I examined#
I went through five areas:
- Agile and project management certificates,
- IT and security certificates,
- Organisational certificates such as ISO 9001 and ISO 27001,
- Licences for drivers, truck drivers and pilots, and
- Certification of physicians.
The question: Is there a measured outcome that improved because of the certificate?
- Accident rate?
- Defect rate?
- Patient outcome
- Project success.
What does not count:
- Salary
- Satisfaction
- Pass rate.
Every number carries its evidence grade:
- A official data,
- B independent research,
- C open measurement series by an association,
- D survey,
- E statement by someone who earns from it.
It is the same grid as in “What really helps?". Grade E numbers are not evidence. They are claims with an interest.
The full report with all sources is attached as a PDF.
Where it works: three mechanisms, no course#
There are exactly three constellations in which certification has demonstrably improved an outcome.
First: a minimum threshold in a dangerous field where no standard existed before. The midwife licences of the US states lowered maternal mortality by 6 to 7 percent between 1900 and 1940 (B, old). The condition is unambiguous: high danger and no threshold before. Whoever installs a floor catches the worst. In the middle of the field nothing changes.
Second: supervised practice plus restricted access. Graduated driver licensing in the US: 15 to 26 percent fewer crashes among 16-year-olds (A/B, data to 2009). Accompanied driving from 17 in Germany: 17 to 22 percent fewer crash involvements in the first year (A/B, 2007 to 2011). Sweden, practice age lowered to 16: around 15 percent fewer crashes per kilometre driven (B, 1990s). The effective component is always the hour behind the wheel with someone sitting next to you. The instruction itself is not: three randomised studies of formal driver education yield a risk ratio of 1.01 to 1.10 (B, old). Skid training even increased crash numbers, for ambulances by around 45 percent (C, old).
Third: an exam whose score predicts a real practice criterion. Among Canadian physicians the risk of a substantiated patient complaint rises by 38 to 51 percent per two standard deviations of worse exam performance (B, 1993 to 2005). In the US, disciplinary actions fall by around 25 percent per standard deviation of better exam result (E with open methodology, 1994 to 2017). The instrument measures something. But: it is the score that predicts. Not the certificate. “Passed” throws away exactly the information that carries the value.
Where it was measured, with a result of zero#
The second group is more uncomfortable. Here somebody measured.
Recertification of American internists: a quasi-experiment with 1,930 physicians and 154,045 patients finds no difference in avoidable hospital admissions (E, the certifier measured it itself, 1999 to 2005). An independent study of 68,213 patients finds no difference in ten quality measures (B, 2012 to 2013). The only randomised trial missed its endpoint. Cost of the programme: 5.7 billion dollars in ten years (B, model calculation 2015).
The British revalidation for physicians: 281,000 physicians, 85,000 respondents, no evidence of better patient safety (B/D, 2015 to 2017). The regulator paid for the evaluation itself.
The EU periodic training requirement for professional drivers, 35 hours every five years: no evaluation with accident figures, in any member state. The British government wrote in 2013 that proof of effect would be “very difficult”. Of 748 respondents, 349 saw no safety benefit (D, 2013). The European Commission calls the expected effect “quite limited” in its own impact assessment (E, 2017).
Stricter dental licences in the US: no better dental health, prices 14 to 16 percent higher (B, old).
Teacher certificates: a difference of at most two to three hundredths of a standard deviation in learning outcomes (B, 1998 to 2005).
Emergency medical services: patients attended by the more highly qualified team survived less often, 9.2 versus 13.1 percent (B, 2009 to 2011). More qualification meant more interventions meant more time.
Where nobody ever measured and the most advertising is done#
Now the class most often demanded in mid-sized companies: Scrum Master, SAFe, PMP, PRINCE2, ITIL, ISTQB, IREB, CISSP, CCNA, cloud certificates.
For not one certificate does an independent study exist that measures the certificate against an objective outcome. No vendor publishes pass rates. No vendor publishes whether its exam predicts anything. No vendor publishes whether two examiners would produce the same result. (Rolling dice might have an equally good hit rate.)
What exists is a review of 24 studies on project management certificates (B, 2004 to 2021): 68 percent surveys, a single one with real project metrics, five studies positive, six with no relationship. The largest data set on agile teams, 4,013 teams, did not even collect the certificate as a variable; the chosen scaling framework explains one to two percent of team performance (B/D, 2021 to 2023).
What else exists is salary figures. The project management association reported a salary premium of 33 percent for its certificate in 2023, 17 percent in 2025 (E, interest: own product). The change in method is not explained. An exam service provider reports that 79 percent of the certified felt better (E, interest: sells the exam).
And what demand shows: 52 percent of recruiters consider the PMP important. Among line managers it is 15 percent (B, US, old). The certificate is demanded by the filter, not by the work.
If a vendor tells you its certificate is “the proof”, ask for three numbers: pass rate, predictive power, examiner agreement. You will not get any.
The certification economy — or — let’s make money#
Now to the question of who earns from what. The chain is always the same.
The standard’s author sells the document. The standard you are to be certified against is a paid product. The trainer sells the course on the document. The certifier sells the exam on the course. The accreditor sells the certifier the permission to examine. And at the end, the examinee pays the examiner.
This last point is the decisive one. The examinee chooses and pays the one who examines him. What that produces is documented:
In medical devices, a newspaper sent a fictitious hip implant to 14 certification bodies in 2012. The bodies competed on price, speed and promised pass rate (C, investigative, old). Devices first approved in Europe through these bodies had a 2.9 times higher risk of recalls and safety warnings than devices that went through the American authority first (B, 2005 to 2016). After the follow-up audits, four of twelve bodies lost their designation (C, 2013).
In the Wirecard case, the supervisory authority’s verdict against the auditor was: two years’ ban on mandates, 500,000 euros, seven of twelve auditors handed back their licences before they could be withdrawn (A, 2023).
For ISO 9001, research shows selection instead of effect:
- The certified firms were already better before certification; afterwards they did not get better (B, Spain, Denmark, US, old).
- If a firm loses the certificate, there is no measurable financial consequence (B, 143 firms, old).
No study measures ISO 9001 against defects, recalls or complaints. Auditors arrive with the result already in their pocket, as an interview study documents (B, old). In process assessments in the automotive sector, the practices that score best are those easily checked on paper: documentation and linking. Worst: whether the test criteria are right and what a failure causes (B, 34 assessments, 2012 to 2019). What gets checked is what is checkable, not what works.
And Germany? With 33 percent, Germany has the highest share of licensed employees in the EU (B/D, 2015). Sweden and Denmark are at 14 to 15 percent, the United Kingdom at 19.5.
Germany evaluates not one of these certificates for outcome! Not the master craftsman requirement: its reinstatement in 2020 rests on register figures from the crafts association, not on a defect statistic; the Federal Monopolies Commission states explicitly that no expert report proves a loss of quality (A, legislative process 2019). Not the specialist physician. Not the continuing education points, which in Bavaria in 2008 only 23 percent of physicians had completed (A, old). Not compulsory driving school. Not the professional driver training. The federal accreditation body reports procedural figures, but no withdrawal statistics and no effect statistics (A, Bundestag printed paper 2024).
What the licensing economy measures overall: wages rise by around 15 percent, employment falls by 29 percent, the welfare loss is 12 percent of the occupation’s total surplus (B, structural model, US 2015 to 2018). Quality: no change.
That is the economy. It produces documents, courses, exams and accreditations. It produces no measured benefit for the buyer. And it becomes densest where the least is measured.
What a certificate would have to be able to do#
From the evidence, not from opinion:
- The exam measures the same thing that counts in practice. The medical exam asks about clinical decisions, and clinical decisions produce complaints. A multiple-choice test on a framework measures whether someone has read the framework.
- The score is validated against a register. Complaints, accidents, defects. Without this validation nobody knows whether the exam predicts anything.
- The score stays visible. “Passed” destroys the information.
- Two examiners reach the same result. The agreement is measured and published.
- The pass rate is published. An exam that almost everyone passes separates nothing.
- Supervised practice is part of the certification, not course plus test.
- The examiner is not chosen and paid by the examinee.
- Recertification hangs on performance data, not on collected points.
No agile, IT or management certificate meets even one of these criteria.
What you can do tomorrow#
Here the diagnosis ends. Now the measure, depending on who you are.
You hire. Strike the certificate from the job advertisement. Instead, demand a work sample and conduct a structured interview. The predictive validity for job performance is 0.42 for the structured interview, 0.40 for the job knowledge test, 0.33 for the work sample, 0.07 for years of experience (B, meta-analysis 2022). Certificates are not even listed separately in this ranking.
The test proposition: if the certificate only measures document knowledge, then a work sample instead of a certificate requirement will reduce your bad hires.
Measure: terminations during probation and rework in the first six months, compared with your last ten hires.
Deadline: twelve months.
Stop criterion: if nothing changes as a result, the certificate was not the problem.
You buy and audit suppliers. Stop reading ISO 9001 as proof of quality. It is proof that someone paid for an audit. Demand the defect rate of the last twelve months, the complaint rate and the response time on deviations. These are numbers the supplier collects itself and against which you can compare it. A supplier who cannot name them on the spot has no quality management, whatever hangs on the wall.
You certify or train. The evidence lies with you, not with me. So no accusation, but a test proposal. Documented deviation: no predictive study exists for your certificate. Possible causes, estimated: the exam measures document knowledge; or it measures competence, but nobody has checked. Test step: take 500 people certified in the last three years, their score and an outcome measure from their employers, such as project terminations or defect rate. Discriminating power: if the score correlates with the outcome, the exam measures competence and you should report the score instead of “passed”. If it does not correlate, the exam measures document knowledge and your certificate is a course attendance record. Deadline: 18 months. Stop criterion: if you find no employers who deliver outcome data, then your certificate has no addressee who needs it for the work. That would be a result.
You make laws or regulations. No certificate requirement without a predefined outcome measure and evaluation date. The EU decided on a training requirement for truck drivers in 2003 and to this day has not collected a single accident figure for it. The crafts sector brought twelve trades back under the master craftsman requirement in 2020 without a defect statistic. Both could have been prevented with one line in the law: “The effect will be measured on date X by measure Y. If the proof is missing, the requirement expires.”
The price of not acting#
You keep paying for courses, exams and audits. You keep receiving documents. You keep not finding out whether your people or your suppliers can do anything. The density of certificates rises, the measurement stays absent, and the one industry where somebody re-measured shows a 2.9 times higher recall risk.