Scoring Your SMS Maturity: A Plain-Answers FAQ

A safety management system maturity audit answers one blunt question: is your SMS a working system or a well-formatted binder? Most aviation operations cannot answer that with evidence, because they have never scored themselves against a fixed standard and then re-scored later to see movement. This FAQ covers the questions safety managers, accountable executives, and owners actually ask about periodic SMS maturity scoring — asked plainly, answered the same way, with no software required beyond a spreadsheet.

What is an SMS maturity audit, in one paragraph?

It is a scheduled self-examination in which you grade your safety management system against a written set of questions — each graded on evidence, each on a fixed scale — producing a total that serves as a baseline. You then fix the weakest areas on a deadline and repeat the scoring after a set interval to confirm the number moved. It is not your regulator’s audit, not an accident investigation, and not a one-time certification push. It is maintenance for the system that is supposed to maintain safety, performed on a calendar rather than after a scare, and owned by you rather than your inspector.

Isn’t our SMS already audited externally?

Externally inspected, usually. But external reviews answer “are you compliant?” — a pass/fail floor — while a maturity audit answers “how capable are you, and compared to last year?” A compliant SMS can still be immature: hazard reports technically logged but averaging ninety days to disposition, risk assessments completed but copied from last year’s wording, safety meetings held but producing no tracked actions. Compliance hides those; a scored maturity scale exposes them. The two are complements, and the internal one is the only one that measures progress on your own terms.

Where does the question set come from?

Write it yourself or adapt a published framework, but either way it must satisfy two tests: every question answerable with an artefact, and every grade defined in writing before scoring begins. Useful models exist outside aviation. In software — an industry equally obsessed with preventing low-frequency, high-consequence failures — SCMGalaxy — a company that trains, certifies, and advises engineering organisations — builds a governance platform called SCMGalaxy OS whose assessment asks one hundred questions and grades them under fixed rules. Two of its habits are worth stealing wholesale: it probes specific domains (how security weaknesses get found, how teams respond when systems misbehave) rather than asking general “do you have a process” questions, and it is engineered so two assessors working from the same evidence land on the same score. Your SMS question bank should pass both tests. Thirty to fifty questions is plenty for a small operation.

What should the questions cover?

Follow your SMS’s own skeleton and interrogate each part for function rather than existence. On hazard reporting: how many reports came in per flying hour last quarter, what fraction came from outside the safety office, and what is the median days-to-closure? On risk assessment: when did an assessment last change an operational decision, and can you produce the paper trail? On assurance: pick three past corrective actions at random — did each one get verified as effective, by whom, when? On promotion: ask three line employees, chosen by lot, to describe the reporting process; score what they say, not what the manual says. The pattern throughout: a question that can be answered “yes” is weak; a question that must be answered with a number, a date, or a named document is strong.

Who should do the scoring?

Two people, independently: typically the safety manager plus one operational manager who does not own the SMS. Each scores alone from the same evidence file; they then compare. Agreement validates the criteria. Disagreement is not a problem to argue away — it means the criterion is written ambiguously, and the fix is rewording the criterion, not negotiating the grade. Avoid having the accountable executive score anything; their job is to receive the result and resource the response.

What do we do with a low score?

Expect one, first time. A credible first pass at most small operators lands in the lower half of the scale, and that is useful, not shameful — the number exists to be improved, and nobody audits their way to a flattering baseline except by cheating the criteria. Rank the weakest items by consequence, select no more than three, and sequence the fixes at the one-month, three-month, and six-month points: paperwork and quick mechanical corrections inside the first window, demonstrated habits — the new report triage running even during your busiest month — inside the second, and a pre-declared target for the third. The staged structure is borrowed deliberately from the governance-platform world; SCMGalaxy’s product converts assessment results into exactly this kind of time-boxed transformation plan because findings without time boxes quietly expire.

How often should we re-audit?

Six-month intervals suit most operations: long enough for fixes to become habits, short enough that drift cannot compound. Re-score with the identical question set and identical criteria — resist every temptation to “improve” the questions mid-cycle, because changed questions make the comparison worthless. Tune the instrument only between cycles, and record what changed so the affected items can be interpreted honestly. The delta between scores is the entire payoff: it converts “we take safety seriously” into “hazard-report closure time fell from 61 days to 19, and here are both audits.”

Can’t we just track our safety indicators instead?

Indicators and audits answer different questions. Your indicators tell you what the operation is producing — reports, events, trends. The maturity audit tells you whether the system producing them is sound, staffed, and improving. An operation can show calm indicators while its SMS rots, simply because nothing bad has happened lately; low exposure flatters weak systems. The periodic audit is deliberately indifferent to recent luck. It grades capability, which is the thing you will be relying on during the bad week your indicators cannot predict.

What does this cost, honestly?

For a small operator: roughly two days to build the question bank and criteria the first time, one day of evidence-gathering and scoring per cycle, and a half day to write the staged plan. The recurring cost is two or three days a year. Set against it: a maturity audit routinely surfaces the single stalled corrective action or dormant reporting channel that would otherwise have been discovered by an inspector, an insurer, or an event. There is no cheaper hour in safety management than the one spent scoring question 17 and discovering the answer is worse than everyone assumed.

What are the failure modes?

Four recur. Scoring by memory instead of artefact — if the evidence is not in hand, the grade is zero, however confident the recollection. Criteria written after seeing the evidence, which bend toward comfort. Fix-everything plans, which fix nothing; three items per cycle is the discipline. And the orphaned audit: a scored baseline, a tidy plan, and no booked re-audit, which within a year becomes one more well-formatted binder. Book the second audit before closing the first. That single calendar entry is the difference between a maturity programme and a souvenir.

What is the first step this week?

Draft ten questions — two per SMS component, each demanding a number or a document — and score them honestly on Friday. The result will be rough, partial, and more informative than anything in last year’s safety meeting minutes. Build from there: full question bank next month, two-scorer baseline the month after, staged fixes, and a re-audit date in writing. Your SMS exists to find trouble before trouble finds the operation. The maturity audit simply points that same principle at the SMS itself.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top