Verification closure: from requirements to evidence

Ask what has actually been demonstrated

A passing regression is encouraging. A high coverage number is useful. Neither is sufficient on its own to establish that a design meets its requirements. A test can pass because its checker never observed the intended event. Coverage can be high because the model omitted a difficult behaviour. A formal property can hold because an assumption excluded the failure.

I treat verification closure as a structured argument. Each important requirement should connect to a verification method, an observable outcome and a reviewed result. The purpose is to show why the evidence supports the intended use of the design, including its error and recovery paths.

This changes the review question. Instead of “Have we reached the target?”, ask “What remains untested, what have we assumed, and what would make this evidence misleading?” Those questions are especially valuable near a deadline, when a single aggregate number can conceal very different risks.

Use complementary methods

Simulation is effective for exercising transactions, software interaction and long behavioural sequences. Constrained-random stimulus explores combinations that directed tests may overlook. Assertions check temporal rules close to the interface or state machine. Formal verification can exhaustively explore a bounded model or prove properties under stated assumptions. Structural analysis addresses classes of problems such as clock-domain crossings that ordinary simulation may not reveal.

These methods overlap, but they are not interchangeable. Code coverage describes activity in the implementation. Functional coverage measures scenarios defined by the verification plan. Assertion coverage can help show whether important antecedents occurred. None automatically proves that the requirements were complete.

Spend effort on boundaries and transitions

Normal operation is only one region of the state space. Interfaces become difficult during simultaneous events: a queue fills as a consumer stalls; a configuration update arrives during a calculation; reset interrupts a response; an error is detected while an interrupt is being cleared.

For a packet pipeline, I would explicitly examine malformed length, partial final beats, back-to-back frames, queue overflow, downstream backpressure and reset with a frame in flight. For an arithmetic block, the equivalent boundaries include minimum and maximum values, signedness, rounding, saturation, illegal modes and changes in operand validity.

The verification environment also needs verification. Deliberately corrupt an expected result and confirm that the scoreboard fails. Suppress an output and check the timeout. Drive an illegal sequence and confirm that the monitor reports it. A checker that is never challenged can create a convincing but empty pass signal.

Reusable verification frameworks help organise this work. Their value comes from consistent transaction models, monitors, scoreboards and configuration, rather than from the framework name itself. The environment should remain understandable enough for another engineer to explain why a failure was detected.

Make the remaining risk reviewable

Closure should combine requirement coverage, defect status, stability trends and integration evidence. Open defects need consequences, owners and an explicit disposition. Exclusions need a reason tied to unreachable behaviour, unsupported configurations or another valid boundary. A waiver should explain the engineering decision; it should not serve as a mechanism for making the dashboard green.

Track change as well as totals. A late modification to reset logic can invalidate previously reassuring results. A sudden drop in failures can reflect a disabled test. A long regression with no new bugs is useful only if the stimulus still exercises the relevant behaviours and the checkers remain active.

Keep the final evidence reproducible: source revision, tool version, build parameters, test seeds, constraints, coverage configuration and known limitations. A later team should be able to reconstruct what was tested without relying on the memory of the original engineer.

The decision to release a design is ultimately an engineering judgement. Good verification makes that judgement specific: these requirements have been demonstrated by these methods, under these assumptions, with these remaining issues. That is far more useful than presenting a percentage as if it were a guarantee.

Further reading

• Accellera — Universal Verification Methodology
• OpenTitan — Signoff checklist