Testing: earn confidence, don't assume it

A test is evidence for a decision, not a ritual. This page sets out what four practitioners teach about testing, how we settle the places where they pull in different directions, 160 standards to adopt from, a release gate, and what changes when an AI writes the code or is the code.

Labels matter here. Paragraphs that describe an author's view say so and cite a source (S1 to S33, listed at the end). Paragraphs marked My interpretation or synthesis are our own readings and adoption rules, not claims the authors made or agree on.

1. Four views

Kent Beck: confidence is an outcome, not a ritual

Beck's Test Desiderata describes twelve qualities: isolation, composability, determinism, speed, writability, readability, sensitivity to behavior, insensitivity to structure, automation, specificity, predictive usefulness, and confidence. These qualities can conflict; the goal is a useful balance. A realistic integrated check may be slower but reveal a failure a tiny test cannot. S1

His later account of TDD names ease, reliability, sustainability, and responsibility as desired outcomes. He also specifies prerequisites: reasonably predictable results and cases, useful composition from small checks into larger confidence, and manageable test costs. He explicitly permits a different workflow when those conditions fail. S2

My interpretation: Preserve the feedback mechanism and the ability to change code safely. Do not mistake adherence to the sequence for evidence that the whole product works.

Michael Feathers: discover what exists before deciding what to preserve

Feathers describes characterization testing as documenting actual behavior. An initially guessed expectation can be replaced with the observed result, because the immediate goal is learning and detecting later changes. That is different from establishing that the behavior meets a requirement. He recounts users depending on behavior a developer considered a bug. S3

His seams concept identifies substitution points that let a test control troublesome dependencies. His discussion of sprouting describes adding independently testable behavior beside difficult existing code. S4

My interpretation: Separate three questions: What happens now? What must remain compatible? What should change? A characterization test answers the first and helps with the second; it does not decide the third.

Feathers also warns that AI-generated tests can approve unintended behavior in AI-generated implementation. His proposed response is to express constraints as executable tests before generation. Passing those tests still depends on whether the constraints are adequate. S31

Gerard Meszaros: tests have architecture and maintenance costs

Meszaros treats difficult test code as a potential productivity bottleneck. His worked refactoring example progressively exposes intent and simplifies the test rather than accepting a complicated fixture and verification block as inevitable. His book organizes both test patterns and the smells that indicate trouble. S5 S6

His chapter on doubles is explicitly conditional: substitutes help when dependencies are unavailable, uncontrollable, slow, or produce unwanted effects. But he warns against replacing the behavior being verified and recommends checking important behavior without a double. He also identifies excessive specification through doubles as a source of fragile tests. S7

My interpretation: A test suite is a product used by engineers. Its users need understandable failures, reliable execution, and affordable adaptation. Treat brittleness as a defect in that product.

Glenford Myers: the tester must resist reassuring answers

Myers's psychology chapter challenges definitions of testing centered on demonstrating success or absence of errors. His principles chapter explains why an expected result must be specified: without one, a plausible output can be accepted simply because the observer wants success. S8 S9

My interpretation: Approach a feature with a concrete theory of how it could be wrong. A test discovering a real problem has delivered value. This complements Beck: challenge the system to earn confidence rather than treating confidence as the starting assumption.

The accessible book previews are partial. This profile does not pretend to reproduce or newly verify every guideline in the complete book. Later editions have coauthors; the source register identifies the edition used.

2. Resolving the apparent disagreements

These are recommended decisions, not claims that all four authors agree.

TensionRecommended resolution
Confidence versus error discoverySeek disconfirming evidence, then calibrate confidence to what survived.
Correctness tests versus characterizationLabel intended behavior and observed legacy behavior separately.
Test first versus test afterPrefer test first where outcomes are knowable and costs reasonable; obtain other evidence when they are not.
Isolation versus realismIsolate for control and diagnosis; integrate to test boundary assumptions.
Fast tests versus realistic environmentsKeep development feedback fast and use scheduled or release-stage checks for expensive risks.
Public API versus internal seamPrefer stable behavior boundaries, but introduce a seam when it makes otherwise inaccessible risks testable.
One assertion versus manyOne coherent purpose matters more than an assertion count.
Mocks versus real collaboratorsUse a substitute to control a dependency; test the relevant real contract elsewhere.
Coverage versus confidenceCoverage exposes omissions in execution. It does not establish the quality of the oracle.
Determinism versus randomnessGenerate varied cases while retaining reproducible inputs and traces; separately characterize unavoidable variability.
Automation versus explorationAutomate repeatable checks and preserve human investigation for new risks and questions.
Universal rules versus economicsMake the evidence proportional to the consequence, uncertainty, and cost of failure.

Fowler distinguishes classical and mockist approaches rather than presenting one universal arrangement. Rapid Software Testing emphasizes exploration and critical evaluation beyond automated checking. These are useful counterweights to a purely mechanical doctrine. S10 S32

3. How to use the standards

Select the sections relevant to a component or system. For each applicable rule, record covered, gap, or accepted exception, with evidence and an owner. Do not score an inapplicable hardware requirement against a simple web utility.

Treat the wording as an adoption starting point. If a domain has legally required or safety-specific standards, those need a separate applicability assessment; this reference does not certify conformance.

Jump to: A. Purpose · B. Oracles · C. Readability · D. Isolation · E. TDD · F. Doubles · G. Inputs · H. Properties · I. Legacy · J. Persistence · K. Distributed · L. Load · M. Security · N. Operations · O. Whole systems · P. Governance and AI

4. The 160 standards

A. Purpose, risk, and evidence

Foundations: S2, S8, S9, S11, S12. These are synthesized adoption rules.

  1. Define the decision a test informs. A result should help someone accept, reject, investigate, or change something.
  2. Rank testing by failure impact, exposure, and uncertainty. A rare catastrophic failure may deserve more attention than a frequent cosmetic defect.
  3. Write down critical invariants: conditions that must hold across operations, failures, retries, and recovery.
  4. Separate verification of requirements from validation of user needs. Correct implementation of the wrong requirement is still a product failure.
  5. State what passing actually establishes, including the tested inputs, environment, assumptions, and observation window.
  6. Treat passing tests as bounded evidence. Do not present a finite suite as proof of unrestricted correctness.
  7. Challenge the specification itself. Conflicting examples, ambiguous units, and impossible promises are defects worth discovering before implementation.
  8. Use diverse evidence where consequences justify it: tests, review, static analysis, models, operational observations, and user evaluation.
  9. Define release criteria before seeing results; record exceptions with an owner, rationale, and resolution date.
  10. Scale rigor to the system. Apply relevant standards deliberately rather than imposing every technique on every small change.

B. Expected results and test oracles

Foundations: S8, S9, S18, S19. An oracle is the basis for judging a result.

  1. Establish expected behavior independently of the implementation whenever testing correctness.
  2. Do not compute expected values by calling the same production algorithm being checked.
  3. Use independently calculated examples for business rules, especially money, rounding, units, thresholds, and dates.
  4. Assert meaningful results and state changes. A successful status code alone rarely establishes business success.
  5. Check prohibited side effects: no unauthorized write, duplicate delivery, unintended charge, or data disclosure.
  6. Distinguish observed historical behavior from approved intended behavior in test names or metadata.
  7. Use explicit tolerances for approximate results, justified by the domain and measurement uncertainty.
  8. Use metamorphic relationships when exact answers are difficult: appropriate transformations should preserve or predictably change results.
  9. Use differential testing against another implementation where valuable, while recognizing shared bugs and disagreements need independent adjudication.
  10. Review large snapshots as specifications. Updating a baseline requires an explanation of why the changed behavior is acceptable.

C. Readable and diagnostic tests

Foundations: S1, S5, S6.

  1. Name the scenario and outcome so failures explain the broken promise.
  2. Make setup, action, verification, and cleanup distinguishable.
  3. Give each test one coherent behavioral purpose; use several assertions when they jointly express that purpose.
  4. Keep relevant input values visible. Hide incidental setup behind small, plainly named helpers.
  5. Keep assertions simpler than the production logic they judge.
  6. Avoid branching that silently skips assertions. Generated and table-driven cases should report which case failed.
  7. Use the smallest fixture that exposes the behavior, while retaining realistic relationships when those relationships matter.
  8. Report expected and actual values, identifiers, and useful context without leaking secrets.
  9. Keep cleanup reliable when the action or assertion fails.
  10. Refactor test code as it becomes difficult to understand; preserve the fault-detection behavior during the refactor.

D. Isolation, reproducibility, and time

Foundations: S1, S6, S17.

  1. Make independently runnable tests independent of execution order.
  2. Give parallel tests separate mutable resources, identifiers, files, queues, and database records.
  3. Control clocks, random generators, environment variables, and external dependencies when repeatability requires it.
  4. Record random seeds and generated inputs; preserve schedules or traces when seeds alone cannot reproduce concurrency.
  5. Treat flakes as unresolved information about the test, product, or environment.
  6. Do not silently convert a failure into success by retrying until green. Preserve attempts and classify the cause.
  7. Use observable conditions or synchronization rather than arbitrary sleeps for asynchronous work.
  8. Bound waits with meaningful timeouts and report what condition failed to occur.
  9. Test real time-sensitive behavior separately: expiry, clock skew, daylight-saving transitions, and calendar boundaries where relevant.
  10. Record build, dependency, configuration, and environment identity sufficiently to repeat consequential results.

E. Feedback, architecture, and TDD

Foundations: S1, S2, S3, S10.

  1. Use short feedback loops for development and schedule expensive evidence at an appropriate cadence.
  2. With TDD, observe the intended failure before implementing the behavior; compilation or setup failure is not the same evidence.
  3. Implement enough to satisfy the current behavior, then improve structure while preserving that behavior.
  4. Run relevant regression tests after structural change rather than relying on apparent code similarity.
  5. Use difficulty writing tests as a prompt to examine coupling, hidden state, and mixed responsibilities.
  6. Provide control and observation points for important behavior without building elaborate abstractions solely to satisfy a framework.
  7. Test at the smallest scope that can expose the actual failure mode; widen scope when interactions are the risk.
  8. Retain representative integrated journeys even when component tests are excellent.
  9. Treat a test pyramid or similar shape as a heuristic, not a mandatory numerical ratio.
  10. Use other workflows when TDD is a poor fit, and explicitly obtain the needed confidence through them.

F. Doubles, interfaces, and integration

Foundations: S7, S10.

  1. Distinguish stubs, fakes, spies, and mocks; choose the capability the test actually needs.
  2. Never substitute away the behavior the test claims to verify.
  3. Prefer observable outcomes over private call sequences unless the interaction itself is a required contract.
  4. Assert call counts when they matter, such as avoiding repeated external side effects; do not impose them gratuitously.
  5. Check important assumptions made by doubles against real dependency behavior.
  6. Exercise the real database engine for database-specific semantics such as constraints, locking, and transaction isolation.
  7. Test serialization across real boundaries: field presence, nullability, numeric precision, encoding, and version differences.
  8. Verify provider-consumer contracts for both successful responses and relevant failures.
  9. Test timeouts, malformed responses, refusal, and partial responses instead of assuming external calls only succeed.
  10. Separate controlled integration tests from live service probes so a provider outage does not masquerade as a local unit defect.

G. Input partitions, boundaries, and combinations

Foundations: S13, S18, S33. The concrete case catalog is synthesis.

  1. Partition inputs by meaningful behavior before selecting examples.
  2. Test immediately below, at, and above important thresholds.
  3. Distinguish absent, null, empty, blank, zero, and false when the interface gives them different meanings.
  4. Test minimum and maximum sizes, including oversized input and empty collections.
  5. Include invalid values, malformed structure, and contradictory combinations.
  6. Exercise character encoding, Unicode, normalization, locale, and user-visible formatting where supported.
  7. Test duplicate values, unordered data, ties, and ambiguous matches.
  8. Use decision tables when combinations of rules determine outcomes.
  9. Use combinatorial coverage to explore interacting parameters; pairwise coverage is not a guarantee against higher-order faults.
  10. For generated tests, inspect the distribution and rejected cases so apparent volume does not conceal missing scenarios.

H. Properties, state, and sequences

Foundations: S18, S19, S16.

  1. Complement handpicked examples with properties that hold over a defined input domain.
  2. State property preconditions explicitly; do not accidentally test a mathematical law outside its valid domain.
  3. Generate valid complex structures directly where filtering would discard almost everything.
  4. Preserve and minimize useful counterexamples into understandable regression cases.
  5. Test permitted and forbidden state transitions, including transitions from terminal states.
  6. Test sequences of operations when history changes meaning: create, update, cancel, retry, recover, delete.
  7. Use a simpler reference model for complicated workflows and compare observable histories.
  8. Check invariants during intermediate states where the contract requires them, not only at the final result.
  9. Check both safety and progress: forbidden outcomes remain impossible within the tested model, and work finishes under specified assumptions.
  10. Exercise relevant interleavings explicitly; many concurrent workers alone do not guarantee the critical race was tested.

I. Legacy code, refactoring, and replacement

Foundations: S3, S4, S21.

  1. Capture relevant existing behavior before changing unfamiliar code.
  2. Identify seams where dependencies can be controlled with minimal disturbance.
  3. Make the smallest enabling change needed to establish a useful test harness.
  4. Treat surprising historical behavior as a discovery requiring a decision, not an automatic bug fix.
  5. Separate preservation tests from new-behavior tests so intentional changes are reviewable.
  6. Compare old and new implementations on representative and adversarial cases during substantial replacements.
  7. Canonicalize irrelevant output noise without removing differences that matter to users.
  8. Keep pure refactoring separate from behavior changes where practical.
  9. Prioritize coverage around the change and its effects instead of delaying all progress for complete legacy coverage.
  10. Retire temporary characterization checks only when their protection is replaced or the behavior is explicitly abandoned.

J. Persistence, transactions, and migrations

Foundations: S15, S16, S22. The database scenarios below are synthesis.

  1. Test that important writes survive the relevant restart or failure according to the documented durability promise.
  2. Verify atomic operations do not expose or retain prohibited partial changes.
  3. Exercise database constraints directly, including through paths that bypass application validation.
  4. Test competing writes and lost-update risks using actual transaction behavior.
  5. Test migration of representative old data, including historically malformed or incomplete records.
  6. Test coexistence of old and new application versions when rolling deployment permits it.
  7. Interrupt migrations and backfills; verify the documented restart, resume, or recovery procedure.
  8. Reconcile records across boundaries using stable identities and domain totals, not merely matching row counts.
  9. Restore backups and validate the restored application's behavior and data integrity.
  10. Measure achievable recovery time and data loss against explicit recovery objectives.

K. Distributed systems, queues, and retries

Foundations: S14, S16, S17, S22.

  1. Write down delivery, ordering, consistency, and durability guarantees before testing them.
  2. Test duplicate, delayed, reordered, and missing messages as allowed by the transport's failure model.
  3. Test idempotency for repeated requests, simultaneous duplicates, and retries after process restart.
  4. Scope idempotency keys correctly and test conflicting payloads, retention, and reuse behavior.
  5. Distinguish a confirmed failure from an unknown outcome after timeout or connection loss.
  6. Crash processes at critical boundaries, such as after an external effect but before local acknowledgment.
  7. Inject network partitions and verify both behavior during isolation and reconciliation afterward.
  8. Verify retry limits, backoff, and aggregate retry amplification under dependency failure.
  9. Test poison messages, dead-letter handling, replay, and backlog draining without repeating forbidden effects.
  10. Validate observable operation histories against the consistency contract; eventual final agreement can hide earlier violations.

L. Load, performance, and resource limits

Foundations: S12, S14, S17. Specific measurement rules are synthesis.

  1. Define workload, data size, concurrency, and resource budget before interpreting a benchmark.
  2. Measure user-relevant latency distributions, throughput, errors, and resource consumption together.
  3. Ensure the load generator is not the bottleneck or accidentally hiding stalls by reducing offered traffic.
  4. Exercise steady load, bursts, sustained saturation, and recovery after overload.
  5. Include realistic skew, hot keys, expensive requests, and uneven tenant demand.
  6. Test cold starts, warm caches, cache misses, and cache invalidation where performance depends on them.
  7. Use soak tests for leaks and accumulating state rather than assuming short benchmarks reveal them.
  8. Verify queue bounds, admission control, and rejection behavior when capacity is exhausted.
  9. Measure correctness under load; a fast system producing wrong answers has failed.
  10. Compare performance changes using repeated, controlled runs and record uncertainty rather than trusting one measurement.

M. Security, privacy, and abuse resistance

Foundations: S23, S24, S25. These are engineering checks, not a compliance certification.

  1. Derive security tests from assets, trust boundaries, privileges, and credible misuse scenarios.
  2. Test authorization across actor, action, resource, and tenant combinations, including explicit denials.
  3. Verify enforcement on the server or authoritative boundary, independently of interface restrictions.
  4. Exercise session expiry, revocation, credential changes, and replay behavior where applicable.
  5. Test business-rule abuse: skipped steps, repeated benefits, altered ownership, and concurrent attempts to exceed limits.
  6. Check untrusted input handling at parsers, interpreters, file operations, and outbound request boundaries.
  7. Verify secrets and sensitive records do not leak through logs, errors, exports, or diagnostic artifacts.
  8. Test privacy-sensitive retention and deletion behavior against the actual documented scope, including derived stores.
  9. Test security behavior during dependency failure and degraded operation; specify the appropriate fail-safe response.
  10. Use controlled accounts and environments; bound production probes so testing cannot create uncontrolled external effects.

N. Deployment, operations, and recovery

Foundations: S12, S15, S26. The operational checklist is synthesis.

  1. Test the deployable artifact and relevant runtime configuration, not only source code in a developer environment.
  2. Verify clean installation, upgrade, startup, readiness, shutdown, and restart paths.
  3. Test configuration changes and feature-flag combinations with the same care as code changes.
  4. Exercise rollback or roll-forward procedures, including compatibility with already-written data.
  5. Use staged rollout observations tied to user outcomes and predefined stopping criteria.
  6. Test that monitoring detects representative failures and routes actionable signals to the right place.
  7. Verify diagnostic information is sufficient to distinguish likely fault causes without exposing protected data.
  8. Rehearse dependency outages, expired credentials or certificates, storage exhaustion, and loss of critical infrastructure.
  9. Exercise recovery procedures with people and access conditions representative of a real incident.
  10. Turn consequential incidents into targeted regression protection and a review of why existing evidence missed the failure.

O. Whole systems, hardware, and human use

Foundations: S11, S26, S27, S28. Physical-system examples are synthesis.

  1. Trace important system requirements and hazards to appropriate verification evidence.
  2. Validate representative user missions from beginning to end in the intended operating context.
  3. Test hardware-software interfaces, physical units, ranges, timing, and calibration assumptions where applicable.
  4. Use hardware-in-the-loop or real hardware when simulation cannot establish important physical behavior.
  5. Document simulator assumptions and compare predictions with measured behavior.
  6. Test startup, shutdown, brownout, power loss, reconnection, and stale sensor data when relevant.
  7. Exercise environmental limits appropriate to the device, such as temperature, vibration, interference, and intermittent connectivity.
  8. Test operator mistakes, confusing feedback, manual intervention, and recovery from incomplete workflows.
  9. Combine automated accessibility checks with appropriate manual and user evaluation.
  10. For hazardous systems, base safe-state, interlock, emergency-stop, and recovery tests on the hazard analysis and applicable domain requirements.

P. Suite governance, AI, and honest reporting

Foundations: S20, S29, S30, S31, S32. AI-specific adoption rules extend those sources.

  1. Use coverage to find unexercised code, not as a certificate that assertions or requirements are adequate.
  2. Use mutation testing selectively to challenge assertions; inspect surviving mutants and recognize equivalent mutations.
  3. Assign ownership and repair expectations to slow, flaky, quarantined, and obsolete tests.
  4. Report passes, failures, skips, quarantine, infrastructure errors, and unexecuted scopes separately.
  5. Review changes to tests and expectations with the same seriousness as implementation changes.
  6. For AI-written code, derive acceptance criteria from requirements before allowing generated implementation to define success.
  7. Do not let the same generation process be the sole judge of both code and its correctness.
  8. Require explicit review before an agent weakens assertions, updates baselines, removes tests, or changes evaluation criteria.
  9. For nondeterministic AI systems, use representative held-out cases, repeated trials, task-level scoring, and explicit uncertainty.
  10. Evaluate agent actions and external effects as well as answers, including tool failures, unauthorized actions, cancellation, and bounded resource use.

5. A practical release gate

This gate is my synthesis; tighten or simplify it for the risk.

A trivial text adjustment should not trigger a disaster-recovery exercise. A change to charge submission, tenant authorization, or irreversible data transformation should receive much stronger evidence.

6. Worked applications

Payment submission

An illustrative scenario for a Rails service using an external gateway, not a claim about the current behavior or capabilities of a specific integration.

Requirement: An authorized payment attempt uses the correct merchant, amount, currency, and invoice; retries do not create prohibited duplicate financial effects.

RiskValuable testNecessary observation
Wrong merchant or invoiceCross-tenant and mismatched-identifier submissionsDenial before external submission; no local mutation
Client changes amountAltered client-supplied amountServer-authoritative amount or explicit validation
Two simultaneous submissionsSynchronize contenders at the critical boundaryExternal effects and local records remain within contract
Gateway accepts but response is lostSimulate an accepted request followed by transport failureUnknown outcome is represented; no blind duplicate submission
Process dies after chargeCrash before local success persistenceReconciliation reaches the correct state without prohibited replay
Duplicate or reordered webhookReplay event IDs and reorder event historiesNo duplicate effect or invalid state regression
Stale payment statusDelay a previous result until after a newer eventDocumented transition rules preserve the correct outcome
Receipt failure after chargeReject message deliveryPayment remains correctly recorded; notification retry is separate
UI success too earlyDelay server completionDisplayed outcome follows authoritative payment state
Sensitive diagnosticsTrigger errors with synthetic protected valuesLogs and error outputs exclude prohibited data

Critical limitation: Application tests alone cannot establish exactly-once charging across a remote gateway. The claim depends on the gateway's idempotency or lookup behavior, local concurrency control, durable identities, and reconciliation. Test the supported guarantee; do not invent a stronger one.

A useful mix is fast business-rule tests, database-backed transaction tests, controlled gateway-boundary tests, and a small number of representative end-to-end sandbox checks. Counts and ratios should follow risk.

Repair-order or event synchronization

Requirement: Repeated and out-of-order events converge to the correct record under the actual provider contract.

CaseWhat to test
Same event twiceNo duplicate line, notification, or downstream effect
Older event arrives laterExplicit version/order rules; do not blindly assume timestamps totally order events
Partial payloadDefined meaning of omitted fields versus explicit null or deletion
Full snapshot replaces stateCorrect removal and preservation rules
Provider unavailableBounded retries and visible lag
One malformed record in a batchDocumented isolation or failure policy; no silent loss
Worker restart mid-batchSafe restart and accurate checkpoint handling
Mapping changesTenant and external identifiers remain correctly associated
Replay after code upgradeCompatibility and deterministic transformation where promised
Apparent convergenceReconcile field-level meaning and invariants, not just record counts

AI-written code and AI systems

Two different problems need different evidence.

When AI writes conventional code: Human-approved requirements and independent examples define success. Tests derived only from generated implementation can reproduce the same misunderstanding. Feathers' executable-constraint approach directly addresses that risk. S31

When AI is part of the running system (my recommended extension): Exact output strings may be an inadequate oracle. Evaluate completed tasks, prohibited actions, uncertainty, latency, and cost over representative cases. Use deterministic checks for tool authorization and data invariants; use repeated evaluation where model behavior varies. Retain held-out cases and review grader errors.

Practical examples:

These are proposed evaluation standards, not a claim that any finite evaluation makes an autonomous system universally reliable.

7. What not to turn into dogma

SloganMore defensible standard
"100% coverage means tested."Ask what was asserted and which plausible faults survive.
"Every test must have one assertion."Keep one coherent purpose; assert its important consequences.
"Never touch the database."Isolate logic when useful; exercise actual persistence semantics when relevant.
"Mock every collaborator."Control dependencies deliberately and validate real boundary assumptions.
"Never mock anything."Use substitutes for fault control, speed, or otherwise inaccessible conditions.
"TDD proves correctness."TDD is a development feedback workflow with prerequisites and limits.
"All failures are product bugs."Diagnose product, harness, environment, and specification failures separately.
"A retry fixed the test."A later pass does not explain the earlier failure.
"Snapshots are specifications."A reviewed baseline may encode a decision; an automatically accepted one merely records output.
"End-to-end tests cover everything."They sample workflows and often localize faults poorly.
"The simulator passed, so the device is safe."Establish model fidelity and test physical assumptions where necessary.
"More tests always mean more confidence."Value depends on independence, relevance, oracle strength, and maintenance cost.

8. A minimal record for a consequential test

FieldRecord
Requirement or riskWhat could go wrong and why it matters
ScopeComponent, interface, workflow, deployment, or whole system
PreconditionsState, identities, configuration, environment
StimulusInputs, sequence, concurrency, or injected failure
OracleExpected outcomes and the independent basis for them
Forbidden effectsWhat must not happen
EvidenceActual observations, artifact identity, relevant logs
ReproductionCommand/procedure, seed, inputs, trace, environment
LimitsAssumptions and behavior not established
OwnerWho maintains the check and resolves failures

9. In AGENTS.md

Section 17 of AGENTS.md distills this page into rules every coding agent here follows: expected results that don't come from the code under test, the rules for AI-written code (standards 156 to 158), honest reporting, and the release gate above. Our default tools stay Minitest and Rails' own test setup (AGENTS.md section 5).

Research notes and source register

Research used author writing, first-person interviews, authorized publisher excerpts, research papers, and official engineering guidance. Some author-site pages were unavailable, so the Meszaros analysis relies on the publisher's sample chapters and contents. The Myers discussion is intentionally limited to accessible authorized previews rather than presenting the entire book as reviewed. Mutable web references were consulted for this report; historical case studies are not assessments of current product versions.

The named practitioners offer engineering judgment, not a single experimentally proven universal doctrine. The standards combine their documented concerns with explicit synthesis. The sources below support the stated foundations; the release gate and worked scenarios are original applications.


Provided as is, for information only, with no warranty and no liability. Not legal or professional advice. Security and radio material is for authorized, lawful use only. Full disclaimer.

Visits are logged on our own server: IP address, browser and device details, and pages viewed. We use them to understand traffic and spot attacks, and never share or sell them.