What good engineering looks like in 2026: evidence for a safe change

Keep old code running through the additive change

The executable source uses Python 3.12.12 with SQLite 3.51.2. It creates the original table, writes an old completed row, and adds a nullable finished_at column. The old writer names its columns explicitly and can still insert after the change. The old reader selects the existing state field and can read a new completion. The assertions exercise both directions against the changed database.

Download the source and tests below. From the extracted root, run python3 frontend/content-quality/evidence/change-evidence/check.py results.json with Docker available. It executes the exact published program in the digest-pinned Python image, without network access, and checks the three deliberate defects described below. The integer timestamps are synthetic seconds relative to a shared origin. The ten-second deadline is a demonstration choice, not an Ulties service promise.

import sqlite3
import sys

assert sys.version_info[:3] == (3, 12, 12), "runtime"
assert sqlite3.sqlite_version == "3.51.2", "sqlite"
db = sqlite3.connect(":memory:")
db.execute("CREATE TABLE exports (id TEXT PRIMARY KEY, account TEXT, "
           "accepted_at INTEGER, state TEXT)")


def old_write(job, account, accepted_at, state="queued"):
    db.execute("INSERT INTO exports (id, account, accepted_at, state) "
               "VALUES (?, ?, ?, ?)", (job, account, accepted_at, state))


def old_read(job):
    return db.execute("SELECT state FROM exports WHERE id = ?", (job,)).fetchone()[0]


old_write("legacy", "cedar", -10, "complete")
db.execute("ALTER TABLE exports ADD COLUMN finished_at INTEGER")
assert old_read("legacy") == "complete", "old_reader"


def request(job, authenticated_account, requested_account, accepted_at):
    # Synthetic identity context; not an authentication implementation.
    if authenticated_account != requested_account:
        raise PermissionError("account")
    old_write(job, requested_account, accepted_at)


before = db.total_changes
try:
    request("denied", "cedar", "harbour", 0)
except PermissionError:
    pass
else:
    raise AssertionError("deny_cross_account")
assert db.total_changes == before, "deny_before_write"
for job, accepted_at in [("ready", 0), ("failed", 1), ("lost", 2), ("young", 15)]:
    request(job, "cedar", "cedar", accepted_at)
assert db.execute("SELECT finished_at FROM exports WHERE id='young'").fetchone() == (None,), "old_writer"
db.execute("UPDATE exports SET state='complete', finished_at=5 WHERE id='ready'")
db.execute("UPDATE exports SET state='failed', finished_at=4 WHERE id='failed'")
assert old_read("ready") == "complete", "new_write_old_read"


def outcome(accepted_at, state, finished_at, now, deadline=10):
    if accepted_at + deadline > now:
        return "not_due"
    if state == "failed":
        return "failed"
    if state == "complete":
        if finished_at is None:
            return "unknown"
        return "on_time" if accepted_at <= finished_at <= accepted_at + deadline else "late"
    return "overdue"


rows = db.execute("SELECT id, accepted_at, state, finished_at FROM exports ORDER BY id").fetchall()
observed = {job: outcome(accepted, state, finished, now=20) for job, accepted, state, finished in rows}
assert observed == {"failed": "failed", "legacy": "unknown", "lost": "overdue",
                    "ready": "on_time", "young": "not_due"}, "outcome_partition"
due = [value for value in observed.values() if value != "not_due"]
assert (due.count("on_time"), len(due)) == (1, 4), "completion_denominator"
assert outcome(0, "complete", 11, now=20) == "late", "late_completion"
assert outcome(0, "complete", -1, now=20) != "on_time", "invalid_timestamp"
assert outcome(10, "queued", None, now=20) == "overdue", "deadline_boundary"
assert db.execute("SELECT COUNT(*) FROM exports WHERE id='denied'").fetchone()[0] == 0, "denied_absent"
print("Python 3.12.12; SQLite 3.51.2")
print("due=4 on_time=1 failed=1 overdue=1 unknown=1 not_due=1")
print("old reader/writer compatible; cross-account write denied")
db.close()

Read the denominator before trusting the indicator

At observation time 20, the ready export was accepted at 0 and completed at 5. The failed export was accepted at 1 and failed at 4. The lost export was accepted at 2 and is still queued. All have reached their ten-second deadline. The young export was accepted at 15, so the program excludes it from the due cohort rather than treating unfinished, still-eligible work as failure.

The legacy row was accepted at -10 and marked complete by the old writer. Its completion time is NULL after the migration. That is insufficient evidence of timely delivery. The program keeps it in the due denominator and labels its timing unknown. Counting every complete state as timely would turn this missing evidence into apparent success.

The executed partition is due=4, on_time=1, failed=1, overdue=1, unknown=1, with one further not_due row. The on-time fraction is therefore 1/4 for this constructed cohort. It is not a recommended reliability target. Assertions also cover a completion after the deadline, an invalid earlier completion time and the exact deadline boundary.

Google’s SRE guidance distinguishes a measured service indicator from the objective chosen for it. I recommend defining the eligible cohort, deadline, observation time and unknown-data policy before setting a target. In a real monitor, also define the reporting window and how late-arriving or corrected events revise it. This in-memory example does not implement that reporting pipeline.

Make the check fail for the right reason

The companion runner changes the actual program three times. Removing the account check fails deny_cross_account. Writing before checking permission fails deny_before_write, even though an exception still reaches the caller. Counting a missing completion timestamp as on_time fails outcome_partition. Each altered program runs in its own disposable container, and the runner requires the intended assertion marker rather than accepting any crash as a useful failure.

OWASP recommends validating permissions on every request and testing authorization logic. Here, authenticated_account is supplied by the synthetic caller. Equality with requested_account is only a demonstration boundary: it does not verify sessions, membership changes, roles or a download endpoint. The denial assertions establish no row was inserted through this one function. They are not a security assessment of an application.

For the lost row, I recommend inspecting the worker and storage outcome before replaying it. “Queued past its deadline” establishes overdue work in this model, not whether an external file or side effect already exists. Stopping new requests would not by itself reconcile existing work. The example deliberately leaves replay unresolved because it has no storage evidence from which to make that decision.

Know what the compatibility witness leaves out

SQLite supports adding columns subject to its documented restrictions. This demonstration observes one nullable addition and specific old SQL statements. It does not establish compatibility for positional INSERT statements, SELECT * consumers, a later NOT NULL constraint, another database engine or concurrent production processes. Keeping the old statement valid is weaker than proving a rollout safe.

I recommend using this style of evidence in a change review: show the intended behavior, preserve an unknown outcome when data is missing, and demonstrate that a relevant defect breaks the check. A real export release still needs file-content checks, authorization at download time, queue delivery tests, monitoring delivery to an operator, and a recovery exercise. The executable example cannot stand in for those system boundaries.