Conformance
A protocol is only as good as the test that says an implementation follows it. The Monitor Protocol's conformance suite is that test: 31 checks, written twice, one runner in TypeScript and one in Python.
Run the suite
Against any server, over HTTP:
npx @mentu/monitor-protocol conform --base http://127.0.0.1:8124Against the reference server itself, started in the same process:
npx @mentu/monitor-protocol conform --selfWith the Python runner, which needs nothing but Python 3:
python3 conformance/python/run.py --base http://127.0.0.1:8124 --subjects a,b,c --schemas schemasThe Python runner lives in the repository. --subjects names work items the server knows, for the lease checks, and --schemas points at the JSON Schemas for C29. Add --json for machine-readable output.
Each check prints PASS, FAIL or SKIP, with a note that says why. A skip is allowed only where a server cannot be tested for a feature it does not have: C17 needs a server that deletes old entries.
What the checks prove
| Check | What it proves |
|---|---|
| C01 | Discovery returns the protocol versions, capabilities, server identity and a cache lifetime |
| C02 | A filter with an unknown key is refused, and the reply lists the known keys |
| C03 | A types value that is neither declared nor a valid prefix is refused |
| C04 | An agent asserting tier: src is refused with TIER_NOT_ASSERTABLE |
| C05 | A pull does not move the cursor: a second pull returns the same observations |
| C06 | Acknowledging below the stored cursor is refused, and acknowledging the same value again is harmless |
| C07 | A narrow filter does not change a monitor's head |
| C08 | A filter sent with a pull narrows what is returned and never widens it |
| C09 | Every observation is a valid CloudEvent with the protocol's six attributes |
| C10 | sequence is zero-padded to 20 digits and strictly increasing |
| C11 | A subscription without act cannot claim a lease |
| C12 | Two claims on one subject at the same moment: exactly one wins, and the other is told who holds it |
| C13 | Completing a lease that expired and was claimed by someone else is refused with LEASE_LOST |
| C14 | Pausing, resuming and retiring a monitor, and retiring a subscription, are each recorded as observations, and a paused monitor refuses new ones until it is resumed |
| C15 | The state carries its time, coverage, liveness and confidence, and lists a missing input as missing |
| C16 | A rejected input is visible as an ai.mentu.monitor.rejected observation |
| C17 | A pull below the retention floor answers 410 CURSOR_EXPIRED with the floor and where to rebuild |
| C18 | A private monitor is invisible without its token and visible with it |
| C19 | A subscription that stops pulling is retired and recorded, and subscribing again keeps its cursor |
| C20 | Sending the same acknowledgement or claim twice returns the same result |
| C20b | Claiming a lease you already hold returns the same lease, not a conflict |
| C21 | A correction names what it replaces in supersedes, and the original is unchanged |
| C22 | After acknowledging by the specification's own definition of the cursor, those observations do not come back |
| C23 | A wrong token and a missing token are both refused with 401 |
| C24 | A lease claimed by its holder can be completed, so the lease checks are not passing by refusing everything |
| C25 | A reader that has caught up reports nothing waiting, and another monitor's traffic changes neither its count nor its head |
| C26 | Five ways to inflate a claim are refused, a field left out stops below the top, and an agent naming its owner is honoured |
| C27 | A cursor that is not a whole number is refused, in an acknowledgement and in a replay |
| C28 | A stranger cannot get act just by asking for it |
| C29 | Every object the run received validates against its JSON Schema |
| C30 | A request from a web page on an origin the server does not allow is refused, and the same request without an origin is served |
Testing the test
A suite that cannot fail a wrong server is decorative. So the suite was tested too: an independent audit broke the reference server's rules on purpose, one at a time, and ran the suite against each broken server. Some passed that should have failed. The checks C22 to C28 exist because of that experiment, and each was added before the fix it guards, so the suite could be seen failing first.
Two runners do not make two independent readings of the specification. They are kept in step, and what they give you is that a change must satisfy both.
Implementations
| Implementation | Language | Result |
|---|---|---|
| Reference server | TypeScript | Passes all 31 checks |
| Atrio | Python | Passes 30 and skips C17, because it keeps everything and cursor expiry cannot be tested |
To add yours, run the suite against it and describe what the implementation taught the specification in adapters/.