> ## Content Index > Fetch the complete content index at: https://samp-protocol.org/llms.txt > Use this file to discover other available public pages before exploring further. # SAMP Engineering Notes #12 - URL: https://samp-protocol.org/samp-engineering-notes-12/ - Published: 2026-11-07T19:27:07.000Z - Updated: 2026-11-08T12:13:53.000Z - Description: What does SAMP conformance actually mean? Explore role-specific tiers, mandatory requirements, reproducible test matrices, failure injection, and evidence-based interoperability claims. - Author: Stelio E. Efstathiou - Tags: Engineering Notes, SAMP Articles ### What Does SAMP Conformance Actually Mean? *From protocol requirements to reproducible test vectors, failure injection, independent interoperability, and evidence-based implementation claims.* ### 1\. Two Implementations. One Protocol. Different Results. Imagine two independently developed SAMP implementations. Implementation A provides a Manager. Implementation B provides an Agent endpoint. They establish an authenticated HTTPS connection. The Agent successfully announces its profile through DISCOVER. The Manager issues a QUERY. The Agent returns a valid operational status. Everything works. Can both developers now claim SAMP conformance? Not yet. Consider what happens when the same QUERY contains duplicate field paths, or when its expected profile revision no longer matches the endpoint's authoritative snapshot. What happens when a previously accepted DISCOVER announcement arrives after a newer one? Can the Manager distinguish a legitimate retransmission from a conflicting request? And what happens when a side-effecting EXEC request succeeds, but the response disappears before reaching the Manager? These conditions are not exceptional curiosities. They are part of the protocol contract. Two implementations that agree only on successful exchanges may still disagree on the behavior most important for operational safety. **Interoperability is not demonstrated merely because two systems can exchange valid messages. It must also survive the conditions under which the protocol is expected to reject, recover, or preserve uncertainty.** This is the engineering problem addressed by SAMP conformance testing. ### 2\. Conformance Is a Specific Claim A useful conformance statement must identify exactly what is being evaluated. SAMP v0.2 defines conformance separately for two protocol roles: - **Manager:** The management-side participant. - **Endpoint:** The Agent-side participant, whether embedded, implemented as a sidecar, or deployed through another supported model. These roles have different responsibilities. An endpoint that can answer QUERY requests has not demonstrated that it can validate authenticated DISCOVER announcements as a Manager. Likewise, a Manager that correctly issues CONFIG requests has not demonstrated the endpoint-side enforcement obligations associated with accepting those requests. A complete claim therefore needs at least three identifiers: **Specification revision + implementation role + conformance tier.** For example: *SAMP v0.2, draft-efstathiou-samp-agent-management-04, Agent endpoint role, SAMP-Subscribe tier.* This is a precise target for evaluation. It is not automatically a verified result. The implementation must still satisfy every mandatory requirement applicable to that role and tier. A fourth element is equally important: the actual test evidence supporting the claim. ### 3\. The Four SAMP Conformance Tiers SAMP v0.2 defines four additive tiers. | Tier | Required protocol operations | | ------------------ | -------------------------------- | | **SAMP-Observe** | PING, DISCOVER, QUERY, PUSH | | **SAMP-Subscribe** | Observe + SUBSCRIBE, UNSUBSCRIBE | | **SAMP-Configure** | Subscribe + CONFIG | | **SAMP-Execute** | Configure + EXEC | The tiers are cumulative. An implementation claiming SAMP-Execute must also satisfy the requirements of the lower tiers. However, **implementing an operation is not the same as enabling it in a deployment**. An endpoint may correctly implement EXEC while its deployed execution policy keeps EXEC disabled. When disabled, the endpoint must return the specified error rather than silently accepting the operation or exposing an unintended execution path. The declared management impact class and local policy remain binding regardless of tier. A conformance tier describes implemented protocol semantics. It does not grant operational authority. Nor does SAMP-Execute mean that an Agent is more intelligent, more autonomous, or capable of executing arbitrary instructions. It means that the implementation supports the specified management-operation contract, including its restrictions and failure behavior. ### 4\. Mandatory Requirements Apply Even to Observe-Only Implementations A common implementation mistake is to treat security and validation as optional features reserved for higher conformance tiers. SAMP does not follow that model. Even SAMP-Observe implementations have mandatory baseline requirements. These include: - UTF-8 JSON encoding and required PDU envelopes. - RFC 8785 JSON canonicalization. - HMAC-SHA256 authentication support. - HTTP transport and HTTPS support for non-loopback deployments. - Defined error handling and request correlation. - Required profile and recursive type-declaration validation. - Separate trust and liveness semantics appropriate to the implementation's role. The local loopback testing exception does not eliminate mandatory implementation support for authentication or secure transport. Managers must validate profiles. Endpoints must expose valid profiles and validate received inputs. This distinction matters when evaluating a minimal proof of concept. A simple HTTP endpoint returning a JSON object containing an Agent status may be a useful development experiment. It is not, by itself, a conformant SAMP-Observe implementation. **The minimum conformance tier is a complete protocol contract, not an exemption from protocol correctness.** ### 5\. Turning Requirements Into Testable Invariants A protocol specification contains requirements describing how implementations must behave. A conformance test translates one or more of those requirements into an observable experiment. A useful test should define: | Element | Description | | ---------------- | --------------------------------------------------------------------- | | Test ID | Stable identifier for the test | | Requirement | Exact specification section and applicable normative rule | | Initial state | Enrollment, profiles, credentials, revisions and other relevant state | | Stimulus | Message, failure, state transition or sequence of operations | | Expected result | Required protocol response and permitted state changes | | Forbidden result | Behavior that must never occur | | Evidence | Captured messages, state records, logs or reproducible artifacts | The forbidden result is especially useful. For example, a test may verify that an endpoint returns `replay_detected`. But if the endpoint also executed the operation a second time, the implementation has failed the safety requirement even though it returned the correct error. Similarly, a Manager may correctly reject a stale profile announcement while its implementation has already overwritten the registry with stale information. That is not a successful test. Conformance requires evaluating externally visible results and relevant internal state transitions. **A correct error code does not compensate for an incorrect side effect.** The following matrices illustrate selected test vectors derived from SAMP-04\. They are not a substitute for the complete normative requirements or an officially published reference test suite. ### 6\. Test Matrix A: Observe and Common Protocol Behavior The first group covers the baseline behavior expected from Observe-tier participants and the common protocol machinery. | Test ID | Test scenario | Expected result | | ------- | --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | OBS-01 | Valid authenticated PDU using supported protocol version | Accepted according to its operation and policy | | OBS-02 | Unsupported protocol version | Otherwise reply-eligible requests receive version\_unsupported; unsupported responses, errors and notifications, including PUSH, produce no SAMP reply. | | OBS-03 | Modified authenticated message after HMAC calculation | Authentication failure; no unauthorized processing | | OBS-04 | Malformed JSON or duplicate JSON member names | Rejected during validation | | OBS-05 | Invalid profile or type declarations | Appropriate profile/schema validation error; no invalid authoritative installation | | OBS-06 | Stale or conflicting DISCOVER profile announcement | Specified stale/conflict handling; accepted profile remains unchanged | | OBS-07 | QUERY with incorrect expected profile revision | profile\_revision\_mismatch before field resolution | | OBS-08 | QUERY requesting duplicate field paths | invalid\_value | | OBS-09 | Valid QUERY containing present, absent and unknown fields | Exact response partition or specified all-unknown error | | OBS-10 | Manager loses authoritative profile ordering state | Admission fenced until valid recovery or enrollment replacement | These tests involve different classes of validation. Some concern message syntax and cryptographic authenticity. Others concern profile identity, state continuity, and the interpretation of management data. The QUERY cases deserve particular attention. SAMP requires implementations to distinguish: 1. A present, valid field. 2. A declared optional field that is absent. 3. A present field whose value is explicitly null and whose declaration permits null. 4. An unknown or unexposed field. 5. A mixed set containing these different outcomes. 6. Requests whose fields are all absent or all unknown, each with its specified handling. A successful QUERY response must partition its requested fields exactly once across `values`, `absent`, and `unknown`. An implementation cannot simply return null for every unavailable value. That would destroy the distinction between an observed value, an optional absence, and an unsupported field. Similarly, a delayed QUERY response must remain associated with the profile revision under which it was evaluated. The Manager must not reinterpret that response using a newer profile merely because it arrived after an update. These requirements are testable independently of any AI model or agent framework. ### 7\. Authentication and Canonicalization Need Negative Tests Successful HMAC verification is necessary but insufficient. SAMP v0.2 authenticates canonical JSON using HMAC-SHA256. The canonicalization procedure excludes only `auth_tag` from the authenticated PDU object. Other fields remain included, including timestamps, request identifiers, payload content, and optional key identifiers. A useful interoperability experiment should exercise at least three conditions. **Equivalent JSON representations** Two JSON serializations with different insignificant whitespace or object-member ordering represent the same canonical data. Their correctly computed authentication tags must verify consistently. **Modified authenticated content** Changing an authenticated field without recomputing the tag must cause authentication failure. **Invalid input before cryptographic processing** Malformed JSON, duplicate member names, and values incompatible with the canonicalization requirements must be rejected through the appropriate validation path. A test should also cover unknown `key_id` behavior, ensuring the receiver does not silently substitute an unrelated shared secret. These tests are particularly valuable when independently developed implementations use different programming languages and JSON libraries. A Python sender and a Rust receiver, for example, must agree on the authenticated canonical bytes rather than on the incidental formatting produced by their preferred serializers. The important result is agreement with the defined protocol representation, not agreement with a particular library. ### 8\. Test Matrix B: Subscription Lifecycle and Delivery The SAMP-Subscribe tier adds stateful event-delivery management. That introduces ownership, expiration, duplicate-processing, and capacity obligations. | Test ID | Test scenario | Expected result | | ------- | ------------------------------------------------------------- | --------------------------------------------------------------------------- | | SUB-01 | New authorized SUBSCRIBE | A subscription is created with its defined identity and expiry | | SUB-02 | Identical recorded SUBSCRIBE retry | Original result returned; no new allocation or expiry extension | | SUB-03 | Two concurrent copies of the same SUBSCRIBE | At most one allocation; recorded-state serialization | | SUB-04 | Subscription reaches its expiration boundary | Expiry enforced according to accepted TTL | | SUB-05 | One Manager attempts to cancel another Manager's subscription | policy\_denied without ownership information disclosure | | SUB-06 | Replacement subscription overlaps the original | Duplicate event copies retain the same event\_id | | SUB-07 | Subscription quotas exceeded | Appropriate rate\_limited or back\_pressure behavior | | SUB-08 | UNSUBSCRIBE while PUSH is already in flight | No new delivery generation after cancellation; in-flight delivery tolerated | An important test concerns retransmission. A SUBSCRIBE retry is not a request to extend the subscription's lifetime. Another concerns event identity. When one logical event is delivered through overlapping subscriptions, deduplication should rely on the Agent and event identifiers, not the subscription through which each copy arrived. A third concerns failure recovery. Core SAMP does not promise subscription persistence across endpoint restarts or WebSocket reconnections. A conformance test must not invent such a guarantee. If an implementation supports a separately specified persistence extension, the extension's requirements must be evaluated independently. ### 9\. Test Matrix C: CONFIG and EXEC Safety The Configure and Execute tiers introduce consequential operations. These are the operations where incorrect handling may create changes outside the management protocol itself. The test objective must therefore go beyond receiving an apparently successful response. | Test ID | Test scenario | Expected result | | ------- | ----------------------------------------------------------------- | ------------------------------------------------------------------------- | | OPS-01 | CONFIG modifies an undeclared or non-writable field | Rejected; prohibited field unchanged | | OPS-02 | CONFIG includes typed fields with partial outcomes | Results and state changes conform to declared partial-operation semantics | | OPS-03 | CONFIG/EXEC lacks required authorization or approval | Specific denial or pending-authorization outcome; no execution | | OPS-04 | EXEC operation is disabled by local policy | exec\_disabled; no execution | | OPS-05 | Same authenticated request retransmitted during execution | No second admission or duplicate side effect | | OPS-06 | Reused request\_id with conflicting bound content | replay\_detected; no new execution | | OPS-07 | Operation completes but response is lost | Recorded outcome recovered when permitted, or uncertainty preserved | | OPS-08 | Operation outcome changes from indeterminate to terminal | Valid monotonic record\_revision progression | | OPS-09 | Endpoint cannot establish its latest committed operation revision | No fabricated newer outcome; admission/retrieval fenced as specified | | OPS-10 | External effect succeeds while audit persistence fails | Effect and audit-persistence outcomes remain separately represented | The exact-action authorization contract is especially important. A valid authorization decision for one target, operation, or set of parameters must not silently authorize a different action. Similarly, a valid credential or declared capability must not bypass local endpoint enforcement. For recorded consequential operations, SAMP requires explicit outcome revision handling. A newly committed numbered outcome begins at `record_revision` 1. A valid later resolution can advance the revision. An older revision must not replace a newer accepted revision, and conflicting outcomes cannot share the same revision identity. A Manager also needs to preserve the exact outcome revision on which its own decision was based, even if the endpoint later reports a newer resolution. This is essential for reconstructing the decision history. ### 10\. The Test That Matters Most: An Unknown Execution Outcome Consider a non-idempotent operation. The Manager sends EXEC. The endpoint authenticates and admits the request. The operation causes an external effect. Before the Manager receives the response, the connection fails. What should the Manager do? Simply sending a new request with a different identifier would be dangerous. The first operation may already have succeeded. SAMP defines authenticated request identity, recorded-state handling, and replay safeguards to reduce the possibility of unsafe duplicate admission. But these mechanisms do not turn every external system into an exactly-once transactional environment. A proper test should examine a sequence such as: 1. Submit an authorized EXEC request with a fixed `request_id`. 2. Allow the endpoint to admit the operation. 3. Interrupt the response path at a controlled point. 4. Reconnect and attempt a permitted identical authenticated retry. 5. Verify that the endpoint does not initiate a second execution. 6. Verify that any returned historical or current outcome has an established authoritative record. 7. If the outcome cannot be established, verify that uncertainty is preserved rather than replaced with an invented success or failure. This test must also be repeated under different crash points. A crash before admission is not equivalent to a crash after the external effect. An interrupted response is not proof that the operation failed. An empty replay cache is not proof that the operation never occurred. The implementation must apply the specification's declared freshness, retention, and restart-fencing requirements. **The central safety property is not that every outcome is always known. It is that uncertainty must not authorize an unsafe second execution.** ### 11\. Restart, Eviction, and Lost State Many protocol implementations pass ordinary request-response tests but fail after process restarts. This is particularly dangerous when protocol correctness depends on persistent ordering or recorded operations. SAMP defines separate state-continuity requirements for profile announcements and consequential operations. For profile ordering, a Manager must preserve the latest accepted authoritative ordering boundary. If that boundary is lost, the Manager must not assume an older authenticated profile is once again authoritative. For CONFIG, EXEC, SUBSCRIBE, and UNSUBSCRIBE, implementations must preserve the safety properties associated with retained requests, reservations, and their configured replay windows. If required recorded state is unavailable after restart, new admission may need to remain fenced. The specification does not permit an implementation to discard uncertain state and immediately resume potentially conflicting operations merely because its process is running again. A serious test suite should therefore include: - Restart with committed state preserved. - Restart with recorded-state loss. - Replay-cache eviction. - In-progress operation during restart. - Retry at an inclusive freshness boundary. - Backward wall-clock adjustment. - Credential rotation within an enrollment. - Explicit enrollment replacement using a distinct authentication context. These are not equivalent tests. A retained credential may support authorized historical recovery without authorizing new operations in a replacement enrollment. Likewise, a changed credential does not erase external effects already performed. Recovery correctness depends on the actual state and authority boundaries, not on a convenient assumption that a restart creates a clean system. ### 12\. Evidence Testing Is Different From Evidence Claims SAMP supports optional runtime-evidence mechanisms intended to help relate management outcomes to observations and effect boundaries. That does not make every attached diagnostic record independently trustworthy. Referenced evidence introduces additional validation requirements. For applicable referenced-evidence implementations, the test suite must verify controls such as: - Hashing the required identity-coded content bytes before parsing. - Rejecting prohibited or ambiguous content encodings. - Enforcing declared byte and retrieval limits. - Enforcing timeout, redirect, duration, and concurrency limits. - Rejecting evidence that cannot satisfy required pre-admission validation. Tests must also distinguish a valid historical artifact from a currently valid authorization or verification context. For consequential operations, evidence of an external effect and evidence of audit persistence are separate. An operation may have produced an effect while its audit write failed. An observation may be authentic without proving the claimed causal relationship. An event may be delivered without being durably recorded. A useful conformance suite must test these boundaries, not merely confirm that an evidence object contains syntactically valid fields. **Evidence quality is determined by validated bindings and supported guarantees, not by the presence of an evidence-shaped payload.** ### 13\. Trust, Liveness, and Enforcement Must Remain Separate A complete evaluation also needs state-transition tests. Consider an Agent that was previously trusted but becomes unreachable. The Manager may record its liveness as offline. This does not automatically mean its trust state should be reset. When the endpoint becomes reachable again, the Manager should not silently promote or change its trust disposition merely because communication has recovered. Similarly, suppose the Manager decides to quarantine an Agent. The decision to quarantine does not establish that the quarantine has been enforced. SAMP distinguishes enforcement states, including `requested`, `applied`, `failed`, and `indeterminate`. A conformance experiment should verify that requested or indeterminate enforcement is not represented as confirmed isolation. The test should also examine whether the relevant management or evidence endpoint resides inside the failure domain it claims to observe or control. An endpoint that becomes unreachable together with its managed workload cannot automatically provide independent proof of that workload's continued isolation. This distinction affects safety, recovery, and operational reporting. ### 14\. Independent Interoperability Is a Higher Standard of Evidence Different levels of testing provide different kinds of evidence. | Evidence level | What it demonstrates | What it does not automatically prove | | ---------------------------------- | ------------------------------------------------------------------ | ---------------------------------------- | | Static validation | Documents, schemas or examples satisfy checked rules | Runtime protocol behavior | | Unit tests | Individual implementation components behave as tested | End-to-end correctness | | Same-team integration | Components can cooperate in a controlled environment | Independent interoperability | | Failure-injection tests | Particular failure scenarios produce observed results | Coverage of all normative requirements | | Independent interoperability tests | Separately developed systems agree on the tested contract | Universal compatibility or certification | | Independent security evaluation | Specified security properties were assessed within a defined scope | Absence of all vulnerabilities | The higher levels do not eliminate the need for the lower ones. Nor should an interoperability result be interpreted beyond its actual coverage. For example, independently developed Observe-tier implementations may successfully exchange authenticated management information. That does not establish SAMP-Execute conformance. A successful Python-to-Rust exchange does not establish that every other language implementation will behave identically. And a successful test against one endpoint does not prove that every adapter within a larger management environment satisfies the same requirements. The purpose of independent implementation is to expose assumptions that may remain invisible when both sides were written by the same team. **A protocol becomes more credible when different engineers can implement its requirements and obtain consistent results without depending on shared implementation code or private conventions.** ### 15\. What a Reproducible Conformance Report Should Contain An implementation report should contain enough information for another engineer to understand the claim and reproduce the experiment. A practical reporting format might contain: | Report field | Required information | | ----------------------- | ------------------------------------------------------------------------------ | | Protocol reference | Exact Internet-Draft revision | | Implementation | Project, release and source revision | | Role | Manager or Agent endpoint | | Claimed tier | Observe, Subscribe, Configure or Execute | | Transport | HTTP/HTTPS and any tested optional mappings | | Authentication | Supported and exercised authentication configuration | | Test environment | Operating system, runtime, relevant deployment model | | Test vectors | Identifiers and specification mappings | | Results | Pass, fail, skipped or not tested | | Negative tests | Rejection, replay, authorization and failure behavior | | Durability declarations | Freshness, retry, retention and restart characteristics | | Known limitations | Untested requirements, unsupported optional extensions and unresolved failures | | Independent counterpart | Separate implementation and version, if applicable | The report should not convert untested requirements into successful results. A skipped test is not a pass. A locally verified result is not an independent interoperability result. A demonstration is not a security certification. And a known limitation should not disappear from reporting merely because it complicates a marketing statement. A useful report should allow reviewers to identify both what was demonstrated and what remains unknown. ### 16\. The Status of SAMP Conformance Today SAMP v0.2 is specified in the individual Experimental Internet-Draft: **draft-efstathiou-samp-agent-management-04** The specification defines conformance tiers, mandatory protocol behavior, and extensive conformance-testing requirements. However, the Internet-Draft explicitly states that a reference interoperability test suite is not yet available. It also distinguishes earlier AgMS implementation experience from full validation against the later SAMP v0.2 requirements. The existing AgMS experimental environment provides valuable implementation and operational-testing experience. It does not establish that AgMS or all its adapters independently conform to every mandatory requirement of SAMP-04. Nor should work with related protocol initiatives be presented as completed interoperability validation before actual experiments and results exist. The present engineering objective is therefore clear: **Convert the protocol requirements into repeatable test vectors and exercise them across independently developed Manager and endpoint implementations.** A successful result would provide evidence supporting explicitly scoped conformance claims. A failed result would identify a defect, an ambiguous requirement, or an implementation disagreement requiring further investigation. Both outcomes would be useful. The goal is not to produce a certificate as quickly as possible. The goal is to make the protocol independently testable. ### 17\. A Practical Starting Experiment An independent developer does not need to implement every SAMP tier before contributing useful interoperability evidence. One reasonable starting point is SAMP-Observe. Two independently developed implementations could begin with: **Phase A: Authenticated Communication** Exchange valid authenticated PDUs over HTTPS and verify canonicalization and error processing. **Phase B: Profile Admission** Announce a valid profile and exercise malformed, stale, duplicate, conflicting, and out-of-order announcements. **Phase C: Revision-Bound Queries** Validate QUERY behavior across present, absent, nullable, unknown, and mixed field results. **Phase D: Recovery** Restart the Manager, test retained profile ordering, and deliberately remove required state to verify safe failure behavior. **Phase E: Cross-Implementation Comparison** Execute the same vectors against another independent implementation and compare responses and state transitions. Record both successful and failed tests. Publish the implementation versions, environment, supported tier, limitations, and reproducible results. Only after the Observe requirements have been properly evaluated should additional tier claims be made. This staged approach permits interoperability work to begin without weakening the protocol's mandatory requirements. It also avoids treating an unfinished Execute-tier implementation as proof that the underlying Observe-tier behavior is correct. ### 18\. The Engineering Lesson A protocol specification is a contract. An implementation is an attempt to satisfy that contract. A test suite is a mechanism for evaluating whether specific obligations have been met. And an interoperability experiment examines whether independent implementations interpret those obligations consistently. These are four different things. SAMP is intentionally designed as a management plane for heterogeneous AI agents. Its purpose is not to require one agent framework, one programming language, one management application, or one infrastructure architecture. That independence creates an obligation. The management semantics must be explicit enough that separate implementations can be evaluated against the same rules. Successful communication is part of that evaluation. So are rejected messages. So are stale profiles, conflicting identifiers, denied authorization, duplicated requests, missing audit records, unavailable endpoints, and indeterminate execution outcomes. **Conformance is not demonstrated by showing that a system works when everything goes right.** **It is demonstrated by showing that the system also obeys the protocol when things go wrong.** That is the standard against which SAMP implementations should be evaluated. And it is the foundation on which credible interoperability can be built. --- ### Specification Reference This Engineering Note discusses the Experimental SAMP v0.2 specification, particularly Sections 5.13, 7, 8, 9, 10.3, 11.1, 11.2, 12.2 and 17 of: [Simple Agent Management Protocol (SAMP), draft-efstathiou-samp-agent-management-04](https://datatracker.ietf.org/doc/draft-efstathiou-samp-agent-management/04/?ref=samp-protocol.org) SAMP remains an individual Internet-Draft intended for experimental evaluation. It is not an approved IETF standard. The conformance matrices in this article are illustrative selections derived from the specification. They are neither an exhaustive normative test suite nor a claim of completed independent interoperability testing. The Internet-Draft remains authoritative for protocol requirements. *Stelio E. Efstathiou* *SAMP Protocol Project*