5. Operate, diagnose and evolve a workflow
A validated YAML document is the beginning of deployment qualification, not its end. This chapter turns the first workflow into an operational checklist. Keep the complete language reference and upgrade preflight guide nearby.
Distinguish admission from business success
Section titled “Distinguish admission from business success”| Observation | Meaning | Next action |
|---|---|---|
| HTTP 400 application rejection | The submitted command failed ingress admission; it was not pooled. For a static admission rule, the body’s details names the rule, its deny code and, for a batch, the deciding write | Correct its encoding, size or other stated invalidity and submit again |
| HTTP 503 admission unavailable | The admission callback could not complete | Investigate node warnings and plugin health; it is not business acceptance |
| HTTP 202 | The node accepted the submission into its processing path | Wait for finalization, then inspect the receipt |
| Finalized accepted receipt | This source’s cascade committed | Inspect the resulting business state; external delivery may still be pending |
| Finalized rejected receipt | This source’s business writes and effects did not commit | Read the code and failed step, correct the cause and submit a fresh signed message if appropriate |
Finalized ADMISSION_RULE_* receipt | An admission rule refused a step; the whole cascade rolled back | Read the failed step’s rule trace: the rule, clause and deny code say which requirement did not hold |
Early admission cannot know all future state or remaining block capacity. A well-formed command can therefore still fail during execution. The host does not invalidate the whole finalized block because one cascade was rejected. Its terminal receipt remains, while the source’s business changes are rolled back.
Do not retry a finalized rejection using the same message ID: replay returns the original receipt. A fresh envelope/ID is a new attempt, not a bypass of business idempotency, authorization or one-use approval consumption. A capacity failure may succeed in a later block; a bad signature or invalid action needs a real correction.
Where did my command stop?
Section titled “Where did my command stop?”Start from what the submission returned and follow the receipt. Each answer says where the outcome is recorded and whether a new message can help. The rejection-code table has every code.
What you're seeing: start from what your submission returned, then follow the receipt. Each answer says where the outcome is recorded and whether sending a new message can help.
What did the submission return?
- HTTP 400 with a
codeWhich code did the 400 carry?
A 400 means the ingress member refused the message before pooling it. There is no receipt.
UNKNOWN_BINDING_SOURCEThe topic is not a component’s topicEvery component listens on its own ingress topic,
<id>.command.v1unless the document setstopic. A message on any other topic is refused.- Recorded in
- Nothing: the message was not pooled.
- Will a retry help?
- Yes, on the right topic.
MALFORMED_SOURCE_COMMANDThe body did not decodeThe component’s codec could not decode the body. Encode commands with the machine’s public codec, such as
KvRegistryContract.put, not with arbitrary JSON.- Recorded in
- Nothing: the message was not pooled.
- Will a retry help?
- Yes, once the encoding is fixed.
COMMAND_PAYLOAD_TOO_LARGEorCOMMAND_WORK_EXCEEDEDThe command cannot fitEither a subscribed baseline event would exceed the event limit, or the command’s mandatory work exceeds the per-cascade allowance. Both limits are committed in the profile.
- Recorded in
- Nothing: the message was not pooled.
- Will a retry help?
- Only with a smaller command, or after a qualified profile change.
ADMISSION_RULE_*orEXPRESSION_CAPACITY_EXCEEDEDA static admission rule refused itA rule that reads only the command, parameters, configuration and write content is checked at ingress too. The 400 body’s
detailsnames therule, itsdenycode and, for a batch, the decidingwrite.EXPRESSION_CAPACITY_EXCEEDEDmeans evaluating those rules ran out of work.- Recorded in
- The HTTP response only.
- Will a retry help?
- Only with a command the rule allows.
- Another codeThe machine refused it at admission
The target machine’s own stateless admission check refused the command. Look the code up in that machine’s reference.
- Recorded in
- Nothing: the message was not pooled.
- Will a retry help?
- Yes, once the command is corrected.
- HTTP 429The pending pool is full
The ingress member’s pending pool is full, so the message was not kept or relayed.
- Recorded in
- Nothing: the message was not accepted.
- Will a retry help?
- Yes, after the backpressure clears.
- HTTP 503Admission is unavailable
The application’s admission check could not run, the chain is stopped or paused, or this node is not yet an active member (a joiner whose membership epoch is scheduled). This is not a business outcome.
- Recorded in
- The node’s logs and health, and
memberActiveForNextBlockin its status. - Will a retry help?
- Once the node is healthy again, or once it is an active member.
- HTTP 202 and a message id
What does the query
composite/binding-receipt-v1/<message-id>return?Query with the source message id and empty parameters. A 202 only means one member queued the message.
- An empty payloadNo receipt yet
An empty payload means no receipt at the height the node answered from. The message may still be pending. It may also have expired, or been lost when its only holder restarted. Absence is not a rejection.
- Recorded in
- Nothing yet.
- Will a retry help?
- Not blindly. Poll with backoff first. A resubmission is a new message with a new id.
- A receipt with status
ACCEPTEDDid the follow-up you expected happen?
- Yes. I want to confirm it.Every step committed
In an accepted receipt, steps that read
PLANNEDcommitted. Prove the receipt with the key frombindings receipt-key. External effects are delivered later and separately.- Recorded in
- The receipt and the component state.
- Will a retry help?
- Not needed.
- No. Its binding shows a
failedClausefrom 0 to 7.A false condition skipped the bindingThe clause at that index was false, so the binding derived nothing. That is not an error: the rest of the cascade still committed.
- Recorded in
[bindingId, failedClause]in the producing step’sconditions.- Will a retry help?
- Only if the data changes so the condition holds.
- No. Its binding is not in any step’s
conditions.No event selected the bindingA binding runs only when its source component emits its event. Check the component and the event id, not just the topic. Bindings that were never considered are not listed.
- Recorded in
- Its absence from every step’s
conditions. - Will a retry help?
- Not until the source emits that event.
- No, and the source command changed nothing either.Accepted, but nothing changed
Accepted means the cascade committed, not that state changed. A machine can approve a no-op, such as a second vote from the same member. Read the component’s state.
- Recorded in
- The component state, not the receipt.
- Will a retry help?
- Only with a command that changes something.
- Yes. I want to confirm it.
- A receipt with status
REJECTEDWhich kind of code is the receipt’s
code?Read
failedStepOrdinalfirst: it names the step that failed. Nothing in a rejected cascade committed.- An admission rule:
ADMISSION_RULE_*An admission rule refused a stepDENIED: a clause was false.ERROR: a clause or read could not be evaluated, for example an unguarded absent read.INPUT: a kernel value broke its declaration. The whole cascade rolled back.- Recorded in
rulesof the failed step:[heldCount, [ruleId, failedClause, denyCode, writeIndex]].- Will a retry help?
- Only once the input satisfies the rule.
- The target refused:
ADMISSION,MALFORMED_DERIVED_COMMANDor the machine’s own codeThe target refused the commandThe target machine’s codec, admission or decision refused the command. For a derived command, check what the mapping built.
- Recorded in
- The failed step’s
code. - Will a retry help?
- Yes, once the input or the mapping is corrected.
- An evaluation error:
EXPRESSION_*,MAPPING_*,FUNCTION_*,LOOKUP_KEY_*,EVENT_TYPE_ERRORorINVALID_UNICODEA condition or mapping could not be evaluatedUnlike a false condition, an evaluation error rejects the cascade. For a condition, the binding’s
failedClausepoints at the clause.- Recorded in
- The failed step’s
code, and itsconditions. - Will a retry help?
- Not with the same data and document.
- A limit or budget:
LIMIT_*,*_CAPACITY_EXCEEDED,*_WORK_EXCEEDED,*_TOO_LARGE,EFFECT_*A limit or budget ran outCascade limits such as
LIMIT_DEPTHandLIMIT_FANOUTfail the same way every time. Block budgets such asCAPACITY_EXCEEDEDandCRYPTO_WORK_EXCEEDEDreset each block.EXPRESSION_CAPACITY_EXCEEDEDcan be either. Work already spent is not refunded.- Recorded in
- The receipt’s
code. - Will a retry help?
- For a block budget, a new message in a later block can succeed.
- A conflict:
REPLAY_OR_CONFLICTorCONSUMPTION_CONFLICTA replay or consumption conflictREPLAY_OR_CONFLICT: the source id could not be claimed.CONSUMPTION_CONFLICT: two steps of one cascade used the same one-use key.- Recorded in
- The receipt’s
code. - Will a retry help?
- Replaying the same id returns this receipt. A new message is a new attempt.
- A plugin contract violation:
RESERVED_*,STATE_KEY_LIMIT,EVENT_MISSING_FIELD,UNDECLARED_WORK_REFERENCEA kernel broke its plugin contractA kernel wrote a reserved key or event, exceeded the state-key bound, used undeclared work, or emitted an event without a required field.
- Recorded in
- The failed step’s
code. - Will a retry help?
- No. Report it to the plugin’s maintainer.
- An admission rule:
- An empty payload
What did the submission return?
Which code did the 400 carry?
A 400 means the ingress member refused the message before pooling it. There is no receipt.
What does the query composite/binding-receipt-v1/<message-id> return?
Query with the source message id and empty parameters. A 202 only means one member queued the message.
Did the follow-up you expected happen?
Which kind of code is the receipt’s code?
Read failedStepOrdinal first: it names the step that failed. Nothing in a rejected cascade committed.
The ingress member’s pending pool is full, so the message was not kept or relayed.
- Recorded in
- Nothing: the message was not accepted.
- Will a retry help?
- Yes, after the backpressure clears.
The application’s admission check could not run, the chain is stopped or paused, or this node is not yet an active member (a joiner whose membership epoch is scheduled). This is not a business outcome.
- Recorded in
- The node’s logs and health, and
memberActiveForNextBlockin its status. - Will a retry help?
- Once the node is healthy again, or once it is an active member.
Every component listens on its own ingress topic, <id>.command.v1 unless the document sets topic. A message on any other topic is refused.
- Recorded in
- Nothing: the message was not pooled.
- Will a retry help?
- Yes, on the right topic.
The component’s codec could not decode the body. Encode commands with the machine’s public codec, such as KvRegistryContract.put, not with arbitrary JSON.
- Recorded in
- Nothing: the message was not pooled.
- Will a retry help?
- Yes, once the encoding is fixed.
Either a subscribed baseline event would exceed the event limit, or the command’s mandatory work exceeds the per-cascade allowance. Both limits are committed in the profile.
- Recorded in
- Nothing: the message was not pooled.
- Will a retry help?
- Only with a smaller command, or after a qualified profile change.
A rule that reads only the command, parameters, configuration and write content is checked at ingress too. The 400 body’s details names the rule, its deny code and, for a batch, the deciding write. EXPRESSION_CAPACITY_EXCEEDED means evaluating those rules ran out of work.
- Recorded in
- The HTTP response only.
- Will a retry help?
- Only with a command the rule allows.
The target machine’s own stateless admission check refused the command. Look the code up in that machine’s reference.
- Recorded in
- Nothing: the message was not pooled.
- Will a retry help?
- Yes, once the command is corrected.
An empty payload means no receipt at the height the node answered from. The message may still be pending. It may also have expired, or been lost when its only holder restarted. Absence is not a rejection.
- Recorded in
- Nothing yet.
- Will a retry help?
- Not blindly. Poll with backoff first. A resubmission is a new message with a new id.
In an accepted receipt, steps that read PLANNED committed. Prove the receipt with the key from bindings receipt-key. External effects are delivered later and separately.
- Recorded in
- The receipt and the component state.
- Will a retry help?
- Not needed.
The clause at that index was false, so the binding derived nothing. That is not an error: the rest of the cascade still committed.
- Recorded in
[bindingId, failedClause]in the producing step’sconditions.- Will a retry help?
- Only if the data changes so the condition holds.
A binding runs only when its source component emits its event. Check the component and the event id, not just the topic. Bindings that were never considered are not listed.
- Recorded in
- Its absence from every step’s
conditions. - Will a retry help?
- Not until the source emits that event.
Accepted means the cascade committed, not that state changed. A machine can approve a no-op, such as a second vote from the same member. Read the component’s state.
- Recorded in
- The component state, not the receipt.
- Will a retry help?
- Only with a command that changes something.
DENIED: a clause was false. ERROR: a clause or read could not be evaluated, for example an unguarded absent read. INPUT: a kernel value broke its declaration. The whole cascade rolled back.
- Recorded in
rulesof the failed step:[heldCount, [ruleId, failedClause, denyCode, writeIndex]].- Will a retry help?
- Only once the input satisfies the rule.
The target machine’s codec, admission or decision refused the command. For a derived command, check what the mapping built.
- Recorded in
- The failed step’s
code. - Will a retry help?
- Yes, once the input or the mapping is corrected.
Unlike a false condition, an evaluation error rejects the cascade. For a condition, the binding’s failedClause points at the clause.
- Recorded in
- The failed step’s
code, and itsconditions. - Will a retry help?
- Not with the same data and document.
Cascade limits such as LIMIT_DEPTH and LIMIT_FANOUT fail the same way every time. Block budgets such as CAPACITY_EXCEEDED and CRYPTO_WORK_EXCEEDED reset each block. EXPRESSION_CAPACITY_EXCEEDED can be either. Work already spent is not refunded.
- Recorded in
- The receipt’s
code. - Will a retry help?
- For a block budget, a new message in a later block can succeed.
REPLAY_OR_CONFLICT: the source id could not be claimed. CONSUMPTION_CONFLICT: two steps of one cascade used the same one-use key.
- Recorded in
- The receipt’s
code. - Will a retry help?
- Replaying the same id returns this receipt. A new message is a new attempt.
A kernel wrote a reserved key or event, exceeded the state-key bound, used undeclared work, or emitted an event without a required field.
- Recorded in
- The failed step’s
code. - Will a retry help?
- No. Report it to the plugin’s maintainer.
Find the receipt and its proof key
Section titled “Find the receipt and its proof key”Retain the submitted source message ID in your application. Query
composite/binding-receipt-v1/<source-message-id-hex> through the ordinary chain
query API. The canonical receipt contains the overall result, visited steps,
emitted events and condition results. A planned step inside a rejected
receipt is not a committed operation.
To discover its physical authenticated key without scanning state:
./yano.sh appchain bindings receipt-key "$SOURCE_MESSAGE_ID"This offline command returns stateKeyHex and the receipt/key query paths.
Alternatively query composite/binding-receipt-key-v1/<source-message-id-hex>
with empty parameters. Its generic query response’s payloadHex is already the
raw physical key—do not decode it as a second CBOR value.
Key discovery works even if no receipt exists. Request a state proof for the key and verify its presence/value against independently trusted chain identity, root and finality. It is not enough to trust the same server’s claimed root. An app-final proof is not automatically a Cardano-anchored proof; see anchors and verification.
Diagnose “my binding did not fire”
Section titled “Diagnose “my binding did not fire””- Confirm the source finalized on the expected chain and profile.
- Inspect the receipt’s overall status before looking at individual steps.
- Check the source instance and event ID, not just the command topic.
- Inspect the first false/error
failedClause(zero-based).-1usually means all clauses matched, but a pre-clause budget failure also uses it; read the rejection code and step together. Unvisited bindings have no invented trace. - Verify mapping types and target admission/authorization. A successful source
transition alone does not guarantee its derived target succeeds; an
admission rule on the target may refuse it, and each
step’s
rulestrace names that rule. - Check event, work, fan-out, receipt and effect bounds. The rejection-code table distinguishes them.
Reproduce with the exact document/IR, plugin catalog and explicit context, using
appropriate pre-block state, message order and height. For multi-block rehearsals,
carry postState with --prior-result and advance exactly one height each time.
Use messages: [] for intervening empty blocks. The
approval tutorial demonstrates this progression.
Continuation files are editable, unauthenticated test fixtures. They are not node backups or trusted state exports and must not be imported into retained stores. Dry-run does not calculate the predecessor root for your next fixture.
Budget for the actual application
Section titled “Budget for the actual application”The 64-KiB host message limit is not a promise that every 64-KiB command can produce every event or fan-out. Events add metadata; copied values, hashes, expressions and derived dispatch consume bounded work.
The default per-cascade work allowance is 1,048,576 units; the shared block
allowance is 33,554,432. These are deterministic accounting units, not CPU
milliseconds. For the reference source.v1 near-limit raw-body tee, 65,369 body
bytes cost 392,650 cascade units and 196,278 shared units. The binding-work
allowance alone could cover 170 such cascades; host block-byte/message-count
limits and other work may permit fewer. In particular, chapter 1’s 1-MiB block
limit cannot fit 170 such commands. This exact body boundary depends on
metadata length; do not generalize it to every topic or sender layout.
An original source event with no subscribers does not incur shared event-decode work. An unbound source is not rejected merely because binding work is exhausted. Derived dispatch and preparation still consume shared work, even for an unbound target. Rejected attempts do not refund work, and the block budget resets for the next block.
Before fixing your profile, test typical and maximum payloads, longest paths, fan-out, false conditions, failing targets and crowded blocks. Include the trace itself: receipts are bounded too. Increasing one limit cannot establish that all other limits accommodate the workload. Limits are part of the committed program, not node-local knobs you can raise on a busy server.
External effects are a separate boundary
Section titled “External effects are a separate boundary”A cascade can atomically commit an effect intent with its state changes. The external HTTP service, object store or other receiver does not participate in that state transaction. Delivery is at-least-once; implement receiver-side idempotency using the effect identity, observe outcomes and plan retries.
The procurement recipe’s test effect is demonstration plumbing, not a
production webhook integration. Select a real installed executor, configure its
security and credentials outside consensus data, and qualify failure/restart
behavior. Dry-run prints intents; it never proves external delivery. See
webhook effects.
Changing a workflow
Section titled “Changing a workflow”Changing YAML is not hot reload. Sealing commits component settings and defaults, routes, quotas, generation heights and the binding program, including admission rules, their parameters and attachments, into a profile. Machine application versions and query subjects also affect identity. Even a seemingly additive bundle update can make a retained profile impossible to reconstruct.
For a chain created in governed mode, evolution requires an executable catalog that reproduces the necessary historical profiles, qualified target semantics, and the existing proposal/approval/readiness/activation protocol. Changed bindings require a new workflow generation; changed component configuration requires a new component generation. Changed rules, parameters or attachments change the binding program and need a new workflow generation too, for example an ADR-015 proposal that tightens a limit. Neither migrates incompatible stored genesis or state automatically.
Use the read-only preflight against candidate bundles:
./yano.sh appchain bindings profile-check \ --profiles retained-profiles.json --context context.json \ --plugins-directory /absolute/path/to/candidate/pluginsHere retained-profiles.json is a nonempty JSON array of canonical profile hex
strings, genesis first, including the historical/target profiles being checked.
Multiple profiles require the appropriate governed membership context. Follow
the full preflight contract
before constructing these inputs.
A running member reports the profile it currently executes through the
composite/active-profile-v1 query. For the Chapter 4 devnet application
(records-audit on port 7370), this writes a one-profile array:
curl -fsS -X POST \ "http://127.0.0.1:7370/api/v1/app-chain/chains/records-audit/query/composite/active-profile-v1" \ -H 'Content-Type: application/json' -d '{"paramsHex":""}' \ | jq '[.payloadHex]' > retained-profiles.jsonAdd an X-API-Key header if your node requires one for queries. The answer is
one member’s unproven view of the current profile only: compare it across
members and against your deployment records, and add the genesis and any
earlier profiles from those records yourself.
profile-check does not cross-check the context against the profiles or a
deployed chain. It uses context.json only to construct the candidate catalog,
so a context with another chain’s identity can still pass. Take the context
from the same trusted deployment records as the profiles.
Success means byte-exact profile reconstruction only. It does not prove replay,
binary semantic equivalence, migration safety, historical proof correctness or
effect-result compatibility. The current stock provider executes version 1.2.0,
and IR or receipts written before ADR-031.3 (versions 1.1.0 and 1.0.0) fail decode
with a “predates ADR-031.3” error. ADR-031.4 then changed the rule layout in
place without changing that version: IR whose admission rules were written
before it, and receipts that record a rule failure, fail decode with a
“predates ADR-031.4” error. Re-create such experimental chains from their YAML;
do not upgrade them in place or rewrite their identity markers. The
upgrade reference
steps through what profile-check does.
Before sharing a deployment
Section titled “Before sharing a deployment”- Archive the exact host/X artifacts, plugin catalog, YAML, IR, canonical
profiles, sealed
<app>.chainfile and deployment identity; keep key custody separate. - Test accepted and rejected cascades, authorization failures, replay and independent receipt proofs with your real command codecs.
- Compare finalized heights, roots and profile identities across members; exercise catch-up, restart and historical reads.
- Qualify expected peak payloads and throughput, including rejected attempts.
- Check product boundaries: derived writes do not automatically appear in every legacy projection, and derived audit commands are not independently signed message-attestation certificates.
- Keep backups and rehearse upgrades without modifying retained production state.
Return to the learning path, continue with the retained-chain upgrade reference, or author the same documents with guided forms in chapter 6.