Hosting the bounded Clerk workflow
celldega.clerk.run_turn implements the bounded interpretation/evidence loop without
widget traits, a browser transport or a model client. It consumes an initial model
reply and yields plain-data events. The notebook's Clerk._work is one adapter;
celldega-app can provide another using its renderer and Python worker.
The POC scope is: explain the current view, inspect evidence, show a cited view, return to the user's saved view, export and reopen. Reports, the saved-view schema and the workflow can be reused separately. Batch annotation, Make-tier execution and video remain outside this acceptance path.
Engine protocol v1
celldega.clerk.engine.PROTOCOL_VERSION identifies the event protocol. Start with
run_turn(answer, max_rounds=6). Resume a pause using its actions as
pending_actions in a new invocation. The engine suppresses repeated consecutive
actions and stops on missing/failed completion before gathering further evidence.
Event kind |
Host responsibility | Value sent back to the generator |
|---|---|---|
answer |
Display/store text and cited views |
None |
findings |
Store proposals separately from accepted scientist annotations | None |
error |
Display/store text |
None |
pause |
Store actions; offer explicit continuation with a fresh budget |
None |
apply |
Validate actions, execute supported API operations, await completion |
{changes: [...], reads: {...}} or {error: "..."} |
observe |
Capture coordinated Snapshots, run requested reads, preserve evidence | A host-defined plain-data evidence envelope |
follow_up |
Send that evidence to the chosen model backend | Next reply string |
An asynchronous application adapter can drive the synchronous generator without blocking its transport:
from celldega.clerk import run_turn
workflow = run_turn(initial_reply)
response = None
while True:
try:
event = workflow.send(response)
except StopIteration:
break
response = None
if event["kind"] in {"apply", "observe", "follow_up"}:
response = await host.resolve(event)
else:
await host.publish(event)
host is supplied by the embedding application. It owns model/session lifetime,
validation of allowed operations, widget addressing, cancellation, lookups, UI and
durable storage. The engine owns interpretation of action/view blocks, round budgets,
duplicate suppression and the decision to continue. This keeps provider-specific
sessions out of the visualization API.
celldega-app integration points
The current sibling app has an Electron main-process obs_app, scoped window IDs,
renderer-held visualization instances and a persistent Python worker using JSON lines.
Use those boundaries rather than embedding notebook widgets:
- Update the pinned JS dependency and managed Python lock together to a release containing these APIs. The inspected app currently pins Celldega 0.24.2 and uses older low-level rendering entry points; this branch is not a drop-in app update.
- Keep each live visualization in the renderer's registry, addressed by
window_id. Main routes requests carryingrequest_id,window_idandscope_id; validate the current scope for every request because a window can change datasets. - Dispatch only supported operations to
get_state,apply_state,snapshotandrestore_snapshot. Return completion/errors with the same request ID. Save/export remains a JS feature and should not require installing the Python runtime. - For Clerk, keep a workflow in the existing Python worker. Route its
applyandobserverequests to the renderer through main; routefollow_upto the selected model backend. Keep images out of the globally broadcastobs_appchannels. - Bind a proposed action to the evidence's data identity. For example, use
restore_snapshot({...evidenceSnapshot, state: proposedChanges})to validate the dataset before applying the proposed settings. Capture the resulting view again; do not save the old evidence image as if it showed the new settings. - Persist saved-view envelopes and evidence records, not live widget/model objects.
Reopening first loads the dataset, constructs fresh visualizations and invokes
restore_snapshot. A scope ID routes linked windows; the content fingerprint validates their data. These are distinct responsibilities.
The standalone JS example exercises the renderer boundary without Electron. Engine unit tests exercise a host without widgets or model credentials. An app integration should additionally test window closure, cancellation, dataset switching during an action, and responses arriving after a view is replaced. No celldega-app files are changed by this branch.