Device execution

The GPU is the working centre of AOTX. Model execution, tokenisation, memory selection and runtime transitions operate on tables held in device memory. Keeping these operations together gives the runtime direct ownership of both a decision and the state that decision changes. A client submits work to that environment and observes the resulting state.

The host provides the connections that make this usable on a complete machine. It allocates resources, registers shared memory, loads device modules and launches execution graphs. Local clients connect input and display surfaces; a separate gateway handles network transport and authentication. Disk processes read model assets, feed requested file data and maintain the recovery record.

Execution advances through finite CUDA graphs. A tick is one execution of the tick graph, whose dependencies establish the order between admitted input, scheduling, device work and publication. Device queues and parameters determine the batch processed within that structure. Tick and sequence identifiers give recorded transitions a stable order for recovery.

This separates the lifetime of an activity from the lifetime of its stored state. A shared conversation can remain present while it has no active work. When input is ready, the service leases an execution slot within the available capacity; the conversation's identity and memory remain after that slot is released.

Each batch has a resource envelope: slots, context space, working memory and publication capacity. Policy operations participate in bounded execution steps as well. Persistent policy state can therefore remain resident between operations, without requiring a kernel to occupy the device indefinitely.

Tick order describes the system's progress. Elapsed time describes the cost of that progress for a particular model, batch and device.

A state change becomes recoverable through an ordered publication path. Device producers first write complete record contents, then publish their completion. A flush groups records into blocks in pinned host memory, where the disk drain can consume them and write journal segments.

There are three useful positions to observe:

• Accepted: the runtime has admitted an operation.
• Committed on the device: the authoritative transition has taken place.
• Durable on disk: the corresponding recovery boundary has been acknowledged.

These positions can advance at different times. A reply can become visible while its persistent representation is still being written. An application that needs a recoverable outcome uses the durable receipt, which identifies the completed storage boundary.

Finite buffers make storage pressure explicit. Before publication, the runtime accounts for the space a complete record requires. If that capacity is unavailable, affected work remains held under the protocol's credit rules. Device execution can finish its bounded step while the host drain catches up, rather than leaving a kernel waiting on a disk operation.

A usable model configuration includes its weights, tensor layout, tokenizer, message wrapper and termination rules. The loader verifies the selected file against a supported execution path. Media profiles add the encoders, projectors or specialist models required to turn their inputs into representations the selected model can use.

Resource admission considers the whole configuration. Weights share the device budget with execution caches, resident memory, media assets and runtime tables. The requested context and batch also consume capacity. A supported model can therefore be available while a particular request exceeds the resources currently admitted to it.