inference_worker

Owned relays between provider inferences and the agent runtime.

The provider coordinator remains the sole owner of provider terminality. This worker owns the wait and forwards its ordered event stream without blocking the runtime actor.

Types

Worker

opaque </>

An owned inference relay. Its internals never escape the runtime.

pub opaque type Worker

Values

cancel

</>

pub fn cancel(worker: Worker) -> Nil

Cancel provider work owned by this relay. Repeated calls are harmless.

start

</>

pub fn start(
  provider_instance: provider.Provider,
  request: provider.InferenceRequest,
  notify: fn(provider.InferenceEvent) -> Nil,
) -> Worker

Start an inference relay and return before provider work begins.