Pool

llm_annotator.pool ¶

Live clients and annotators built from a validated client config.

Everything here acts on the world outside the process: it constructs provider clients, probes /health endpoints over HTTP and keeps a background thread that admits vLLM servers into a running pool. config holds the models and their validation and knows nothing about this module, so the dependency runs one way: pipeline -> pool -> config.

wait_for_servers ¶

wait_for_servers(
    base_urls: list[str],
    timeout: float,
    min_servers: int = 1,
) -> list[str]

Block until the requested number of vLLM servers answer /health.

Parameters:

Name	Type	Description	Default
`base_urls`	`list[str]`	vLLM base URLs (each ending in `/v1`).	required
`timeout`	`float`	Maximum number of seconds to wait for the pool as a whole.	required
`min_servers`	`int`	Number of ready servers required to continue.	`1`

Returns:

Type	Description
`list[str]`	The base URLs that answered, in the order they did.

Raises:

Type	Description
`TimeoutError`	If fewer than `min_servers` are reachable after `timeout`.

View source on GitHub: src/llm_annotator/pool.py lines 31–85

build_client ¶

build_client(
    client_config: ClientConfig, root: Path
) -> Client | list[Client]

Instantiate the client, or one client per server for a pool.

Parameters:

Name	Type	Description	Default
`client_config`	`ClientConfig`	The step's effective client configuration.	required
`root`	`Path`	Directory that relative paths and globs resolve against.	required

Returns:

Type	Description
`Client \| list[Client]`	A single client, or a list of clients when a pool is configured.

View source on GitHub: src/llm_annotator/pool.py lines 88–140

build_annotator ¶

build_annotator(
    client_config: ClientConfig,
    root: Path,
    verbose: bool = False,
) -> Annotator

Instantiate the annotator that drives a step's client.

A pool of servers yields a VLLMQueueAnnotator, with a background thread that admits the servers that are not ready yet; everything else yields a plain Annotator.

Parameters:

Name	Type	Description	Default
`client_config`	`ClientConfig`	The step's effective client configuration.	required
`root`	`Path`	Directory that relative paths and globs resolve against.	required
`verbose`	`bool`	Whether the annotator should log progress information.	`False`

Returns:

Type	Description
`Annotator`	The annotator, ready to run.

View source on GitHub: src/llm_annotator/pool.py lines 143–189