Skip to content

OpenAI-Compatible Provider

AccumulatorStreamingMixin

Generic streaming hot path using extract_delta + process_delta.

This follows the accumulator pattern described in plugins/docs/provider-plugins.md.

OpenAICompatibleProvider

OpenAICompatibleProvider()

Bases: AccumulatorStreamingMixin, ProviderPlugin

OpenAI-compatible chat-completions provider.

Designed to work with OpenAI-like endpoints, including Ollama's OpenAI-compatible API.

Source code in reference_openai_compatible_provider/provider.py
393
394
395
def __init__(self) -> None:
    self._inflight_lock = threading.Lock()
    self._inflight_requests: dict[str, _InflightRequest] = {}