API Reference

Complete reference for every public function, class, and field in 0.1.

beval.init(...)

Initialize the default BEval client. Call once at app startup.

beval.init(
    api_key: Optional[str] = None,
    *,
    api_url: Optional[str] = None,
    project_id: Optional[str] = None,
    default_model_id: Optional[str] = None,
    redact: Optional[Callable[[dict], dict]] = None,
    debug: bool = False,
    **kwargs,  # any Config field
) -> BevalClient

Returns a BevalClient but you rarely need it — the module-level beval.log, beval.flush, etc. target the default client.

beval.log(...)

Enqueue a log entry. Non-blocking. Returns True if enqueued, False if the SDK is not initialized.

beval.log(
    *,
    # Content
    input: Optional[str] = None,
    output: Optional[str] = None,
    thinking_trace: Optional[str] = None,
    extracted_json: Optional[dict] = None,
 
    # Identity
    kind: Union[LogKind, str] = "llm",
    name: Optional[str] = None,
    external_id: Optional[str] = None,
    project_id: Optional[str] = None,
 
    # Model + perf
    model_id: Optional[str] = None,
    model_version: Optional[str] = None,
    latency_ms: Optional[int] = None,
    tokens_in: Optional[int] = None,
    tokens_out: Optional[int] = None,
    cost_usd: Optional[float] = None,
 
    # Status
    status: Union[LogStatus, str] = "success",
    error_message: Optional[str] = None,
 
    # VLM
    image: Optional[Union[str, bytes]] = None,
    image_mime: str = "image/png",
 
    # Arbitrary metadata
    extra: Optional[dict] = None,
 
    # Server-side behavior
    run_deterministic: bool = True,
) -> bool

Field notes

• kind — one of llm, vlm, agent, embedding, ocr, completion. Defaults to llm. Auto-promoted to vlm if image is set. See Concepts → Kinds.
• name — short label shown in the dashboard list. Good for grouping (e.g. "rag-chain", "tool:search").
• external_id — your ID for joining back to your own system. Auto-populated inside @beval.trace.
• project_id — overrides the default project for this log only.
• status — "success" or "failure". Pair with error_message on failures.
• image — bytes, base64 string, data: URL, or http(s) URL. Goes into extra.image_data_url.
• extra — merged shallow over Config.extra from init().
• run_deterministic — if False, skip server-side deterministic verifiers on this log. Default True.

beval.wrap(client)

Auto-instrument an LLM SDK client.

beval.wrap(client: Any) -> Any

Detects the client by its top-level module (openai or anthropic) and patches the relevant methods. Raises TypeError for unsupported clients.

The returned object is the same client — patched in-place. Use it or the original; they’re the same object.

@beval.trace

Function decorator. Logs calls as a BEval entry.

@beval.trace
def fn(...): ...
 
@beval.trace(
    name: Optional[str] = None,
    kind: str = "agent",
    capture_args: bool = True,
    capture_return: bool = True,
)
def fn(...): ...

• name — defaults to f"{fn.__module__}.{fn.__qualname__}".
• capture_args / capture_return — toggle to skip serializing big inputs/outputs.

Works on both def and async def functions. Exceptions are logged with status="failure" and then re-raised.

beval.flush(timeout=5.0)

Wait up to timeout seconds for the in-memory queue to drain.

beval.flush(timeout: float = 5.0) -> None

Safe to call any time. No-op if the SDK isn’t initialized.

beval.shutdown()

Drain the queue and close the HTTP client. Called automatically at interpreter exit via atexit.

beval.shutdown() -> None

beval.LogKind

class LogKind(str, Enum):
    llm = "llm"
    vlm = "vlm"
    agent = "agent"
    embedding = "embedding"
    ocr = "ocr"
    completion = "completion"

Use the enum or the string — both work in kind=.

beval.LogStatus

class LogStatus(str, Enum):
    success = "success"
    failure = "failure"

beval.Config

The dataclass that backs init().

beval.BevalClient

The underlying client object. Normally accessed via module-level functions, but you can hold one explicitly for advanced multi-config setups:

from beval import BevalClient, Config
 
prod_client = BevalClient(Config(api_key="bv_prod", project_id="..."))
staging_client = BevalClient(Config(api_key="bv_staging", project_id="..."))
 
prod_client.log(input="...", output="...")
prod_client.flush()

Methods: log(...), flush(timeout=5.0), shutdown().