Pixeltable Observability & Monitoring with OpenTelemetry

What is Pixeltable Observability?

Pixeltable is declarative data infrastructure for multimodal AI apps. Images, video, audio, documents, and model calls live in tables, and a computed column runs its transform on insert and update. Pixeltable observability records that work with OpenTelemetry and sends it to SigNoz.

The Pixeltable OpenTelemetry bridge emits one trace per table operation. With span_level='debug' the trace contains the rows, each UDF call, and, when you add an LLM instrumentor, a child span for the model request:

pixeltable.insert
└─ pixeltable.store.insert_rows
   ├─ pixeltable.row
   ├─ pixeltable.udf.chat_completions
   │  └─ ChatCompletion
   └─ pixeltable.sa.insert_rows

UDF spans carry pxt.udf_path, pxt.column, pxt.resource_pool, and pxt.retries. pixeltable.store.write_column spans carry pxt.column. The model name, prompt, completion, and token counts are attributes of the LLM child span. SigNoz AI Observability reads the gen_ai.* attributes on that child span.

Prerequisites

• A SigNoz Cloud account with an active ingestion key, or a self-hosted SigNoz instance
• Python 3.11 or later
• An OpenAI API key if you run the example below (OPENAI_API_KEY)

Tested with Python 3.11, pixeltable 0.7.23, opentelemetry-instrumentation-pixeltable 0.2.0, and openinference-instrumentation-openai 0.1.69.

Monitor Pixeltable with OpenTelemetry

Call pxt_otel.init() once per process, before the first Pixeltable operation. It builds an OTLP/HTTP exporter and instruments Pixeltable. Add OpenAIInstrumentor so each openai.chat_completions cell gets a child span. For the Pixeltable API, see the Pixeltable documentation.

Step 1: Install the packages

Install Pixeltable with the OpenTelemetry extra, the openai client, and the OpenAI instrumentor.

pip install 'pixeltable[otel]' openai openinference-instrumentation-openai

pixeltable[otel] installs opentelemetry-instrumentation-pixeltable together with the OpenTelemetry SDK and the OTLP exporters. Neither Pixeltable nor the instrumentor installs openai, which chat_completions() calls.

Step 2: Point the exporter at SigNoz

Set the endpoint with environment variables, or pass it to init() in code. init() appends /v1/traces and /v1/metrics to the endpoint, so pass the base URL without a path.

export OTEL_SERVICE_NAME=pixeltable-app
export OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.<region>.signoz.cloud:443"
export OTEL_EXPORTER_OTLP_HEADERS="signoz-ingestion-key=<your-ingestion-key>"
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_SPAN_LEVEL=debug
export OPENAI_API_KEY=<your-openai-api-key>

pxt_otel.init(
    endpoint='https://ingest.<region>.signoz.cloud:443',
    headers='signoz-ingestion-key=<your-ingestion-key>',
    service_name='pixeltable-app',
    span_level='debug',
)

Step 3: Initialize the bridge and insert a row

Initialize the bridge, instrument OpenAI, and insert a row. If you used the Code tab in Step 2, replace pxt_otel.init() with your init() call.

import opentelemetry.instrumentation.pixeltable as pxt_otel
from openinference.instrumentation.openai import OpenAIInstrumentor
from opentelemetry import trace
 
pxt_otel.init()
OpenAIInstrumentor().instrument()
 
import pixeltable as pxt
from pixeltable.functions import openai
 
table = pxt.create_table('articles', {'id': pxt.Int, 'body': pxt.String}, if_exists='replace')
table.add_computed_column(
    summary=openai.chat_completions(
        messages=[{'role': 'user', 'content': 'Summarize in one sentence: ' + table.body}],
        model='gpt-4o-mini',
    ).choices[0].message.content
)
table.insert([{'id': 1, 'body': 'Pixeltable is declarative data infrastructure for multimodal AI apps.'}])
trace.get_tracer_provider().force_flush()

force_flush() sends the batched spans right away. Without it, the SDK sends them when the process exits normally. Metrics go out on a periodic reader, about once a minute. To send them right away, flush the meter provider as well:

from opentelemetry import metrics
 
metrics.get_meter_provider().force_flush()

init() reuses a tracer provider that your application already registered.

OpenInference sets llm.model_name, llm.provider, and llm.token_count.* on the ChatCompletion span. The default gen_ai.llm group in Attribute Mapping copies them to the gen_ai.* attributes that AI Observability reads. To emit gen_ai.* from the process, set OPENINFERENCE_ENABLE_GENAI_SEMCONV=true.

Validate

Open the Traces Explorer and filter on the service name and the operation:

service.name = 'pixeltable-app' AND name = 'pixeltable.insert'

Open the trace. The waterfall shows pixeltable.udf.chat_completions under pixeltable.store.insert_rows, and ChatCompletion under the UDF span. The prompt and completion are attributes on ChatCompletion.

Pixeltable also exports metrics when an endpoint is set. In the Metrics Explorer, query:

• pixeltable.udf.input_tokens and pixeltable.udf.output_tokens, with attribute pxt.model (and pxt.udf_path)
• pixeltable.udf.calls and pixeltable.udf.latency
• pixeltable.rows.written, with attribute pxt.table

The metric scope name is pixeltable. The same insert recorded input and output tokens on pxt.model=gpt-4o-mini.

collect() writes a pixeltable.collect trace and does not call the model again. Computed values are stored. New LLM spans appear on insert, on update, and when add_computed_column backfills rows that already exist.

View in AI Observability

AI Observability shows the cost, token usage, latency, and errors of the LLM calls that Pixeltable computed columns make. In SigNoz, open AI Observability, and select your service in the service_name filter.

AI Observability counts the ChatCompletion span as an LLM call. After one insert, it shows 1 LLM call for gpt-4o-mini-2024-07-18, the model name that OpenAI returns, with its input and output tokens. SigNoz calculates the cost from its built-in price for that model. One call costs a fraction of a cent, so the Total cost tile rounds it to $0, and the Cost breakdown panel shows the input and output split. For a model without a price, cost reads $0 until you add a model pricing rule for the model name. Time to first token reads 0s for a non-streaming call.

To query single traces, open the Explorer tab. Trace View shows one row for each trace, with totals such as tokens, cost, and call counts. You can filter, aggregate, and sort by these totals, for example trace.total_tokens > 1000. For details, see AI Observability Explorer.

If the panels stay empty, make sure that your spans carry the attributes in Telemetry Requirements. To map other attribute names, use Attribute Mapping.

Limitations

• Every operation is a trace. create_table, drop_table, add_computed_column, and collect each start their own trace. span_level='debug' adds per-row spans, per-UDF spans, and finer catalog and store spans such as pixeltable.catalog.*. At info, those per-row and per-UDF spans are dropped, and the LLM span stays in the insert trace without a pixeltable.udf.chat_completions parent.
• ChatCompletion carries the prompt and the completion, so both leave the process. Pixeltable log export stays off unless you pass logs=True. It then exports the records of the pixeltable loggers, such as catalog and scheduler messages. An OpenAI error record shows the API key masked, except for its last 4 characters.

Failed to export spans batch code: 401, reason: Unauthorized

Related integrations

• DSPy observability with OpenTelemetry for module, chain, and tool spans from DSPy programs
• OpenAI monitoring for OpenAI API calls outside Pixeltable
• LangChain and LangGraph observability for chains, agents, and tool calls
• LlamaIndex observability for RAG queries and retrievers
• AI Observability telemetry requirements for the gen_ai.* attributes the dashboards read

Browse all LLM observability integrations to instrument the rest of the stack.