API reference¶
Generated reference documentation for each package. The pages are rebuilt and published with this
site from main.
| Language | Reference | Generated by |
|---|---|---|
| TypeScript | TypeScript API | TypeDoc |
| Rust | Rust API · docs.rs | rustdoc |
| Go | pkg.go.dev | Go doc comments |
| Python | below | mkdocstrings |
| Java | Java API | Javadoc |
Build them locally with make docs (per language: make -C <lang> docs).
Python¶
Grow Codes SDK for Grafana Tempo.
from grow_grafana_tempo import TempoClient, TraceQL, init_tracing client = TempoClient.from_env() # doctest: +SKIP client.query.search(TraceQL.span({"resource.service.name": "api"})) # doctest: +SKIP
AsyncBaseTraceQueryClient ¶
Async :class:BaseTraceQueryClient.
query_trace_id ¶
The id wait_for_trace passes to get_trace; override to keep native ids.
AsyncHttpTransport ¶
Asynchronous transport (one per async client); options as for :class:HttpTransport.
Cancelling the calling task raises :class:CancelledError (an asyncio.CancelledError
subclass) so asyncio's cancellation machinery keeps working. Every call after
:meth:aclose raises :class:CancelledError ("client is closed").
AsyncTempoClient ¶
Asynchronous Grafana Tempo client; use async with or call :meth:aclose.
Same methods as :class:TempoClient, awaited; client.query is an
:class:AsyncTempoQueryClient. Cancel a call by cancelling its task: the SDK raises
:class:CancelledError, which is also an :class:asyncio.CancelledError.
AsyncTempoQueryClient ¶
Bases: AsyncBaseTraceQueryClient
Asynchronous :class:TempoQueryClient (same methods, awaited). Cancel a call by
cancelling its task.
aclose
async
¶
Close the underlying client when this query client owns it (idempotent).
from_env
classmethod
¶
An owned async query client configured from the environment.
get_dependencies
async
¶
get_dependencies(*, start: datetime | None = None, end: datetime | None = None) -> list[DependencyLink]
Always :class:UnsupportedError: Tempo has no dependency API.
get_trace
async
¶
get_trace(trace_id: str, *, start: _TimeLike | None = None, end: _TimeLike | None = None) -> Trace | None
See :meth:TempoQueryClient.get_trace.
list_operations
async
¶
list_operations(service: str, *, start: _TimeLike | None = None, end: _TimeLike | None = None) -> list[str]
See :meth:TempoQueryClient.list_operations.
list_services
async
¶
See :meth:TempoQueryClient.list_services.
search
async
¶
search(traceql: str | TraceQL | None = None, *, start: _TimeLike | None = None, end: _TimeLike | None = None, limit: int | None = None, spss: int | None = None) -> TempoSearchResult
See :meth:TempoQueryClient.search.
search_tag_values
async
¶
search_tag_values(tag: str, *, query: str | TraceQL | None = None, start: _TimeLike | None = None, end: _TimeLike | None = None) -> list[TagValue]
See :meth:TempoQueryClient.search_tag_values.
search_tags
async
¶
search_tags(*, scope: TagScope | None = None, query: str | TraceQL | None = None, start: _TimeLike | None = None, end: _TimeLike | None = None) -> dict[str, list[str]]
See :meth:TempoQueryClient.search_tags.
search_traces
async
¶
See :meth:TempoQueryClient.search_traces.
traceql_metrics
async
¶
traceql_metrics(query: str | TraceQL, *, start: _TimeLike | None = None, end: _TimeLike | None = None, since_s: float | timedelta | None = None, step_s: float | timedelta | None = None, exemplars: int | None = None) -> TraceQLMetricsResult
See :meth:TempoQueryClient.traceql_metrics.
AsyncTraceQueryClient ¶
Bases: Protocol
Asynchronous trace query API (same methods, awaited).
AuthenticationError ¶
Bases: TempoError
Credentials missing or rejected (HTTP 401).
BaseConfig
dataclass
¶
Common, immutable client settings. Durations are float seconds (timedelta accepted).
Service configs subclass this (@dataclass(frozen=True, repr=False)), override url
with the service default, add their own fields (Secret for credentials), extend
:meth:_env_values and :meth:validate, and list vendor variables in ENV_VENDOR.
BaseTraceQueryClient ¶
Base for service query clients; every method raises UnsupportedError until overridden
(wait_for_trace uses :func:wait_for_trace on top of get_trace, passing the id
through :meth:query_trace_id).
query_trace_id ¶
The id wait_for_trace passes to get_trace; override to keep native ids.
BuildInfo
dataclass
¶
Build information from /api/status/buildinfo (empty strings for unset fields).
CancelToken ¶
Thread-safe cancellation flag passed to sync calls as cancel=.
Cancelling interrupts an in-flight request (the call returns immediately with
:class:CancelledError; the abandoned HTTP exchange finishes in the background) and any
retry backoff sleep.
CancelledError ¶
Bases: TempoError, CancelledError
The caller cancelled the operation (task cancel, :class:CancelToken, client close).
Also an :class:asyncio.CancelledError so asyncio keeps treating the task as cancelled.
ConfigError ¶
Bases: TempoError
Invalid or missing configuration.
ConflictError ¶
Bases: TempoError
Conflict or failed precondition (HTTP 409, 412, CAS mismatch).
EnvReader ¶
Reads GROW_<KEY_UPPER>_<NAME> and vendor variables from an injectable mapping.
Every getter returns None when no variable is set (empty strings count as unset). A
present but invalid value raises :class:ConfigError that names the variable, never its
value.
ErrorCode ¶
Bases: str, Enum
Canonical error codes; the value is the wire form used by the conformance suite.
ErrorInfo
dataclass
¶
Passed to Hooks.on_error; error is the (redacted) SDK error.
HealthStatus
dataclass
¶
Result of health(); status is ok/degraded/unhealthy/unreachable or a vendor word.
Hooks
dataclass
¶
Optional callbacks invoked on every attempt. Exceptions raised by hooks are ignored.
HttpTransport ¶
Synchronous transport (one per client).
Service options (all optional):
request_id_headers: vendor request id headers, checked before :data:REQUEST_ID_HEADERS.retry_after_headers: headers read (in order) for the delay a 429/503 response asks for; default("retry-after",), e.g.("retry-after", "x-ratelimit-reset"). See :func:retry_after_from_headers.error_mapper:mapper(error, response)returns a service-specific error for an HTTP error response (orNoneto keep the default). The mapped error is whatHooks.on_error, the retry planner (retry_after) and the caller see; whether a status is retried stays the SPEC rule. Exceptions raised by the mapper are logged and ignored.
Every call after :meth:close raises :class:CancelledError ("client is closed").
LoggerLike ¶
Bases: Protocol
Structured logger interface accepted in configuration (SPEC Logger).
MetricExemplar
dataclass
¶
An exemplar span behind a series (labels include trace:id).
MetricSample
dataclass
¶
One sample of a TraceQL metrics series (Tempo omits zero values; they read as 0).
MetricSeries
dataclass
¶
One series: labels (__name__, by() attributes, quantile p), samples,
exemplars.
NotFoundError ¶
Bases: TempoError
Missing record (HTTP 404).
PermissionDeniedError ¶
Bases: TempoError
Authenticated but not allowed (HTTP 403).
RateLimitError ¶
Bases: TempoError
Too many requests (HTTP 429); retry_after holds the requested delay when known.
Request
dataclass
¶
One logical API call (possibly several attempts).
path is joined to the base URL ({name} style values must already be encoded; use
:func:urllib.parse.quote); an absolute http:///https:// URL is used as is (for
pagination links the service returns; the default headers, config.headers included,
are sent there too, so only pass URLs of the same service). json is encoded when not
None; content sends raw bytes. idempotent overrides the method rule; an
idempotency_key sends the Idempotency-Key header and marks the request idempotent.
raise_for_status=False returns error responses instead of raising (retries still
apply). Statuses in allow_status are expected answers (a 404 that carries data, a
health endpoint's 503): the response is returned at once, like a success, without retries
or on_error.
RequestInfo
dataclass
¶
Passed to Hooks.on_request; url and headers are redacted.
ResourceRegistry ¶
Holds :class:Stoppable objects; :meth:stop_all stops each once, ignoring errors.
register ¶
Track item; when the registry is already closed the item is stopped now.
Response
dataclass
¶
A completed HTTP response (body fully read).
ResponseInfo
dataclass
¶
Passed to Hooks.on_response; headers are the redacted response headers (rate
limit headers and the like stay readable).
RetryPolicy
dataclass
¶
Retry settings. Durations are seconds (datetime.timedelta is accepted).
Backoff for retry n (0-based) is full jitter:
random(0, min(max_backoff, initial_backoff * multiplier ** n)).
random returns a float in [0, 1); inject it in tests for deterministic sleeps.
Secret ¶
Wraps a credential so it never shows up in str/repr/logs.
s = Secret("hunter2") str(s), s.reveal() ('[REDACTED]', 'hunter2')
ServiceStats
dataclass
¶
Spans and errors of one service in a search hit.
Span
dataclass
¶
A finished span as returned by a query API.
Ids are normalized on construction (an all-zero or empty parent_span_id becomes
None); a negative duration_micros (the default) is computed from start and end.
A subclass with normalize_ids = False keeps native ids verbatim (an empty
parent_span_id still becomes None); link_class is used by :meth:from_json.
SpanBatchOptions
dataclass
¶
BatchSpanProcessor settings; durations are float seconds (timedelta accepted).
SpanLink
dataclass
¶
A link to another span. Set normalize_ids = False on a subclass for native ids.
SpanNode
dataclass
¶
A span with its children (sorted like :func:sort_spans).
SpanStatus
dataclass
¶
from_json
classmethod
¶
Parse {"code", "message"?} (a bare code string is accepted too).
TagValue
dataclass
¶
One tag value. value is always text; type tells how to read it (string,
int, keyword …).
TempoClient ¶
Synchronous Grafana Tempo client (query API, port 3200). Thread-safe; use as a context
manager or call :meth:close.
client.query gives the query API (get_trace, search, search_tags,
search_tag_values, traceql_metrics and the SPEC TraceQueryClient methods). Every
I/O method accepts cancel= (a :class:CancelToken) to abort from another thread.
with TempoClient.from_env() as client: # doctest: +SKIP ... trace = client.query.get_trace("5b8efff798038103d269b633813fc60c")
build_info ¶
GET /api/status/buildinfo (see :meth:TempoQueryClient.build_info).
from_env
classmethod
¶
Build a client from GROW_GRAFANA_TEMPO_* (then TEMPO_URL,
TEMPO_TENANT_ID); keyword overrides win.
health ¶
Readiness: GET /ready (no tenant needed). ready → healthy ok; HTTP 503
while Tempo starts → unhealthy with details["reason"]; network errors →
unreachable. Never raises for an unhealthy or unreachable service.
ping ¶
Tenant-aware round trip GET /api/v2/search/tags?scope=intrinsic; returns the
latency in milliseconds. A multi-tenant Tempo without a tenant →
:class:AuthenticationError.
ready ¶
GET /ready → True when ready, False while starting; raises on network
errors.
request ¶
Send a raw request through the client's transport (retries, hooks, redaction).
server_version ¶
Server version ("3.1.1") from /api/status/buildinfo, falling back to the text
of /status/version; :class:UnsupportedError when neither reports one.
TempoConfig
dataclass
¶
Bases: BaseConfig
Immutable settings for :class:~grow_grafana_tempo.TempoClient (query API, port 3200).
Common fields (url, timeouts, retry, tls, proxy, headers,
user_agent_suffix, hooks, logger) come from :class:BaseConfig. Durations are
float seconds; datetime.timedelta is accepted.
Tempo has no authentication of its own. token (sent as Authorization: Bearer) and
headers exist for an authenticating reverse proxy in front of Tempo; both are redacted
in repr(), logs and error messages.
Environment (from_env), precedence explicit > GROW_GRAFANA_TEMPO_<NAME> > vendor >
default: URL (vendor TEMPO_URL, GROW_TEMPO_URL), TENANT_ID
(TEMPO_TENANT_ID, GROW_TEMPO_TENANT_ID), TOKEN (GROW_TEMPO_TOKEN),
CONNECT_TIMEOUT, REQUEST_TIMEOUT, TOTAL_TIMEOUT, MAX_RETRIES, CA_CERT,
CLIENT_CERT, CLIENT_KEY, INSECURE_SKIP_VERIFY, PROXY (vendor aliases
GROW_TEMPO_<NAME>).
TempoQueryClient ¶
Bases: BaseTraceQueryClient
Synchronous client for Tempo's HTTP query API (port 3200).
Get one from client.query, or build one from a config (it then owns its client and
:meth:close closes it). The tenant (X-Scope-OrgID) comes from the config. Every method
accepts cancel= (a :class:CancelToken).
build_info ¶
GET /api/status/buildinfo → version, revision, branch, build user/date, Go.
from_env
classmethod
¶
An owned query client configured from GROW_GRAFANA_TEMPO_* / TEMPO_*.
get_dependencies ¶
get_dependencies(*, start: datetime | None = None, end: datetime | None = None) -> list[DependencyLink]
Always :class:UnsupportedError: Tempo has no dependency API.
get_trace ¶
get_trace(trace_id: str, *, start: _TimeLike | None = None, end: _TimeLike | None = None, cancel: CancelToken | None = None) -> Trace | None
Fetch a trace by id: GET /api/v2/traces/{id} (v1 /api/traces/{id} when the
v2 route is missing). None when the trace does not exist or belongs to another tenant.
start/end narrow the backend blocks searched (recent data is returned anyway).
:class:ValidationError for an invalid id.
list_operations ¶
list_operations(service: str, *, start: _TimeLike | None = None, end: _TimeLike | None = None, cancel: CancelToken | None = None) -> list[str]
Span names of one service (name values filtered by the service), sorted.
list_services ¶
list_services(*, start: _TimeLike | None = None, end: _TimeLike | None = None, cancel: CancelToken | None = None) -> list[str]
Service names (resource.service.name values) seen recently or in the window,
sorted.
ready ¶
GET /ready: True when Tempo answers ready, False while it starts (HTTP
503) or reports another state. Raises on network errors (unlike health()).
search ¶
search(traceql: str | TraceQL | None = None, *, start: _TimeLike | None = None, end: _TimeLike | None = None, limit: int | None = None, spss: int | None = None, cancel: CancelToken | None = None) -> TempoSearchResult
TraceQL search: GET /api/search?q=…&start&end&limit&spss (no q without a
query). Invalid TraceQL → :class:ValidationError (HTTP 400).
search_tag_values ¶
search_tag_values(tag: str, *, query: str | TraceQL | None = None, start: _TimeLike | None = None, end: _TimeLike | None = None, cancel: CancelToken | None = None) -> list[TagValue]
Values of one tag: GET /api/v2/search/tag/{tag}/values. tag is a TraceQL
identifier (resource.service.name, span.http.response.status_code, name,
status, span.grow weird.key); an unscoped name is a :class:ValidationError.
search_tags ¶
search_tags(*, scope: TagScope | None = None, query: str | TraceQL | None = None, start: _TimeLike | None = None, end: _TimeLike | None = None, cancel: CancelToken | None = None) -> dict[str, list[str]]
Tag names per scope: GET /api/v2/search/tags → {"resource": [...], …}.
Without scope every scope that has data; query keeps tags of matching spans.
search_traces ¶
search_traces(query: TraceSearchQuery | None = None, *, cancel: CancelToken | None = None) -> list[TraceSummary]
SPEC §5 search mapped onto one TraceQL spanset (see :func:search_traces_traceql);
durations have millisecond precision.
traceql_metrics ¶
traceql_metrics(query: str | TraceQL, *, start: _TimeLike | None = None, end: _TimeLike | None = None, since_s: float | timedelta | None = None, step_s: float | timedelta | None = None, exemplars: int | None = None, cancel: CancelToken | None = None) -> TraceQLMetricsResult
TraceQL metrics over a range: GET /api/metrics/query_range (e.g.
{ resource.service.name = "api" } | rate()). since_s is a window ending now
instead of start/end. Zero samples omitted by Tempo read as 0. A query without a
metrics pipeline → :class:ValidationError.
TempoSearchResult
dataclass
¶
Result of search: traces plus query statistics (inspectedBytes …) as numbers.
TempoSearchSpan
dataclass
¶
One span of a search spanset. name is set only when the query selects it.
TempoSearchTrace
dataclass
¶
One trace found by search. trace_id is padded back to 32 hex characters;
duration_micros has millisecond precision (Tempo reports whole milliseconds).
TempoSpanSet
dataclass
¶
Spans of one trace that matched; matched may exceed len(spans) (see spss).
TimeoutError ¶
Bases: TempoError
A connect, request or total deadline was exceeded (also HTTP 408, 504).
TlsConfig
dataclass
¶
TLS settings. insecure_skip_verify is for tests only and logs a warning once.
Trace
dataclass
¶
All spans of one trace, sorted by start time then span id.
A subclass with normalize_ids = False keeps a native trace id verbatim; span_class
is used by :meth:from_json.
TraceQL ¶
Immutable TraceQL builder: span-set filters, structural operators and select().
Field names are validated (unknown scopes or intrinsics raise :class:ValidationError,
because some Tempo endpoints silently ignore an invalid query); attribute names with spaces
or special characters are quoted after the scope (span."grow weird.key"); the bare
intrinsic duration is written as span:duration. Every method returns a new builder.
str(query) renders the TraceQL text.
ancestor ¶
self << other: spans of other above self at any depth.
and_ ¶
Add && <field> <op> <value> to the last span-set filter.
descendant ¶
self >> other: spans of other below self at any depth.
field
staticmethod
¶
Render a field name: validate the scope or intrinsic and quote odd attribute names.
or_ ¶
Add || <field> <op> <value> to the last span-set filter.
quote
staticmethod
¶
Quote text as a TraceQL string literal (\, " and control characters
escaped; Unicode passes through).
select ¶
Add fields to | select(…) so search returns them on every matched span.
span
classmethod
¶
Start a span-set filter; each filters entry becomes <field> = <value> joined
with && in insertion order. TraceQL.span() alone matches every span ({}).
structural ¶
Join other's span-set filters with a span-set operator: > (child), >>
(descendant), < (parent), << (ancestor), ~ (sibling), && / ||
(both / either). other's select() is ignored.
value
staticmethod
¶
Render value for field (a name already rendered by :meth:field).
where
classmethod
¶
Start a span-set filter with one condition: { <field> <op> <value> }.
TraceQLMetricsResult
dataclass
¶
Result of traceql_metrics; step_ms is the step Tempo used, when reported.
TraceQueryClient ¶
Bases: Protocol
Synchronous trace query API.
TraceSearchQuery
dataclass
¶
matches ¶
Client-side filter: a span must match service/operation/tags; duration and time bounds apply to the whole trace.
TraceSummary
dataclass
¶
TracingASGIMiddleware ¶
ASGI middleware: one SERVER span per HTTP request, parent taken from request headers.
Non-HTTP scopes (lifespan, websocket) and excluded_paths pass through untraced.
TracingConfig
dataclass
¶
Settings for :func:init_tracing_with. repr redacts credential headers.
protocol and endpoint are interpreted by the service's exporter factory; the
template default protocol is http/protobuf. propagators beyond tracecontext,
baggage, b3 and b3multi need a factory from the service package.
protocol and propagators remember whether the caller set them: left out (or passed
as None) they read as :data:DEFAULT_TRACE_PROTOCOL / :data:DEFAULT_TRACE_PROPAGATORS
exactly as before, and :attr:protocol_set / :attr:propagators_set are False so a
service default can apply instead. The marker travels with the value, so
:meth:with_changes, dataclasses.replace and TracingConfig(**fields) keep an unset
field unset; passing the default value explicitly counts as set.
TracingHandle ¶
TracingWSGIMiddleware ¶
WSGI middleware: one SERVER span per request; ends when the response is closed.
UnavailableError ¶
Bases: TempoError
Network error or server unavailable (HTTP 500, 502, 503).
UnsupportedError ¶
Bases: TempoError
The service or deployment does not offer this capability (HTTP 501, missing API).
ValidationError ¶
Bases: TempoError
Bad input (HTTP 400, 413, 422, other 4xx, client-side validation).
add_event ¶
Add an event to the active span (no-op without one).
apoll_until
async
¶
apoll_until(fn: Callable[[], Awaitable[T | None]], *, timeout_s: float = 30.0, interval_s: float = 1.0, predicate: Callable[[T], bool] | None = None, operation: str = 'poll', clock: Callable[[], float] = time.monotonic) -> T
Async :func:poll_until; fn is an async callable.
async_httpx_event_hooks ¶
async_httpx_event_hooks(*, tracer: Tracer | None = None, propagator: TextMapPropagator | None = None) -> dict[str, list[Callable[..., Any]]]
event_hooks for httpx.AsyncClient (same behaviour as :func:httpx_event_hooks).
async_wait_for_trace
async
¶
async_wait_for_trace(client: AsyncTraceQueryClient, trace_id: str, *, timeout_s: float | timedelta = 30.0, interval_s: float | timedelta = 1.0, min_spans: int = 1, normalize: Callable[[str], str] | None = normalize_trace_id) -> Trace
Async form of :func:wait_for_trace.
build_propagator ¶
build_propagator(names: Sequence[str], factories: Mapping[str, PropagatorFactory] | None = None) -> TextMapPropagator
Composite propagator for names; factories adds or overrides propagators by name.
build_resource ¶
Resource with service.name/service.version/deployment.environment and extras.
The environment is written both as deployment.environment.name (current semantic
conventions) and as the older deployment.environment. Merged on top of the SDK default
resource (telemetry.sdk.*, OTEL_RESOURCE_ATTRIBUTES); explicit config values win.
build_sampler ¶
OpenTelemetry sampler for a TracingConfig.sampler name.
build_tree ¶
Parent/child forest; spans whose parent is missing become roots.
check_traceql ¶
Light client-side check of a TraceQL text; returns it trimmed.
The query must start with { and quotes, braces and parentheses must balance. Tempo's tag
endpoints ignore an invalid q and return unfiltered values, so the SDK refuses obviously
broken queries there (:class:ValidationError).
create_tempo_exporter ¶
The official OpenTelemetry OTLP exporter for a resolved config.
grpc → OTLP/gRPC (x-scope-orgid metadata; the grpc extra); http/protobuf and
native → OTLP/HTTP protobuf to <endpoint>/v1/traces. Applies headers, TLS,
compression and the export timeout.
current_span ¶
The active span (a non-recording invalid span when none is active).
current_span_id ¶
16-char lower-hex span id of the active span, or None.
current_trace_id ¶
32-char lower-hex trace id of the active span, or None.
datetime_to_micros ¶
Microseconds since the epoch (naive datetimes are UTC).
default_tracer ¶
Tracer of this package from the global provider.
dependencies_from_spans ¶
Service dependency links (parent service → child service, different services only).
error_for_code ¶
Return the error class for a code (ErrorCode member or wire value).
error_from_status ¶
error_from_status(status: int, operation: str, *, content: bytes = b'', headers: Headers | None = None, known_secrets: list[str] | None = None, retry_after: float | None = None, request_id_headers: Sequence[str] = ()) -> ServiceError
Map an HTTP error status to the SPEC 2.1 error with a redacted, truncated message.
request_id_headers are checked before the default :data:REQUEST_ID_HEADERS.
extract ¶
extract(carrier: Mapping[str, str], *, context: Context | None = None, propagator: TextMapPropagator | None = None) -> otel_context.Context
Read a context from carrier (exact key match; use :func:extract_headers for HTTP).
extract_headers ¶
extract_headers(headers: HeaderSource, *, context: Context | None = None, propagator: TextMapPropagator | None = None) -> otel_context.Context
Read a context from HTTP headers (case-insensitive; mapping or ASGI-style byte pairs).
from_any_value ¶
Convert an OTLP AnyValue to a plain value (None for an empty value).
intValue strings become int, doubleValue float, arrays lists, key/value lists
dicts; bytesValue stays base64 text.
from_key_values ¶
Convert an OTLP KeyValue list to a dict (entries with empty values are dropped).
get_logger ¶
The package logger (LOGGER_NAME), silent by default (NullHandler).
go_duration ¶
Format a positive duration (seconds or timedelta) as a Go duration (5s,
1500ms); :class:ValidationError otherwise.
httpx_event_hooks ¶
httpx_event_hooks(*, tracer: Tracer | None = None, propagator: TextMapPropagator | None = None) -> dict[str, list[Callable[..., Any]]]
event_hooks for httpx.Client: a CLIENT span per request with injected headers.
A request that fails before a response arrives (connection error) never reaches the
response hook, so its span is not exported; use :func:instrument_requests_session style
wrapping or with_span around the call when such failures must be traced.
init_tracing ¶
init_tracing(config: TracingConfig | None = None, *, tenant_id: str | None = None, env: Mapping[str, str] | None = None, propagator_factories: Mapping[str, PropagatorFactory] | None = None, span_processors: Sequence[SpanProcessor] = (), id_generator: IdGenerator | None = None, **fields: Any) -> TracingHandle
Set up OpenTelemetry tracing that exports to Grafana Tempo over OTLP.
Resource, sampler, batch span processor, propagators and (unless set_global=False)
global registration come from the template :func:init_tracing_with; settings are resolved
by :func:tempo_tracing_config (explicit > GROW_GRAFANA_TEMPO_OTLP_* > OTEL_* >
default). Call shutdown() before the process exits.
A multi-tenant Tempo answers an export without a tenant with HTTP 503 / gRPC
Unavailable, which the exporter retries until it drops the spans: set tenant_id
whenever Tempo runs with multi-tenancy.
handle = init_tracing(service_name="checkout", tenant_id="team-a") # doctest: +SKIP with_span("work", lambda span: do_work()) # doctest: +SKIP handle.shutdown() # doctest: +SKIP
init_tracing_with ¶
init_tracing_with(config: TracingConfig, exporter_factory: ExporterFactory, *, propagator_factories: Mapping[str, PropagatorFactory] | None = None, span_processors: Sequence[SpanProcessor] = (), id_generator: IdGenerator | None = None, sampler: Sampler | None = None) -> TracingHandle
Build a TracerProvider for config with a BatchSpanProcessor around the exporter.
Services wrap this in their init_tracing(config). propagator_factories lets a
service add datadog/sw8 style propagators; span_processors (extra processors,
e.g. a second exporter) are added after the batch processor; sampler replaces the one
built from config.sampler/sampler_ratio (remote or vendor samplers). With
config.set_global the provider and propagator become the
OpenTelemetry globals (the API allows one global provider per process: is_global on
the handle tells whether it took effect).
inject ¶
inject(carrier: MutableMapping[str, str] | None = None, *, context: Context | None = None, propagator: TextMapPropagator | None = None) -> MutableMapping[str, str]
Write the current (or given) context into carrier and return it.
Uses propagator or the global propagator (set by init_tracing_with).
inject_headers ¶
inject_headers(headers: MutableMapping[str, str] | None = None, *, context: Context | None = None, propagator: TextMapPropagator | None = None) -> MutableMapping[str, str]
Add propagation headers (traceparent, baggage …) to headers and return them.
instrument_requests_session ¶
instrument_requests_session(session: Session, *, tracer: Tracer | None = None, propagator: TextMapPropagator | None = None) -> requests.Session
Trace every request sent through session (CLIENT span, headers injected).
Wraps session.send once; calling it again on the same session is a no-op.
map_tempo_error ¶
Tempo-specific mapping on top of the SPEC status mapping.
A 401 no org id (multi-tenant Tempo without X-Scope-OrgID) becomes an
:class:AuthenticationError that names the missing tenant header; with unsupported_404 a
404 on a feature route becomes :class:UnsupportedError.
mark_span_error ¶
Set ERROR status on span and record the exception (when error is one).
micros_to_datetime ¶
Exact UTC datetime for microseconds since the epoch.
nanos_to_micros ¶
Integer microseconds from nanoseconds given as a decimal string or number (0 if unset).
normalize_headers ¶
Lower-cased header names → values; accepts mappings and (name, value) pairs incl. bytes.
normalize_span_id ¶
16-char lower-hex span id (left-padded); raises ValidationError when invalid.
normalize_trace_id ¶
32-char lower-hex trace id; 64-bit ids (16 hex chars, ints, 8 bytes) are left-padded.
Raises ValidationError for non-hex, too long or all-zero ids.
otlp_http_traces_url ¶
The OTLP/HTTP traces URL: a bare scheme://host:port gets /v1/traces appended.
otlp_id_to_hex ¶
Convert an id from trace-by-id JSON (base64 bytes) to lower-case hex of hex_length.
Ids that already are hex of that length are kept. Returns None for a missing, empty,
undecodable or all-zero id.
parse_bool ¶
Parse true/false/1/0/yes/no/on/off (case-insensitive); raise :class:ValueError.
parse_duration ¶
Parse a duration into seconds.
Accepts integer milliseconds ("1500" → 1.5) or Go-style units ("500ms", "10s",
"2m", "1h", "1m30s"). Raises :class:ValueError for anything else.
parse_key_value_list ¶
Parse key=value,key2=value2 (values percent-decoded) as in
OTEL_EXPORTER_OTLP_HEADERS; :class:ConfigError naming variable when malformed.
parse_retry_after ¶
Parse Retry-After as delta seconds or an HTTP-date; return seconds (>= 0) or None.
poll_until ¶
poll_until(fn: Callable[[], T | None], *, timeout_s: float = 30.0, interval_s: float = 1.0, predicate: Callable[[T], bool] | None = None, operation: str = 'poll', sleep: Callable[[float], None] = time.sleep, clock: Callable[[], float] = time.monotonic) -> T
Call fn every interval_s until it returns a ready value; return that value.
A value is ready when it is not None and predicate(value) is true (without a
predicate: not an empty container). Retryable SDK errors raised by fn are remembered
and polling continues; other errors propagate. Raises :class:TimeoutError after timeout_s.
record_exception ¶
Record error as an exception event on the active span (status unchanged).
redact_fields ¶
Recursively redact values whose key looks sensitive in nested mappings and lists.
redact_headers ¶
Copy of headers with sensitive values replaced by [REDACTED].
redact_text ¶
Replace every known secret value (length >= 4) in text with [REDACTED].
redact_url ¶
Redact user info and every query value (all values, not only sensitive ones).
Query names are kept so the URL stays useful in logs.
retry_after_from_headers ¶
retry_after_from_headers(headers: Headers | Mapping[str, str], names: Sequence[str] = RETRY_AFTER_HEADERS, *, now: Callable[[], float] = time.time) -> float | None
Server requested delay in seconds from the first of names that parses.
Retry-After is delta seconds or an HTTP-date. Other headers (X-RateLimit-Reset,
RateLimit-Reset …) may be delta seconds or a Unix timestamp in seconds or milliseconds;
timestamps are turned into the remaining delay.
search_traces_traceql ¶
The TraceQL search_traces sends: one spanset with resource.service.name,
name, unscoped .tag filters and trace:duration bounds in microseconds.
set_error ¶
Mark the active span as failed: ERROR status, plus an exception event for exceptions.
sort_spans ¶
Spans sorted by start time, then span id.
span_from_readable ¶
Convert a finished OpenTelemetry SDK span (e.g. from an in-memory exporter).
span_kind ¶
SpanKind from an enum member or a name ("server"); None → INTERNAL.
spans_from_resource_spans ¶
Flatten OTLP resourceSpans → scopeSpans → spans into model spans.
Ids base64 → hex, nanoseconds → microseconds, enums → names, service_name from the
resource service.name ("unknown_service" when absent). Spans with unusable ids are
skipped. Sorted by start time, then span id.
tempo_tracing_config ¶
tempo_tracing_config(config: TracingConfig | None = None, *, tenant_id: str | None = None, env: Mapping[str, str] | None = None, **fields: Any) -> TracingConfig
Resolve the Tempo tracing settings from explicit values and the environment.
Explicit values are config (a :class:TracingConfig, whose protocol counts as
explicit) and/or keyword fields of :class:TracingConfig (they win over config).
Unset values come from the environment, per field:
endpoint:GROW_GRAFANA_TEMPO_OTLP_ENDPOINT>OTEL_EXPORTER_OTLP_TRACES_ENDPOINT(verbatim) >OTEL_EXPORTER_OTLP_ENDPOINT(+/v1/tracesfor HTTP) > default (http://127.0.0.1:4318, or:4317forgrpc);protocol:GROW_GRAFANA_TEMPO_OTLP_PROTOCOL>OTEL_EXPORTER_OTLP_TRACES_PROTOCOLOTEL_EXPORTER_OTLP_PROTOCOL>http/protobuf;headers:OTEL_EXPORTER_OTLP_TRACES_HEADERS>OTEL_EXPORTER_OTLP_HEADERS, under the explicit headers;service_name:OTEL_SERVICE_NAME>service.nameinOTEL_RESOURCE_ATTRIBUTES;compression:GROW_GRAFANA_TEMPO_OTLP_COMPRESSION>OTEL_EXPORTER_OTLP_TRACES_COMPRESSION>OTEL_EXPORTER_OTLP_COMPRESSION(gzipornone; applies whilecompressionis the defaultgzip);sampler/sampler_ratio:OTEL_TRACES_SAMPLER/OTEL_TRACES_SAMPLER_ARG(while they hold the defaultsparentbased_always_on/1.0);- tenant:
tenant_id>GROW_GRAFANA_TEMPO_TENANT_ID>TEMPO_TENANT_ID>GROW_TEMPO_TENANT_ID; it becomes theX-Scope-OrgIDheader.
env defaults to os.environ. Raises :class:ConfigError naming an invalid variable
(never its value). The result is not validated (init_tracing does that).
to_seconds ¶
Normalise a duration given as float seconds or :class:datetime.timedelta.
trace_fields ¶
{"trace_id": …, "span_id": …} of the active span for log fields; {} without one.
traced ¶
traced(name_or_fn: str | Callable[P, R] | None = None, /, *, attributes: Attributes = None, kind: SpanKindLike = None, tracer: Tracer | None = None) -> Callable[P, R] | Callable[[Callable[P, R]], Callable[P, R]]
Decorator that runs a function (sync or async def) inside a span.
@traced, @traced() and @traced("name") all work; the default span name is the
function's qualified name. Async generators are not supported (the span would end before
iteration).
traceql_text ¶
Render a query argument (text or :class:TraceQL) to text.
truncate ¶
Truncate text to limit characters (adds an ellipsis when cut).
user_agent ¶
grow-codes-<key>/<version> (python) plus <suffix> when given.
validate_tenant_id ¶
Raise :class:ConfigError when tenant_id breaks Tempo's tenant id rules.
Each tenant is 1-150 characters of letters, digits and !-_.*'(); | joins several
tenants for a federated query ("tenant-b|tenant-a"). None means "no tenant".
The message never echoes the value.
wait_for_trace ¶
wait_for_trace(client: TraceQueryClient, trace_id: str, *, timeout_s: float | timedelta = 30.0, interval_s: float | timedelta = 1.0, min_spans: int = 1, normalize: Callable[[str], str] | None = normalize_trace_id) -> Trace
Poll get_trace until the trace has min_spans spans; TimeoutError otherwise.
NotFoundError and retryable errors count as "not there yet". normalize turns
trace_id into the id passed to get_trace (default: 32-char lower hex); None
passes native ids verbatim.
with_span ¶
with_span(name: str, fn: Callable[[Span], R] | None = None, *, attributes: Attributes = None, kind: SpanKindLike = None, links: Sequence[Link | SpanContext] | None = None, tracer: Tracer | None = None) -> R | AbstractContextManager[otel_trace.Span]
Run fn(span) inside a new current span, or return a context manager without fn.
The span always ends. An exception is recorded, sets ERROR status and is re-raised
(cancellation, asyncio.CancelledError or the SDK CancelledError, is not). The
context-manager form also works inside async code (context is kept in contextvars).