Logging¶
The logging interface is the same for every logging service of Grow Codes. The Datadog Log Management client
gives you a LogClient for sending (a batching client on top of the Datadog Log Management exporter) and, when
the service has a query API, a LogQueryClient for searching (see Functions for
the accessors and the service options such as streams, indexes or labels). The examples call
them logs and query.
Records¶
| Field | Type | Notes |
|---|---|---|
timestamp |
time | defaults to now |
severity |
Severity |
OpenTelemetry numbers: Trace=1, Debug=5, Info=9, Warn=13, Error=17, Fatal=21 |
severityText |
string | TRACE … FATAL or the vendor's text |
body |
string | the message; structured bodies are JSON-encoded |
attributes |
map | string, number, boolean, lists and nested maps |
resource |
map | service.name, service.version, deployment.environment, host.name |
traceId / spanId / traceFlags |
hex / int | filled from the active OpenTelemetry span when missing |
Send logs¶
send and sendBatch only enqueue: a background worker batches records and exports them.
Each record passes the RedactionProcessor first. Call flush() when you need the records out
now and close() on shutdown (flush + stop; idempotent).
import { Severity } from "@grow-codes/datadog-logs";
logs.send({ message: "payment accepted", severity: Severity.Info, attributes: { orderId: "o-42" } });
logs.sendBatch([{ message: "a" }, { message: "b", severity: Severity.Warn }]);
const log = logs.logger("checkout", { region: "eu" }); // facade, minSeverity Info
log.info("cart created", { items: 3 });
log.child({ userId: "u-1" }).error("card declined", { code: "05" });
await logs.flush();
console.log(logs.stats()); // { queued, sent, dropped, failed, retries }
await logs.close();
use grow_datadog_logs::logging::{LogClient, LogInput, Severity};
use grow_datadog_logs::attrs;
logs.send(LogInput::new("payment accepted").attribute("orderId", "o-42").into());
logs.send_batch(vec!["a".into(), LogInput::new("b").severity(Severity::Warn).into()]);
let log = logs.logger(Some("checkout"), attrs! { "region" => "eu" });
log.info("cart created", attrs! { "items" => 3 });
logs.flush(None).await?;
println!("{:?}", logs.stats());
logs.close().await?;
logs.Send(ctx, datadoglogs.LogInput{
Message: "payment accepted",
Severity: datadoglogs.SeverityInfo,
Attributes: map[string]datadoglogs.AttrValue{"orderId": "o-42"},
})
logs.SendBatch(ctx, datadoglogs.Entries(
datadoglogs.LogInput{Message: "a"},
datadoglogs.LogInput{Message: "b", Severity: datadoglogs.SeverityWarn},
))
log := logs.Logger("checkout", map[string]datadoglogs.AttrValue{"region": "eu"})
log.Info(ctx, "cart created", "items", 3) // slog-style key/value pairs
if err := logs.Flush(ctx); err != nil {
return err
}
fmt.Printf("%+v\n", logs.Stats())
_ = logs.Close(ctx)
from grow_datadog_logs import LogInput, Severity
logs.send(LogInput("payment accepted", attributes={"orderId": "o-42"}))
logs.send_batch([LogInput("a"), LogInput("b", severity=Severity.WARN)])
log = logs.logger("checkout", {"region": "eu"})
log.info("cart created", {"items": 3})
log.child({"userId": "u-1"}).error("card declined", {"code": "05"})
logs.flush()
print(logs.stats())
logs.close()
AsyncBatchingLogClient offers the same API with await.
logs.send(LogInput.builder("payment accepted").attribute("orderId", "o-42").build());
logs.sendBatch(List.of(
LogInput.builder("a").build(),
LogInput.builder("b").severity(Severity.WARN).build()));
Logger log = logs.logger("checkout", Map.of("region", "eu"));
log.info("cart created", Map.of("items", 3));
logs.flush();
System.out.println(logs.stats());
logs.close();
Batching¶
| Option | Default | Meaning |
|---|---|---|
maxQueueSize |
2048 | records waiting in memory |
maxBatchSize |
512 | records per export |
maxBatchBytes |
1 MiB | UTF-8 JSON size per export |
flushInterval |
1 s | export at least this often |
exportTimeout |
30 s | deadline of one export |
dropPolicy |
drop_newest |
when the queue is full: drop_newest, drop_oldest or block |
gzip |
true |
compress exports |
onDrop(records, reason) |
— | queue_full, shutdown or export_failed |
onError(error, records) |
— | export errors (after retries) |
add/send never block unless dropPolicy is block. After close(), new records are dropped
with reason shutdown. stats() reports queued, sent, dropped, failed, retries.
Redaction¶
Every record passes a RedactionProcessor before it is queued. Attribute keys that contain one
of the default keys (case-insensitive, recursive) get the value [REDACTED]:
password, passwd, token, authorization, secret, cookie, api_key, apikey,
access_key, private_key. Add your own keys and regular expressions (applied to the body and
string attribute values):
Framework adapters¶
Send the logs of your existing logging framework through the log client:
| Language | Adapter |
|---|---|
| TypeScript | pino: pino(createPinoTransport(logs)); winston: new WinstonTransport(logs) |
| Rust | tracing_subscriber::registry().with(LogLayer::new(logs)) (feature tracing-layer); LogBridge::new(logs).init() for the log crate (feature log-bridge) |
| Go | slog.New(datadoglogs.NewSlogHandler(logs, nil)); zap: zapadapter.NewLogger(logs, zapadapter.Options{}) |
| Python | logging.getLogger().addHandler(LogHandler(logs)); structlog: structlog_processor(logs) |
| Java | Logback GrowLogbackAppender.install(context, logs), Log4j2 GrowLog4j2Appender, java.util.logging GrowJulHandler |
When an OpenTelemetry span is active and the record has no trace id, traceId, spanId and
traceFlags come from the active span, so logs and traces correlate.
Search logs¶
LogQueryClient is available when Datadog Log Management has a query API (otherwise its methods raise
UNSUPPORTED; see Compatibility).
| Function | Returns | Notes |
|---|---|---|
search(query, opts?) |
LogSearchResult { records, nextCursor?, total? } |
opts: start, end, limit (100), order (desc/asc), cursor |
tail(query, onRecord, opts?) |
TailHandle (stop()) |
streams or polls; UNSUPPORTED when the service can do neither |
listStreams() / listLabels() |
names | |
waitForLog(query, opts?) |
LogRecord |
polls search until a record matches; TIMEOUT after timeout (30 s) |
LogSearchResult result = query.search("orderId:o-42", SearchOptions.builder().limit(20).build());
result.records().forEach(r -> System.out.println(r.timestamp() + " " + r.body()));
LogRecord record = query.waitForLog("payment accepted",
WaitForLogOptions.builder().timeout(Duration.ofSeconds(10)).build());
Testing¶
The testing module has an in-memory log client and query client (see Docker testing).