Skip to content

Logging

The logging interface is the same for every logging service of Grow Codes. The Amazon CloudWatch Logs client gives you a LogClient for sending (a batching client on top of the Amazon CloudWatch Logs exporter) and, when the service has a query API, a LogQueryClient for searching (see Functions for the accessors and the service options such as streams, indexes or labels). The examples call them logs and query.

Records

Field Type Notes
timestamp time defaults to now
severity Severity OpenTelemetry numbers: Trace=1, Debug=5, Info=9, Warn=13, Error=17, Fatal=21
severityText string TRACE … FATAL or the vendor's text
body string the message; structured bodies are JSON-encoded
attributes map string, number, boolean, lists and nested maps
resource map service.name, service.version, deployment.environment, host.name
traceId / spanId / traceFlags hex / int filled from the active OpenTelemetry span when missing

Send logs

send and sendBatch only enqueue: a background worker batches records and exports them. Each record passes the RedactionProcessor first. Call flush() when you need the records out now and close() on shutdown (flush + stop; idempotent).

import { Severity } from "@grow-codes/aws-cloudwatch-logs";

logs.send({ message: "payment accepted", severity: Severity.Info, attributes: { orderId: "o-42" } });
logs.sendBatch([{ message: "a" }, { message: "b", severity: Severity.Warn }]);

const log = logs.logger("checkout", { region: "eu" });   // facade, minSeverity Info
log.info("cart created", { items: 3 });
log.child({ userId: "u-1" }).error("card declined", { code: "05" });

await logs.flush();
console.log(logs.stats()); // { queued, sent, dropped, failed, retries }
await logs.close();
use grow_aws_cloudwatch_logs::logging::{LogClient, LogInput, Severity};
use grow_aws_cloudwatch_logs::attrs;

logs.send(LogInput::new("payment accepted").attribute("orderId", "o-42").into());
logs.send_batch(vec!["a".into(), LogInput::new("b").severity(Severity::Warn).into()]);

let log = logs.logger(Some("checkout"), attrs! { "region" => "eu" });
log.info("cart created", attrs! { "items" => 3 });

logs.flush(None).await?;
println!("{:?}", logs.stats());
logs.close().await?;
logs.Send(ctx, cloudwatchlogs.LogInput{
    Message:    "payment accepted",
    Severity:   cloudwatchlogs.SeverityInfo,
    Attributes: map[string]cloudwatchlogs.AttrValue{"orderId": "o-42"},
})
logs.SendBatch(ctx, cloudwatchlogs.Entries(
    cloudwatchlogs.LogInput{Message: "a"},
    cloudwatchlogs.LogInput{Message: "b", Severity: cloudwatchlogs.SeverityWarn},
))

log := logs.Logger("checkout", map[string]cloudwatchlogs.AttrValue{"region": "eu"})
log.Info(ctx, "cart created", "items", 3) // slog-style key/value pairs

if err := logs.Flush(ctx); err != nil {
    return err
}
fmt.Printf("%+v\n", logs.Stats())
_ = logs.Close(ctx)
from grow_aws_cloudwatch_logs import LogInput, Severity

logs.send(LogInput("payment accepted", attributes={"orderId": "o-42"}))
logs.send_batch([LogInput("a"), LogInput("b", severity=Severity.WARN)])

log = logs.logger("checkout", {"region": "eu"})
log.info("cart created", {"items": 3})
log.child({"userId": "u-1"}).error("card declined", {"code": "05"})

logs.flush()
print(logs.stats())
logs.close()

AsyncBatchingLogClient offers the same API with await.

logs.send(LogInput.builder("payment accepted").attribute("orderId", "o-42").build());
logs.sendBatch(List.of(
    LogInput.builder("a").build(),
    LogInput.builder("b").severity(Severity.WARN).build()));

Logger log = logs.logger("checkout", Map.of("region", "eu"));
log.info("cart created", Map.of("items", 3));

logs.flush();
System.out.println(logs.stats());
logs.close();

Batching

Option Default Meaning
maxQueueSize 2048 records waiting in memory
maxBatchSize 512 records per export
maxBatchBytes 1 MiB UTF-8 JSON size per export
flushInterval 1 s export at least this often
exportTimeout 30 s deadline of one export
dropPolicy drop_newest when the queue is full: drop_newest, drop_oldest or block
gzip true compress exports
onDrop(records, reason) — queue_full, shutdown or export_failed
onError(error, records) — export errors (after retries)

add/send never block unless dropPolicy is block. After close(), new records are dropped with reason shutdown. stats() reports queued, sent, dropped, failed, retries.

Redaction

Every record passes a RedactionProcessor before it is queued. Attribute keys that contain one of the default keys (case-insensitive, recursive) get the value [REDACTED]: password, passwd, token, authorization, secret, cookie, api_key, apikey, access_key, private_key. Add your own keys and regular expressions (applied to the body and string attribute values):

import { DEFAULT_REDACT_KEYS, RedactionProcessor } from "@grow-codes/aws-cloudwatch-logs";

const redaction = new RedactionProcessor({
  keys: [...DEFAULT_REDACT_KEYS, "iban"],
  patterns: [/\b\d{16}\b/g],
});
use grow_aws_cloudwatch_logs::logging::{RedactionProcessor, DEFAULT_REDACT_KEYS};

let keys = DEFAULT_REDACT_KEYS.iter().copied().chain(["iban"]);
let redaction = RedactionProcessor::new(keys, vec![regex::Regex::new(r"\b\d{16}\b")?]);
redaction := cloudwatchlogs.NewRedactionProcessor(cloudwatchlogs.RedactionOptions{
    Keys:     append(slices.Clone(cloudwatchlogs.DefaultRedactKeys), "iban"),
    Patterns: []*regexp.Regexp{regexp.MustCompile(`\b\d{16}\b`)},
})
import re
from grow_aws_cloudwatch_logs import DEFAULT_REDACT_KEYS, RedactionProcessor

redaction = RedactionProcessor(keys=[*DEFAULT_REDACT_KEYS, "iban"], patterns=[re.compile(r"\b\d{16}\b")])
RedactionProcessor redaction = RedactionProcessor.builder()
    .addKeys("iban")
    .pattern(Pattern.compile("\\b\\d{16}\\b"))
    .build();

Framework adapters

Send the logs of your existing logging framework through the log client:

Language Adapter
TypeScript pino: pino(createPinoTransport(logs)); winston: new WinstonTransport(logs)
Rust tracing_subscriber::registry().with(LogLayer::new(logs)) (feature tracing-layer); LogBridge::new(logs).init() for the log crate (feature log-bridge)
Go slog.New(cloudwatchlogs.NewSlogHandler(logs, nil)); zap: zapadapter.NewLogger(logs, zapadapter.Options{})
Python logging.getLogger().addHandler(LogHandler(logs)); structlog: structlog_processor(logs)
Java Logback GrowLogbackAppender.install(context, logs), Log4j2 GrowLog4j2Appender, java.util.logging GrowJulHandler

When an OpenTelemetry span is active and the record has no trace id, traceId, spanId and traceFlags come from the active span, so logs and traces correlate.

Search logs

LogQueryClient is available when Amazon CloudWatch Logs has a query API (otherwise its methods raise UNSUPPORTED; see Compatibility).

Function Returns Notes
search(query, opts?) LogSearchResult { records, nextCursor?, total? } opts: start, end, limit (100), order (desc/asc), cursor
tail(query, onRecord, opts?) TailHandle (stop()) streams or polls; UNSUPPORTED when the service can do neither
listStreams() / listLabels() names
waitForLog(query, opts?) LogRecord polls search until a record matches; TIMEOUT after timeout (30 s)
const result = await query.search("orderId:o-42", { limit: 20, start: new Date(Date.now() - 3600_000) });
for (const r of result.records) console.log(r.timestamp, r.severityText, r.body);
const record = await query.waitForLog("payment accepted", { timeoutMs: 10_000 });
let result = query.search("orderId:o-42", SearchOptions { limit: Some(20), ..Default::default() }).await?;
for r in &result.records {
    println!("{:?} {} {}", r.timestamp, r.severity_text, r.body);
}
result, err := query.Search(ctx, "orderId:o-42", cloudwatchlogs.SearchOptions{Limit: 20})
for _, r := range result.Records {
    fmt.Println(r.Timestamp, r.SeverityText, r.Body)
}
record, err := query.WaitForLog(ctx, "payment accepted", cloudwatchlogs.WaitForLogOptions{Timeout: 10 * time.Second})
result = query.search("orderId:o-42", limit=20)
for r in result.records:
    print(r.timestamp, r.severity_text, r.body)
record = query.wait_for_log("payment accepted", timeout_s=10.0)
LogSearchResult result = query.search("orderId:o-42", SearchOptions.builder().limit(20).build());
result.records().forEach(r -> System.out.println(r.timestamp() + " " + r.body()));
LogRecord record = query.waitForLog("payment accepted",
    WaitForLogOptions.builder().timeout(Duration.ofSeconds(10)).build());

Testing

The testing module has an in-memory log client and query client (see Docker testing).