Tramai Logo
Tramai

Custom Observers

TramAI exposes engine lifecycle events through EngineEventObserver, letting you build custom monitoring, alerting, and metrics without depending on OpenTelemetry.

What This Covers

  • EngineEventObserver SPI
  • Observable engine events and their attributes
  • Custom metrics recording
  • Event-driven behavior patterns
  • Standalone and Spring Boot registration

When to Use Custom Observers

Use EngineEventObserver when you need:

  • custom alerting for DLP rejections or circuit breaker events
  • cost tracking and token budget monitoring
  • custom metrics in a non-OTel format (Prometheus, Micrometer, custom dashboard)
  • operational logging for security-relevant events
  • event-driven behavior (pausing workflows, triggering remediation)

For full OpenTelemetry tracing, use OpenTelemetryOperationObserver from tramai-observability instead.

Minimum Setup

SPI Interface

fun interface EngineEventObserver {
    fun onEngineEvent(name: String, attributes: Map<String, Any?>)
}

A fun interface with a single method — implement it inline or as a class.

Standalone Registration

val tramai = Tramai {
    provider(OpenAiProvider(System.getenv("OPENAI_API_KEY")), name = "openai", default = true)
    model("gpt-4o", "openai")

    engineEventObserver(EngineEventObserver { name, attributes ->
        println("Engine event: $name $attributes")
    })
}

Spring Boot Registration

@Bean
fun loggingEngineEventObserver(): EngineEventObserver = EngineEventObserver { name, attributes ->
    log.info("Engine event: {} {}", name, attributes)
}

At most one EngineEventObserver bean is supported. If more than one is found, auto-configuration throws at startup.

Observable Events

DLP Events

Event nameWhen it firesAttributes
dlp.rejectionDLP interceptor rejected outputoperationInterface, operationMethod, providerId, modelName, reason
dlp.redactionDLP interceptor redacted contentredactionCount, ruleIds, contentType

Circuit Breaker Events

Event nameWhen it firesAttributes
circuit_breaker.openedCircuit breaker trippedproviderId, failureCount, durationMs
circuit_breaker.closedCircuit breaker reset after cooldownproviderId, openDurationMs
circuit_breaker.half_openCircuit breaker in test stateproviderId

Token Budget Events

Event nameWhen it firesAttributes
token_budget.warningSoft token limit crossedcurrent, limit, operationInterface, operationMethod
token_budget.exceededHard token limit crossed (operation failed)current, limit, operationInterface, operationMethod

Retry and Fallback Events

Event nameWhen it firesAttributes
retry.scheduledRetry attempt scheduledattempt, maxRetries, delayMs, reason
retry.exhaustedAll retry attempts exhaustedattempts, reason
fallback.activatedFallback provider selectedoriginalProvider, originalModel, fallbackProvider, fallbackModel
fallback.failedFallback also failedprovider, model, reason

Example: Security Alerting

val securityAlertObserver = EngineEventObserver { name, attributes ->
    when (name) {
        "dlp.rejection" -> {
            val source = "${attributes["operationInterface"]}.${attributes["operationMethod"]}"
            val reason = attributes["reason"] ?: "unknown"
            securityAlert("DLP block in $source: $reason")
        }
        "circuit_breaker.opened" -> {
            val provider = attributes["providerId"] ?: "unknown"
            securityAlert("Provider $provider circuit breaker opened")
        }
    }
}

Example: Custom Metrics

Record engine events as Prometheus or Micrometer metrics:

class MetricsEngineEventObserver(
    private val meterRegistry: MeterRegistry,
) : EngineEventObserver {

    private val dlpRejections = meterRegistry.counter("tramai.dlp.rejections")
    private val tokenWarnings = meterRegistry.counter("tramai.token_budget.warnings")
    private val circuitBreakers = meterRegistry.counter("tramai.circuit_breaker.opens")

    override fun onEngineEvent(name: String, attributes: Map<String, Any?>) {
        when (name) {
            "dlp.rejection" -> dlpRejections.increment()
            "token_budget.warning" -> tokenWarnings.increment()
            "circuit_breaker.opened" -> circuitBreakers.increment()
        }
    }
}

Registration:

@Bean
fun metricsEngineEventObserver(registry: MeterRegistry): EngineEventObserver =
    MetricsEngineEventObserver(registry)

Example: Event-Driven Behavior

Trigger external actions when events occur:

class RemediationEngineEventObserver(
    private val pager: PagerDutyClient,
    private val workflowPauser: WorkflowPauser,
) : EngineEventObserver {

    override fun onEngineEvent(name: String, attributes: Map<String, Any?>) {
        when (name) {
            "circuit_breaker.opened" -> {
                val provider = attributes["providerId"] as? String ?: return
                pager.trigger("Provider $provider circuit breaker opened")
            }
            "token_budget.exceeded" -> {
                val method = "${attributes["operationInterface"]}.${attributes["operationMethod"]}"
                workflowPauser.pauseOperations(method)
            }
        }
    }
}

Relationship to OpenTelemetry

FeatureEngineEventObserverOpenTelemetryOperationObserver
Engine eventsDirect callbackMapped to OTel events
Latency spansNoYes
Token usage metricsVia custom codeBuilt-in
Exporter dependencyNoneRequires OTel SDK + exporter
Best forCustom alerting, small deployments, non-OTel stacksFull observability pipeline

Both can coexist — the engine dispatches events to all registered observers.

Limitations

  • At most one EngineEventObserver bean in Spring Boot auto-configuration
  • The observer receives events synchronously — heavy processing should be dispatched to a separate thread
  • Event names and attribute keys are not yet documented as API-stable (use them, but expect possible renaming)
  • No built-in event buffering or batching (implement in your observer if needed)

Next Steps

Previous
Custom Interceptors
Next
Custom Providers