Design Deep Dive

Version: 0.4.0 Implementation language: Kotlin Framework policy: Framework-agnostic core, optional Spring adapter Serialization: Jackson (with kotlin-module)

Backend engineers on the JVM are increasingly asked to integrate AI capabilities into existing applications. The tooling available is either Python-native (LangChain, LlamaIndex) or a direct port of Python idioms into Java (LangChain4j) — neither of which feels idiomatic to a Kotlin/Java engineer who thinks in types, interfaces, and coroutines.

Complication

LangChain4j exists but inherits LangChain's mental model: chains, agents, and prompt templates as first-class abstractions. This model has known production problems — hard to test, hard to observe, leaky abstractions that expose model internals to application code. Structured outputs are an afterthought. Observability requires manual instrumentation.

Question

Can we build a JVM-native AI library where typed interfaces are the only abstraction, structured outputs are the default contract, and every AI interaction is automatically observable — with zero boilerplate?

Answer

Yes. TramAI is that library.

Core Abstraction

The single unit of TramAI is the AI Operation — an annotated Kotlin interface method.

@AiService
interface InvoiceAnalyzer {

    @Operation(
        prompt = "Analyze this invoice and extract line items with cost categories",
        model = "claude-sonnet-4-20250514"
    )
    suspend fun analyze(invoice: Invoice): InvoiceAnalysis
}

Rules

Interface must be annotated with @AiService
Each method must be annotated with @Operation
Method parameters are serialized as structured context
Return type determines behavior:
- String -> raw model output, no parsing
- Unit -> fire-and-forget
- Any data class or POJO -> structured output with schema validation and retry
- suspend fun -> coroutine-native, non-blocking
- Blocking methods -> supported for Java consumers

Structured Output Pipeline

This is the core intellectual work of the library.

1. SCHEMA GENERATION
   Java return type -> JSON Schema
   Uses Jackson SchemaGenerator + custom annotations
   Schema is cached per method at startup

2. PROMPT CONSTRUCTION
   User prompt + input serialization + schema injection
   Schema injected as system instruction: "Respond ONLY with valid JSON matching this schema"

3. MODEL CALL
   Request dispatched to ModelProvider
   Raw text response captured

4. RESPONSE PARSING
   Raw text -> JSON extraction (handles markdown fences, preamble)
   JSON -> Jackson deserialization into return type
   Bean Validation (JSR-380) applied if validator on classpath

5. VALIDATION FEEDBACK LOOP (if parsing fails)
   Validation error appended to conversation
   Re-sent to model, up to @Operation(maxRetries = N)
   Default: 2 retries

6. FAILURE
   After maxRetries exhausted: throw TramaiStructuredOutputException
   Exception contains: original prompt, last raw response, validation error, attempt count

Schema Annotations

data class SpendAnalysis(
    @AiDescription("Total spend in USD, always positive")
    val totalSpend: Double,

    @AiDescription("List of cost reduction recommendations, ordered by impact")
    @AiMinItems(1)
    val recommendations: List<Recommendation>,

    @AiDescription("Confidence score between 0.0 and 1.0")
    @AiRange(min = 0.0, max = 1.0)
    val confidence: Double
)

Provider Model

Provider Interface

interface ModelProvider {
    suspend fun complete(request: ModelRequest): ModelResponse
    fun supportsStructuredOutput(): Boolean = false
    fun providerId(): String = this::class.simpleName ?: "unknown"
}

Provider Resolution

providers are registered explicitly
models are registered explicitly against providers
explicit override via @Operation(provider = "ollama") remains available
unknown or unregistered models produce a deterministic resolution error

Scenario	Default	Behavior
Structured output parse failure	2 retries	Feeds validation error back to model
Provider transient error (429, 503)	3 retries	Exponential backoff
Non-retryable (auth, invalid model)	0 retries	Throws immediately

Exception Hierarchy

TramaiException (base, unchecked)
-> TramaiStructuredOutputException   — exhausted retries on parsing
-> TramaiProviderException           — provider returned unrecoverable error
-> TramaiConfigurationException      — misconfiguration detected at startup
-> TramaiTimeoutException            — call exceeded configured timeout

Prompt Rendering

Template syntax maps to parameter names:

@Operation(prompt = "Analyze spend data for tenant {{tenantId}}")
suspend fun analyze(tenantId: String, data: SpendRecord): SpendAnalysis

For multi-turn prompts, @SystemPrompt at the interface level:

@AiService
@SystemPrompt("You are a cost optimization expert. Be concise and actionable.")
interface SpendAnalyzer

Framework Integration

TramAI has no framework dependency in its core. Framework adapters are thin modules that wire TramAI's standalone API into the framework's DI and configuration systems.

Spring Boot: TramaiAutoConfiguration scans for @AiService interfaces, registers proxies as beans, reads application.yml config
Quarkus/Micronaut: Not shipped in v1. Standalone API works in both — instantiate TramAI manually and register as a CDI bean or singleton
Plain JVM/CLI: No framework, no classpath scanning, no magic. TramAI is a library, not a container.

Testing Support

The tramai-testing module provides:

MockAiProvider — canned responses, no network calls
SimulatedFailureProvider — retryable and non-retryable failures
RecordingOperationObserver — verify call counts and behavior
TramaiAssertions — fluent assertion API

Architecture Overview — high-level architecture
Module Design — module boundaries
Why TramAI — comparison to alternatives