Workspace Observability

Workspace Observability is designed for developers, AI engineers, and workspace managers. Scoped strictly to a single workspace, it provides the granular diagnostic details needed to inspect request payloads, verify pre- and post-processing runtime modules, and optimize model performance and costs.

---

Access & Permissions

Observability details inside a workspace are restricted to users with appropriate workspace-level permissions:

• View Logs: Allows members to search logs and inspect request payloads and trace spans.
• View Metrics: Allows members to access the workspace-level metrics dashboards.

---

Request Logs & Drawer Details

The workspace logs console (/workspaces/[wsId]/logs) is the developer's primary workspace console. Click any request row to open the Log Detail Drawer:

• Metadata Header: Displays the unique Request ID, Trace ID, workspace API key hint used to authenticate the client, and custom end-user identifiers.
• Payloads Tab: Offers full JSON code blocks showing the exact prompt/messages input sent to the gateway and the final text/JSON response returned from the LLM.
• Token Usage: Displays the counts for input tokens, output tokens, and tokens saved by context caching.
• Cost Metrics: Evaluates uncached input cost, cached input cost, and output cost in micro-dollars, calculated automatically using your collection pricing.

---

Distributed Tracing & Spans

The Workspace Traces console (/workspaces/[wsId]/traces) provides end-to-end execution timeline visibility into all multi-step AI operations, agent workflows, and deployment failover retries within the workspace.

When a request traverses multiple Runtime Modules, executes tools, or triggers fallback retries across multiple LLM providers, Infralo tracks the transaction lifecycle using Trace Spans.

Trace Span Types

• Gateway / Request Spans: Entry point capturing initial request interception, model routing, and endpoint classification (chat, embeddings, responses).
• LLM Spans: Records calls to model providers. Multi-model failover retries (e.g., Attempt #1 503 Service Unavailable -> Attempt #2 success via Fallback routing) are logged as separate child spans under the same trace.
• Tool Spans: Logs discrete tool executions, database operations, or external API calls (instrumented via @infralo.tool() or trace.tool()).
• Module Spans: Logs Pre-processing (PRE) and Post-processing (POST) runtime modules (e.g., measuring PII Tokenization or TOON normalization latency).
• Approval Spans: Logs Human-in-the-Loop Approval requests (pending, approved, rejected, expired), actor identity, comments, and event timestamps.

Trace Header Metrics

When opening the Full Trace Timeline modal, summary KPI widgets display:

• Total Duration: Wall-clock latency from initial invocation to final response.
• Spans Executed: Count of actual execution steps (LLMs, tools, modules, approvals).
• Errors Encountered: Number of errored sub-spans (e.g. 1 error attempt recorded during a failover retry sequence).
• Tokens Utilized: Total prompt/input and completion/output tokens consumed across all LLM spans.
• Estimated Cost: Cumulative USD cost evaluated across all model calls in the trace.

Trace Conclusion Status

A trace's overall conclusion status (Success, Error, Partial) reflects the final request outcome:

• If Attempt 1 encounters a temporary provider error (503) but Attempt 2 succeeds via Fallback routing, the overall trace status displays as Success, while the timeline detail retains the 1 Error marker for full diagnostic transparency.

---

Workspace Analytics Dashboards

The workspace metrics dashboard (/workspaces/[wsId]/metrics) compiles telemetry into several primary analytics tabs, preceded by a high-level KPI Card Grid showing total requests, costs, tokens, average latency, and cache hit rates.

1. Usage & Cost

Tracks consumption and throughput over time:

• Trends: Monitors request volume, token counts, and cost charts over time to track integration activity.

2. Performance

Analyzes latency and response distributions:

• Latency Percentiles: Displays P50 (median), P95, and P99 latency markers to help developers track tail-latency degradation.
• Model Latencies: Compares the average response time of different whitelisted models within the workspace.

3. Models & Deployments

Provides granular breakdown per execution target:

• Usage & Cost Share: Displays which models and deployments are consuming the majority of your budget or processing the most requests.
• Detailed Table: Lists request count, token volume, average latency, and costs grouped by model alias.

4. Cache

Tracks the optimization metrics generated by Infralo's Response Cache (configured in Deployments):

• Hit Rate: The percentage of requests served directly from the cache.
• Savings Analysis: Calculates the count of input/output tokens saved and cumulative financial cost savings.
• Latency Gain: Compares cache hit times (sub-millisecond) against cache miss times (provider network latency).

5. Insights

Provides advanced patterns and optimization audits:

• Distributions: Visualizes request finish reasons (e.g., stop, length, tool_calls, content_filter).
• Optimization: Flags anomalies and highlights opportunities for token/cost optimizations.