LLM Observability Managed Service (Arize Phoenix) by Horizon Digital

Horizon Digital runs Arize Phoenix observability for your production LLM workloads on Amazon Bedrock or Amazon SageMaker. We instrument every prompt and tool call, define and run evaluation rubrics on a schedule, detect drift before your users do, and deliver monthly model and cost telemetry reports to engineering and finance leadership.

Each month we deliver:

• Instrumentation: Phoenix tracing wired to every LLM invocation in scope - prompt, response, latency, token usage, tool calls. Continuous improvement as your workloads evolve.
• Evaluation: Scheduled evaluation runs against a rubric we define with you (correctness, faithfulness, safety, tone). New failures surfaced with examples and prompt-level diff.
• Drift detection: Embedding and outcome drift monitored across the rolling 30-day window. Alerts to Slack or Teams when drift crosses agreed thresholds.
• Cost telemetry: Token consumption and cost per model per use case, reconciled to AWS Cost Explorer via the aws-apn-id tag. Identifies the most expensive prompts and the most cost-effective model substitutions.
• Monthly review session: A 60-minute review with engineering and product, with a one-page summary suitable for the CFO and the customer.

Who this is for:

• SMB tier - Bronze retainer: One LLM model in production, weekly Phoenix review, monthly summary report, drift alerts to a single channel.
• Enterprise tier - Gold retainer: Up to 10 production models, 8x5 AEST with 1-hour response SLA, daily evaluation runs, drift and cost alerts wired to PagerDuty or Opsgenie, cost telemetry reconciled to PRM tags for partner revenue attribution.

Why Horizon Digital:

• AWS AI Services Competency partner. Arize Phoenix is HD default observability stack on every Bedrock engagement we deliver.
• Australian-based delivery team. Privacy-policy-aligned to the Australian Privacy Principles (APP).
• Recurring monthly billing - the retainer is PRM-attributable to your AWS spend every month it remains active.

Related AWS services: Amazon Bedrock, Anthropic Claude on Bedrock, Amazon SageMaker, AWS Lambda, Amazon S3, Amazon OpenSearch, AWS Cost Explorer, AWS CloudWatch.