Caldron Accelerates TNetennba Engineering Validation with AWS, Making Full Attention Practical at Million-Token Scale

As LLM context windows continue to expand, the industry’s focus is shifting from how much information a model can accept to whether it can keep using that information accurately across longer inputs. For applications such as AI agents, enterprise knowledge search, research assistance, and code understanding, valuable long context means more than a larger input window. Models must also be able to access and use information distributed across long texts while keeping resource costs under control. Balancing context length, effective information use, and inference efficiency has therefore become a critical engineering challenge for bringing LLMs into production.

Caldron chose to address this challenge at the level of the LLM inference architecture. To make full attention practical at million-token scale, Caldron developed TNetennba as a full-attention replacement module for existing LLM backbones. It preserves the model’s ability to access and use information across the full context while using a more compact information representation to reduce KV cache requirements and long-context inference costs. On supported open-source LLM backbones, a standard self-attention module can be replaced with TNetennba using a training workload comparable to a typical fine-tuning run.

Figure: Standard decoder block and TNetennba-modified decoder block. TNetennba replaces the original attention component, reducing the BF16 KV cache in the Qwen3-8B example from 4,096 × 36 bytes per token to 104 × 36 bytes per token. Source: caldron.ai.

In a BF16 KV cache example, Qwen3-8B requires 4,096 × 36 bytes per token; with TNetennba, this is reduced to 104 × 36 bytes. During decoding, when performance is constrained primarily by memory access rather than computation, models equipped with TNetennba can achieve a speedup of more than 20×.

While converting a supported backbone can require compute comparable to a typical fine-tuning run, developing and validating a new architecture requires repeated training, evaluation, and iteration. Testing a single hypothesis may require the rapid allocation of large-scale resources, and the results directly shape the next architecture and training plan. Caldron CTO Hao Li describes the process as “an experimental environment with no predetermined answers.” This nonlinear, results-driven R&D cadence places greater demands on infrastructure elasticity and responsiveness.

For this reason, Caldron rejected the traditional capital-intensive approach of provisioning compute capacity in advance and chose to work with AWS. Caldron sought to advance the practical deployment of TNetennba by drawing on AWS elastic compute infrastructure and its comprehensive support for startups—from agile R&D to global expansion.

“AWS has built a strong industry reputation and a substantial body of proven experience. Its mature portfolio of AI models and services spans the full lifecycle from compute to model development and deployment. AWS also offers cost advantages and strong price performance for core compute resources, together with mature, stable, and scalable enterprise infrastructure that can meet Caldron’s current training needs and support our future growth.”

Hao Li, CTO, Caldron

To keep traditional infrastructure constraints from limiting frontier exploration, Caldron moved away from a capital-intensive model that locks in hardware resources in advance. Instead, it built an agile R&D environment on Amazon EC2 that could accommodate a high degree of uncertainty, using GPU compute resources for model training and inference validation.

With AWS elastic compute capabilities, Caldron can provision and quickly release high-performance computing resources on demand, rapidly scale up for validation when required, and deprovision resources once experiments are complete. This reduces idle capacity and overall cost. The on-demand approach avoids validation delays that could reach 3–5 days for an individual node under traditional scaling models. Caldron estimates that elastic compute support has accelerated its overall R&D progress by 30%–50%.

Alongside hardware resources, AWS provided ongoing technical support during deployment and operations. In one urgent troubleshooting case involving low-level hardware compatibility, the issue was resolved with support from AWS technical experts within 15 minutes of Caldron raising the request. AWS support substantially reduced operational complexity, improving Caldron’s operational efficiency by 80% and allowing the team to focus its time on optimizing the underlying attention architecture and advancing its algorithms.

“AWS provides professional, responsive service and has consistently met Caldron’s diverse, ad hoc requirements with elastic compute resources. That has made a lasting impression on us.”

Hao Li, CTO, Caldron

As a Singapore-based AI technology company with a global outlook, Caldron sees efficient access to customers worldwide as central to commercialization. Caldron is preparing to make its breakthrough long-context inference offerings available on AWS Marketplace. Its planned phased releases include a general-purpose long-context product with a 2M–4M-token context window and commercial offerings for AI agent developers and enterprise customers.

Through AWS Marketplace, Caldron can benefit from AWS’s global reach and brand recognition while gaining access to customers worldwide. Caldron plans to offer pre-integrated long-context inference solutions through AWS Marketplace, enabling customers to access these capabilities within their existing AWS environments. This will help shorten Caldron’s path from technology to revenue and support its vision for global delivery.

 

“We recognize the value of AWS industry resources and look forward to engaging with them. From content and customer-story sharing to executive forums and industry events, AWS has built a mature, open, and professional resource ecosystem that helps companies understand technology trends and access high-quality resources. As our business grows, Caldron hopes to participate more actively in AWS programs and exchange ideas with more customers and partners.”

Hao Li, CTO, Caldron

With elastic compute and stable infrastructure from AWS, Caldron completed engineering validation of its underlying technology efficiently and achieved a series of measurable technical outcomes, helping make costly LLM inference more affordable and efficient:

• Public tests show that TNetennba reduces KV cache requirements to approximately 1/40–1/100 of those of conventional approaches.
• Above 512K tokens, where attention becomes the dominant computational cost, the associated inference cost is reduced to approximately 1/20–1/60 of that of conventional approaches.
• Complete RULER evaluation has been conducted up to 2M tokens. Qwen3-8B + TNetennba achieved average RULER scores of 90.9 at 1M tokens and 88.4 at 2M tokens, with average RULER performance remaining relatively stable as context length scales into the millions of tokens.

Figure: Effective Context—Qwen3-8B + TNetennba maintains relatively stable average RULER performance as context length increases. Qwen3-8B and Qwen3-32B baselines are also shown. Source: caldron.ai.

Figure: Task Stability Across Context Length—RULER task scores for Qwen3-8B + TNetennba from 64K to 2M tokens. Darker cells indicate higher task scores, while comparisons across context lengths show how task performance changes as context grows. Source: caldron.ai.
 

Extending TNetennba Beyond 10M Tokens

Building on its complete public RULER evaluation through 2M tokens, Caldron has extended its evaluation of TNetennba far beyond the underlying LLM backbone’s native operating range. In evaluations using Qwen3-8B as the backbone, TNetennba achieved scores of 59.18 at 2M tokens, 44.92 at 10M tokens, 43.95 at 22M tokens, and 36.72 at 60M tokens on Caldron’s Long-Range Multi-Hop Benchmark, demonstrating measurable performance at tens-of-millions-of-tokens scale and pointing to its potential to unlock even stronger long-context capabilities in larger, more capable LLM backbones.

Figure: Long-Range Multi-Hop Benchmark. Qwen3-8B + TNetennba demonstrates measurable performance from 2M to 60M tokens. Source: caldron.ai.

 

Together, these results—from lower KV cache requirements and inference costs to public RULER performance at million-token scale and internal performance on the Long-Range Multi-Hop Benchmark at up to 60M tokens—define TNetennba’s role. It does not replace the foundation model’s knowledge or reasoning capabilities. Instead, by restructuring the attention module, it gives existing LLMs a longer and more efficient working context that they can continue to use effectively, creating an engineering foundation for productization and deployment at scale.

 

Making Long Context a Persistent Working State

When long context can be retained continuously at lower cost, models can do more than process a single long document. TNetennba’s value therefore extends beyond benchmark performance to use cases including enterprise knowledge, professional research, code understanding, and complex AI agents:

• Preserve relationships that can be lost through chunking: TNetennba allows legal texts, research materials, and large codebases to be retained more completely in the active context, enabling models to retrieve, compare, and synthesize information across a broader scope.
• Turn domain knowledge into dynamically loadable context: With highly compressed and concatenable KV representations, relevant domain materials can be loaded dynamically as tasks require. Models can answer and reason directly from the supplied sources, improving traceability and reducing some hallucinations caused by insufficient grounding.
• Lower the barriers to iterating on vertical applications: Enterprises can load knowledge materials by topic, department, or workflow and build topic-specific domain applications on smaller models. This reduces the need to train and maintain a dedicated model for every scenario and enables faster iteration of specialized knowledge services.

Giving Complex AI Agents More Complete Working Memory

By compressing the KV cache and enabling direct concatenation of context segments, TNetennba allows more working state to remain available throughout inference, reducing the need to repeatedly compress, discard, or reconstruct context. This provides a new foundational capability for AI agents running long-duration, multistep workflows that require reasoning across documents.

 

“With AWS elastic compute resources and support from its expert team, the ability of models equipped with Caldron’s technology to retrieve and use information consistently across ultra-long contexts has improved significantly. This makes many commercial applications possible. The industry no longer needs to rely as heavily on aggressive RAG chunking or task-specific fine-tuning simply to work around context limitations. More complete and traceable domain systems can be built directly from full-context source materials.”

Hao Li, CTO, Caldron

 

As the race to scale model size begins to slow, inference efficiency and engineering feasibility are emerging as key differentiators. With ultra-long context as a critical variable, Caldron is turning underlying inference advances into repeatable, deliverable system capabilities. Through deeper collaboration with AWS, the company is accelerating the transition from technical innovation to business value and exploring new possibilities for AI across more use cases worldwide.