K2-Horizon-7B

Overview

K2 Horizon 7B is an open-weight language model from the Institute of Foundation Models (IFM) at MBZUAI. Build coding assistants, document-analysis tools and conversational applications with text generation, reasoning, streaming responses and structured tool calls.

Deploy through Amazon SageMaker AI in your own AWS account. You control the endpoint, access permissions and resource lifecycle. Your application supplies conversation history and relevant documents, executes authorized tools and verifies results.

Key capabilities

• Text generation and reasoning: Generate answers, explanations and multi-step reasoning chains for coding, math, science and general knowledge tasks.

• Tool calling: Define function schemas in your request and receive structured tool-call JSON that your application validates and executes.

• Streaming responses: Choose InvokeEndpoint for complete responses or InvokeEndpointWithResponseStream for token-by-token streamed output.

• Flexible integration: Supply JSON conversation messages with role and content fields. Embed system instructions, user queries and assistant history in a single request.

Deployment configuration

• 32K context: 32,768 tokens on ml.g6.2xlarge with one NVIDIA L4 GPU.

This configuration uses FP8 model weights, BF16 activations and KV cache, and one active sequence. Availability depends on the AWS Region, account quota and instance capacity.

Deployment and security model

The model runs as a SageMaker endpoint inside your AWS account. You provision the endpoint, assign IAM roles and manage the instance lifecycle. Because the endpoint operates within your account, you can apply your organization's existing network controls, monitoring and access policies.

SageMaker endpoints support VPC deployment, allowing you to run inference traffic within your private network. AWS encrypts SageMaker storage volumes and supports encryption of data in transit between your client and the endpoint using TLS. For details on SageMaker security controls, refer to the Amazon SageMaker AI documentation (https://docs.aws.amazon.com/sagemaker/ ).

Getting started

After subscribing, create a SageMaker model package and deploy it to a real-time endpoint on the ml.g6.2xlarge instance type. Send a test request using the SageMaker Python SDK or the AWS CLI to confirm the endpoint is operational. For model details, configuration guidance and measurement notes, see the K2 Horizon 7B FP8 documentation on GitHub (https://github.com/MBZUAI-IFM/k2-horizon-aws-marketplace/blob/main/docs/k2-horizon-7b-fp8.md ).

Using the model

• Input: JSON conversation messages containing a role and text content. Include optional function schemas for tool calling.

• Output: Generated text or structured tool-call requests. Your application validates and executes any requested tools.

The context limit includes input, conversation history, tool definitions, generated reasoning and the final answer. Reserve space for generation; requests exceeding the combined budget are rejected. Check completion status because reasoning can consume the output allowance before a final answer.

Training and measured behavior

IFM reports approximately 20 trillion pretraining tokens per Horizon model, combining web, code, mathematics, science, multilingual, domain-specific and synthetic material. For benchmark results and evaluation context, see the K2 Horizon technical blog (https://ifm.ai/blog/k2/ ). These evaluations were not reproduced on this AWS deployment. A single arithmetic request on the 32K ml.g6.2xlarge configuration took 3.968 seconds for 23 input and 102 completion tokens, including reasoning, with one active sequence. This is an observed functional sample, not a production latency guarantee. See the model details and measurement limits (https://github.com/MBZUAI-IFM/k2-horizon-aws-marketplace/blob/main/docs/k2-horizon-7b-fp8.md ).

Limitations

This offering accepts and generates text. Responses may contain factual errors, defective code or malformed tool arguments. Evaluate outputs for your use case. Browsing, code execution, persistent memory and external data access require application-provided components.

Software fee and license

No software fee. AWS charges separately for the SageMaker instances, storage, data transfer and other infrastructure you provision. Delete deployment resources when they are no longer needed.

The model is distributed under the Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0 ). Included software components retain their respective licenses and notices.

Next steps

Subscribe to this listing and deploy your first endpoint to evaluate K2 Horizon 7B FP8 in your own environment. For questions about deployment, integration or evaluation, reach out to the IFM support team through the contact information in the Support section below.