Preparing data for supervised fine-tuning Part 1: Formatting and quality

Data preparation determines the ceiling of any supervised fine-tuning (SFT) project. You’ve evaluated your foundation model (FM), and out-of-the-box performance isn’t meeting your production requirements. Maybe the model doesn’t follow your output schema reliably, struggles with your domain’s classification taxonomy, or can’t maintain the tone your application demands. The question isn’t whether to customize, it’s how. This post assumes you have decided to fine-tune a foundation model and are evaluating how to prepare data for that work.

Post-training customization provides three distinct levers. Each addresses a different gap between what the model can do today and what you need it to do. Continued pre-training (CPT) ingests large volumes of unstructured domain text to expand the model’s knowledge base. Use CPT when the model lacks familiarity with your domain’s terminology, concepts, or data patterns. Supervised fine-tuning (SFT) trains on curated input-output pairs to reshape the model’s behavior. SFT teaches the model how to respond: following instructions, adhering to schemas, adopting a specific tone, or producing structured outputs. It doesn’t inject new knowledge. It teaches the model to apply what it already knows in the way that you need, an idea sometimes called the Superficial Alignment Hypothesis. Reinforcement fine-tuning (RFT) optimizes behavior through reward signals rather than explicit demonstrations. RFT works when you can programmatically evaluate output quality but can’t easily demonstrate the reasoning path at scale.

These techniques aren’t mutually exclusive. A production pattern is CPT, then SFT, then RFT: first expand knowledge, then shape behavior, then optimize through feedback. In practice, CPT is used less often and only required when the base model lacks critical domain vocabulary or knowledge your task requires. Foundation models like Amazon Nova are already pre-trained on a broad corpus, so SFT followed by RFT is usually sufficient.

This post, the first of a two-part series, covers the foundations of SFT data preparation: quality checks, formatting requirements, and train/evaluation splits. We use code snippets from Amazon Bedrock documentation to illustrate key concepts while keeping the guidance applicable to any model you choose. The second post covers advanced strategies: readiness evaluation, data subset selection and filtering, data augmentation, and data mixing.

Data quality checks

Before you invest in formatting or training infrastructure, audit your raw data. Catching problems early saves substantial time and compute cost downstream.

Accuracy and correctness

Every response in your dataset should be a gold-standard answer you would be comfortable deploying to production. Incorrect examples can teach the model a persistent bad habit that’s difficult to unlearn. This risk is especially acute in SFT, where the model isn’t learning new facts so much as learning which patterns to imitate. A wrong demonstration gets imitated.

The practical upshot is that quality beats quantity by a wide margin. LIMA showed that 1,000 carefully curated examples can match models trained on orders of magnitude more data. AlpaGasus showed that filtering an instruction set down to its cleanest 20 percent can train faster and score higher than the full set. If you’re working with human-annotated data, implement a multi-review process before examples enter your training set.

Diversity of examples

Dataset diversity is one of the strongest predictors of SFT success. Research on supervised fine-tuning scalability identifies two properties that govern how well fine-tuning generalizes. The first is semantic coverage, the breadth of task domains and prompt phrasings represented. The second is information depth, the richness of individual examples. A dataset that covers only a narrow slice of your production traffic will produce a model that performs well on that slice and poorly everywhere else.

Audit your dataset for prompt variation first. Your examples should include the different ways users phrase the same intent, because a model trained only on “Summarize this document” won’t reliably handle “Give me the key takeaways.” Check domain and topic breadth next. If your task spans multiple domains, verify that each is represented proportionally to its production frequency. Then look at difficulty range: include straightforward cases alongside complex, multi-step problems, weighted toward the complexity profile of real traffic. Finally, explicitly include edge cases such as ambiguous inputs, incomplete information, and out-of-scope requests, paired with the responses you want the model to produce in those situations.

A practical approach is to cluster your examples by embedding similarity and inspect the resulting clusters for gaps. Sparse or missing clusters indicate areas where the model will lack training signal. For example, a customer-support dataset might cluster into password resets, billing questions, and shipping updates. If refunds are a common request in production but no cluster covers them, that empty region is your gap. The model will have little signal for refund conversations, so collect or write refund examples before training. The second post in this series shows how to automate this idea with data subset selection methods.

Consistency within similar tasks

While diversity across your dataset is critical, examples that handle the same type of situation should be internally consistent. Inconsistency within the same task type sends contradictory signals about correct behavior. For example, if similar prompts produce both bullet-point and paragraph-form answers, the model won’t learn a reliable default for response structure. If you need concise, two-sentence answers, don’t include paragraph-length responses in your training data.

Deduplication

Duplicate or near-duplicate examples cause the model to overfit on those patterns, inflating their importance relative to the rest of your dataset. Duplication is less critical in SFT than in pre-training, but it can creep in when you merge data from multiple annotators, combine datasets across projects, or use synthetic data generation. In those cases, apply both exact-match and semantic deduplication before training.

Toxicity and safety screening

Scan your dataset for harmful, biased, or inappropriate content. Even if your use case is narrow, the model can internalize patterns from problematic examples and surface them in unexpected contexts. Use automated classifiers, such as the open Llama Guard, to flag content for human review, and establish clear guidelines for what constitutes acceptable training data in your domain.

Data formatting

With quality checks complete, you can structure your data for training. Formatting isn’t just about syntax. How you structure examples shapes the model’s learned behavior, because SFT teaches the model to respond to a specific shape of input, not just its content.

System prompt in the sample

Include a system prompt in your training examples when you plan to use one during inference. The system prompt establishes context, persona, and constraints that shape model behavior. If training data lacks system prompts but inference includes them, you create a distribution mismatch that can degrade behavior.

Conversational format (JSONL)

Most modern SFT pipelines, including Amazon Nova recipes, use a conversational JSONL format where each line is a self-contained JSON object representing a conversation. The Amazon Nova 2.0 models use the Converse API format:

{
  "schemaVersion": "bedrock-conversation-2024",
  "system": [{"text": "You are a helpful coding assistant."}],
  "messages": [
    {
      "role": "user",
      "content": [{"text": "Write a Python function to check if a string is a palindrome."}]
    },
    {
      "role": "assistant",
      "content": [
        {"text": "def is_palindrome(s):\n    cleaned = s.lower().replace(' ', '')\n    return cleaned == cleaned[::-1]"}
      ]
    }
  ]
}

Key formatting rules to follow:

• One JSON object per line, with no pretty-printing across multiple lines.
• Validate every line parses as valid JSON before uploading.
• Maintain strict role alternation between user and assistant turns.
• Include system messages when your production setup uses them.

Reasoning traces

For models with reasoning capabilities (such as Amazon Nova 2.0 with reasoning_enabled: true), include intermediate thinking steps using the reasoningContent field in assistant turns. Training on reasoning traces is what transfers the behavior popularized as chain-of-thought from a prompting trick into the model itself.

{
  "schemaVersion": "bedrock-conversation-2024",
  "system": [{"text": "You are a financial analyst. Provide data-driven answers with supporting calculations."}],
  "messages": [
    {
      "role": "user",
      "content": [{"text": "Calculate YoY revenue growth. 2024: $4.2M, 2025: $5.1M"}]
    },
    {
      "role": "assistant",
      "content": [
        {
          "reasoningContent": {
            "reasoningText": {
              "text": "The user asks for year-over-year revenue growth. I need to calculate the percentage change: (new - old) / old x 100. That gives (5.1 - 4.2) / 4.2 x 100 = 21.43%."
            }
          }
        },
        {"text": "YoY revenue growth is approximately 21.4 percent: ($5.1M - $4.2M) / $4.2M x 100."}
      ]
    }
  ]
}

The hardest part of using reasoning traces isn’t the format. It’s getting traces that actually teach the model something. Useful traces are faithful to the answer, meaning the reasoning actually leads to the final response. They’re proportional to difficulty: short problems get short traces and long problems get long ones, a trade-off that difficulty-aware trace compression makes explicit. They’re complete, with no thought leaps, because expert-written rationales often skip intermediate steps the model hasn’t yet learned. And quality matters more than quantity: s1 and LIMO both show that under 1,000 carefully curated reasoning demonstrations can elicit strong reasoning in a capable base model. The second post in this series covers how to source these traces at scale through distillation and self-generation.

Use plain text for reasoning content and keep it directly relevant to the problem-solving process. When reasoning is enabled during training, it should also be enabled during inference for consistent behavior. Keep in mind that training on a non-reasoning dataset with reasoning_enabled: true can cause the model to lose its reasoning capabilities, because it learns to generate responses without applying reasoning.

Tool calling and multimodal formats

SFT supports training models for tool use (function calling) and multimodal understanding (documents, images, video). For tool calling, toolUse blocks appear in assistant turns and toolResult blocks in user turns, each referencing a unique toolUseId:

{
  "schemaVersion": "bedrock-conversation-2024",
  "system": [{"text": "You are an expert in composing function calls."}],
  "toolConfig": {
    "tools": [
      {
        "toolSpec": {
          "name": "getItemCost",
          "description": "Retrieve the cost of an item from the catalog",
          "inputSchema": {
            "json": {
              "type": "object",
              "properties": {
                "item_id": {
                  "type": "string",
                  "description": "The ASIN of item to retrieve cost for"
                }
              },
              "required": ["item_id"]
            }
          }
        }
      }
    ]
  },
  "messages": [
    {
      "role": "user",
      "content": [{"text": "How much does item id-456 cost?"}]
    },
    {
      "role": "assistant",
      "content": [
        {
          "toolUse": {
            "toolUseId": "getItemCost_0",
            "name": "getItemCost",
            "input": {"item_id": "id-456"}
          }
        }
      ]
    },
    {
      "role": "user",
      "content": [
        {
          "toolResult": {
            "toolUseId": "getItemCost_0",
            "content": [
              {"text": "{"name": "getItemCost", "results": {"cost": "$29.99"}}"}
            ]
          }
        }
      ]
    },
    {
      "role": "assistant",
      "content": [
        {"text": "Item id-456 costs $29.99."}
      ]
    }
  ]
}

Key constraints for tool calling data:

• toolUse must appear only in assistant turns, and toolResult only in user turns.
• Each toolResult must reference a valid toolUseId from a preceding assistant turn, used exactly once per conversation.
• toolResult content should be text or JSON only.
• The inputSchema within toolSpec must be a valid JSON Schema object.

For multimodal training, document and image content blocks appear in user turns alongside text, with sources referencing Amazon Simple Storage Service (Amazon S3) locations:

{
  "schemaVersion": "bedrock-conversation-2024",
  "system": [{"text": "You are a document analysis assistant."}],
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "document": {
            "format": "pdf",
            "name": "quarterly_report",
            "source": {"s3Location": {"uri": "s3://<your-bucket-name>/report.pdf"}}
          }
        },
        {"text": "Summarize the key findings from this quarterly report."}
      ]
    },
    {
      "role": "assistant",
      "content": [
        {"text": "The quarterly report highlights three key findings..."}
      ]
    }
  ]
}

Staying close to the original model’s chat template

Every foundation model is post-trained with a specific chat template: the exact token sequences that delimit system prompts, user turns, and assistant turns. Your training data should match this template exactly. Deviating from it, even subtly, forces the model to learn a new input format alongside the new behavior, splitting its learning budget across two objectives. In the worst case, template mismatch causes the model to ignore its system prompt or produce malformed outputs at inference time.

Practical guidelines for template alignment:

• Use the model provider’s official formatting utilities. For Amazon Nova, use the Converse API schema. For open source models, use the tokenizer’s apply_chat_template() method from Hugging Face Transformers.
• Never hand-roll delimiters. If you’re converting data from another format such as ShareGPT or Alpaca, run it through the target model’s chat template function rather than manually inserting role marker strings.
• Validate tokenization round-trips. After formatting, tokenize a sample of your data and decode it back to text. Confirm that role boundaries, special tokens, and content are preserved exactly.
• Match inference-time structure during training. If your production pipeline injects a system prompt, tool definitions, or retrieval context at specific positions, your training data should include those elements in the same positions.

Train/evaluation split

Hold out 10–20 percent of your data as an evaluation set. This isn’t optional. Without it, you can’t distinguish genuine learning from overfitting. The evaluation set should be representative of your production distribution, not a random slice that might under-represent rare but important categories.

For small datasets (under 1,000 samples), use stratified splitting so each task category appears in both training and evaluation sets. For larger datasets, random splitting usually suffices, but verify the distribution afterward. Before training, establish baseline performance on your evaluation set using the unmodified model. This gives you a concrete target to beat and helps you quantify the value of supervised fine-tuning.

Conclusion

Data preparation is the most impactful activity in any SFT workflow. The patterns covered here, including systematic quality checks, schema-compliant formatting, and a representative train/evaluation split, apply whether you’re fine-tuning for a narrow classification task or a complex multi-turn dialogue system. Invest the time upfront, and your training jobs will converge faster, generalize better, and avoid the costly cycle of debugging model behavior that traces back to data problems.

In the second post of this series, we cover the advanced side of SFT data preparation: evaluating data readiness with learning curves, selecting high-value subsets, augmenting your data, and mixing data sources to prevent catastrophic forgetting.

To get started, visit the Amazon Bedrock console to explore model customization options. Learn more about Amazon Nova on the service detail page, review the Amazon Nova documentation for customization guides, and explore the Amazon SageMaker HyperPod recipes repository for ready-to-run training configurations. For related reading, see Customize Amazon Nova models with Amazon Bedrock fine-tuning on the AWS Machine Learning Blog.

About the authors