Amazon Nova Forge

For customers with intermediate volumes of unstructured data, Nova Forge provides model checkpoints and recipes to introduce data in mid-training, where the propensity to learn from new training data is not set as high as in pre-training. As with pre-training, customers can blend their proprietary data with Amazon Nova-curated training data during the mid-training phase.  This allows the model to absorb domain-specific knowledge while retaining general capabilities like language comprehension and reasoning, but with more conservative learning rate to avoid catastrophic forgetting. 

Supervised Fine-Tuning uses labelled data, such as instruction-response datasets, to teach the model how to respond to patterns of queries. Customers can choose to run Supervised Fine-Tuning on a custom base checkpoint they have created, Nova’s pre-built base checkpoint, or Nova’s instruction-tuned model – depending on their needs and training data availability. As with CPT, customers can mix their proprietary data with Amazon Nova-curated labelled data sets during Supervised Fine-Tuning. This enables customers to train the model for their specialized applications while maintaining broad model capabilities such as instruction-following.

Reinforcement Learning (RL) further refines the model's behavior using reward signals and human feedback. In the RL phase, Nova Forge customers can align their model using feedback from remote reward functions. This allows them to use feedback from custom environments with proprietary tools and verifiers through an API during alignment. Unlike the industry standard of simple reward functions defined in small Python files, this API-based approach enables customers to integrate sophisticated custom environment harnesses and run them at scale. Examples of custom reward functions include physics simulators, complex code evaluation using internal systems with nested tool-calling, and robotics tasks against proprietary testing frameworks.