Smart Training, Not Bigger Models: the 15B Apriel-1.5-Thinker

Paper at a Glance

• Paper Title: Apriel-1.5-15B-Thinker: Mid-training is all you need
• Authors: Shruthan Radhakrishna, Aman Tiwari, Aanjaneya Shukla, Masoud Hashemi, Rishabh Maheshwary, et al.
• Affiliation: SLAM Lab, ServiceNow
• Published in: arXiv, October 2025
• Link to Paper: https://arxiv.org/abs/2510.01141

The Gist of It: TL;DR

In one sentence: This paper introduces Apriel-1.5-15B-Thinker, a 15-billion parameter open-weights multimodal model that achieves top-tier reasoning performance by proving that a meticulously designed training strategy can close the gap with models that are orders of magnitude larger.

Why It Matters: The Big Picture

The world of AI is currently dominated by a simple, powerful, and incredibly expensive philosophy: “scale is all you need.” Giants in the field build ever-larger models, consuming vast amounts of data and computational power to push the boundaries of performance. While effective, this creates a massive barrier to entry, leaving smaller research labs, universities, and companies on the sidelines. The key question for the democratization of AI is: can we get to the frontier without a frontier-sized budget?

This paper from ServiceNow’s SLAM Lab answers with a resounding “yes.” It challenges the brute-force scaling paradigm by focusing on training design rather than sheer size. The authors present a compact, 15-billion parameter model that can be deployed on a single GPU yet competes with behemoths in the field. This work provides a practical blueprint for building powerful, accessible, and efficient AI systems.

The Core Idea: How It Works

The central philosophy of the paper is encapsulated in its subtitle: “Mid-training is all you need.” Instead of the standard two-step process of pre-training and fine-tuning, the authors introduce a deliberate, multi-stage methodology that focuses heavily on the intermediate training phases to build sophisticated reasoning capabilities.

1. The Starting Point: Smart Scaling

The team didn’t train their model from scratch, which is computationally expensive. Instead, they started with a strong, existing 12-billion parameter open-source model, Pixtral-12B. Their first move was to efficiently grow it into a 15-billion parameter model using depth upscaling—adding more layers to the model’s decoder. This is a far more efficient way to increase a model’s capacity than training a larger model from the ground up.

2. The “Mid-Training” Core: Staged Continual Pre-Training (CPT)

This is the heart of the innovation. After upscaling, the model undergoes a two-stage continual pre-training process designed to systematically build its reasoning skills.

• CPT Stage 1: Building a Solid Foundation. The model is first trained on a diverse mix of data. 50% is high-quality text focused on reasoning (math, science, code), while 30% is multimodal data (chart analysis, document understanding, image descriptions). This phase establishes broad textual and visual understanding.
• CPT Stage 2: Honing Visual Reasoning. Next, the training becomes more targeted. The authors use a synthetic data generation pipeline to create custom training examples for specific visual skills: image reconstruction (understanding part-whole relationships), visual matching (fine-grained discrimination), object detection (grounding), and counting.

As shown in Table 1 of the paper, this second CPT stage provides a significant boost. For example, performance on the MathVerse (Vision Dominant) benchmark jumped by nearly 10 points after Stage 2, proving the effectiveness of this targeted curriculum.

3. The Final Polish: High-Quality Supervised Fine-Tuning (SFT)

The final step is to teach the model to be a helpful assistant. The authors curated a massive dataset of high-quality instruction-response pairs. Crucially, each response includes an explicit reasoning trace, showing the model the step-by-step logic to arrive at an answer. This “chain-of-thought” data is vital for developing transparent and reliable reasoning. To make this process cost-effective, they used a powerful open-source model (gpt-oss-120b) to help generate the training data.

Key Experimental Results

The results demonstrate that this thoughtful approach pays off, allowing Apriel-1.5-15B-Thinker to punch far above its weight class.

• Competitive with the Giants: On the Artificial Analysis Intelligence Index, a composite benchmark of ten challenging reasoning tasks, Apriel-1.5-15B-Thinker scores a 52. As shown in Figure 2, this matches the performance of DeepSeek-R1-0528 and is competitive with proprietary systems like Gemini-2.5-Flash, despite being significantly smaller.
  

• Strong Multimodal Performance: Across ten different image benchmarks, the model’s performance is, on average, within just five points of leading proprietary models like Gemini-2.5-Flash and Claude Sonnet-3.7 (Figure 4). It particularly excels at tasks that blend vision and text, such as document and chart understanding (CharXiv).
  

• The Efficiency King: Figure 3 plots model intelligence against size. Apriel-1.5-15B-Thinker sits firmly in the “most attractive quadrant,” offering a superior cost-to-intelligence trade-off. It delivers high-end performance in a package that is practical and deployable for organizations with limited infrastructure.
  

A Critical Look: Strengths & Limitations

Strengths / Contributions

• A Blueprint for Efficiency: The paper’s primary contribution is a clear and effective recipe for building frontier-level AI without needing massive scale. The “mid-training” philosophy is a powerful counter-narrative to the “scale is everything” mindset.
• Democratizing High-Performance AI: By releasing the model weights, training recipes, and evaluation protocols under a permissive license, the authors have made a major contribution to the open-source community, enabling wider access to and research on powerful multimodal models.
• Validated Data-Centric Approach: The work successfully demonstrates that a carefully curated, staged data curriculum can unlock sophisticated reasoning abilities, isolating the impact of training design from sheer parameter count.

Limitations / Open Questions

• Gap in Purely Visual Logic: The authors note that while the model excels at tasks blending text and vision, its performance is more moderate on tasks requiring purely visual reasoning (like LogicVista). There remains a gap between understanding a document’s structure and solving a complex visual puzzle.
• Reliance on Upstream Models: The methodology’s efficiency comes from building upon existing models (Pixtral-12B for the base, gpt-oss-120b for annotation). This means its ultimate performance is partially tethered to the quality of these upstream components.
• No Preference Tuning: The model was trained without reinforcement learning (RLHF) or preference optimization. While this cleanly demonstrates the power of their CPT+SFT pipeline, it may lack the nuanced alignment and safety guardrails found in leading commercial models that undergo extensive post-training.

Contribution Level: Significant Improvement. This paper does not introduce a fundamentally new architecture. Instead, it offers a highly significant and practical methodology that reframes how we think about building powerful models. By proving the immense value of strategic “mid-training,” it addresses the critical bottleneck of computational cost and makes a compelling case that smarts can, to a large extent, beat scale.

Conclusion: Potential Impact

Apriel-1.5-15B-Thinker is more than just another model release; it’s a proof of concept with profound implications. It shows that the path to advanced AI reasoning is not solely paved with more GPUs and larger parameter counts. For researchers and organizations worldwide, this work offers a more accessible, sustainable, and democratized approach to developing frontier AI. The future of AI may not just be bigger, but also smarter in how it learns.