Source: https://dhruvipaprunia.com/blog/

Writing

# Hi :) this is where I overshare about AI.

[Paper deep dive

### Your model found the right answer. It might never learn to find it again.

What GRPO, a reward-based training method, changes across several tries.

15 min read + optional walkthroughs Reward training

![Rare correct proof](https://dhruvipaprunia.com/blog/a-rare-correct-answer-can-still-get-lost-in-training/cover.png)](https://dhruvipaprunia.com/blog/a-rare-correct-answer-can-still-get-lost-in-training/) [Lab deep dive · Ai2

## The model weights are open. Can we see how the model learned?

Why open model training matters, and how to test the steps that shape an assistant.

8 min read Open models

![Open training notebook](https://dhruvipaprunia.com/blog/the-model-weights-are-open-can-we-see-how-the-model-learned/cover.png)](https://dhruvipaprunia.com/blog/the-model-weights-are-open-can-we-see-how-the-model-learned/) [Paper deep dive

## A million tokens can still lose the right document

Why seeing every document is not the same as finding the right one.

10 min read Retrieval

![Wall of documents](https://dhruvipaprunia.com/blog/a-million-tokens-can-still-lose-the-right-document/cover.png)](https://dhruvipaprunia.com/blog/a-million-tokens-can-still-lose-the-right-document/) [Paper deep dive

## The agent did everything right. The website still said no.

What a verifier can learn when the assistant took the right steps but the website failed.

8 min read AI Agents

![Train ticket attempt: steps completed, verification failed, ticket not booked](https://dhruvipaprunia.com/blog/the-agent-did-everything-right-the-website-still-said-no/thumbnail.png)](https://dhruvipaprunia.com/blog/the-agent-did-everything-right-the-website-still-said-no/) [Paper deep dive

## Can one reward teach an assistant everything?

A reading of MAHALO, where accuracy, human values and tutoring pull an assistant in different directions.

7 min read AI Alignment

![Can one reward teach an assistant everything? cream and teal cover](https://dhruvipaprunia.com/blog/can-one-reward-teach-an-assistant-everything/3-1-amman-cover.png)](https://dhruvipaprunia.com/blog/can-one-reward-teach-an-assistant-everything/) [Reading notes

## The one percent we stopped looking at

Muon, Never Give Up, and the cases an average can hide.

7 min read Machine Learning

![The one percent we stopped looking at cover](https://dhruvipaprunia.com/blog/one-percent-we-stopped-looking-at/1-cover.png)](https://dhruvipaprunia.com/blog/one-percent-we-stopped-looking-at/) [Paper deep dive

## Never Give Up: why easy problems eat the RL budget

A deep dive into the Matthew Effect, and how adaptive sampling gives hard problems another chance.

8 min read Reinforcement Learning

Never Give UpFour attempts are enough for the easy problem. The hard problem stays in the queue and gets another chance.PAPER DEEP DIVE · NEVER GIVE UPStop letting the easy oneseat the budget.EASY PROBLEM✓ ✓ ✓ ✓→ stopHARD PROBLEM✗ ✗ ✗ ✗→ try againdhruvi paprunia · notes on RL](https://dhruvipaprunia.com/blog/never-give-up/) [Paper deep dive

## A coin flip made this model better at math. Or did it?

Notes on "Spurious Rewards", a paper about what RL training really teaches a model.

3 min read Reinforcement Learning

![](https://dhruvipaprunia.com/blog/spurious/39-cover.png)](https://dhruvipaprunia.com/blog/coin-flip-math/) [My research

## The polite sentence that broke my model

How we taught small language models to call tools with GRPO, and why the strictest rule worked best.

6 min read Reinforcement Learning

![](https://dhruvipaprunia.com/blog/grpo/32-cover.png)](https://dhruvipaprunia.com/blog/grpo-tool-use/)

Coming next

**Paper deep dive** One research paper, explained simply

**Lab map** What one AI lab is working on, and why it matters
