this is big... 50 AI researchers from Bytedance, Alibaba, Tencent, and other labs/universities just published a 300-page paper with surprising lessons about coding models and agents (data, pre and post-training, etc).
key highlights:
> small LLMs can beat proprietary giants RL (RLVR specifically) gives small open-source models an edge over big models in reasoning. a 14B model trained with RLVR on high-quality verified problems can match the performance of OpenAI's o3.
> models have a hard time learning Python. mixing language models during pre-training is good, but Python behaves different from statically typed languages. languages with similar syntax (Java and C#, or JavaScript and TypeScript) creates high positive synergy. mixing Python heavily into the training of statically typed languages can actually hurt because of Python's dynamic typing.
> not all languages are equal (coding scaling laws) the amount of data required to specialize a model on a language drastically depends on the language. paper argues like C# and Java are easier to learn (less training data required). languages like Python and Javascript are actually more tricky to learn, ironically (you see AI most used for these languages :)
> MoE vs Dense (ability vs stability) MoE models offer higher capacity, but are much more fragile during SFT than dense models. hyperparams in training have a more drastic effect in MoE models, while dense models are more stable. MoE models also require constant learning rate schedules to avoid routing instability.
> code models are "insecure" by default (duh) training on public repos makes models learn years of accumulated insecure coding patterns. safety fine-tuning often fails to work much on code. a model might refuse to write a hate speech email but will happily generate a SQL-injection vulnerable function because it "works."
a senior engineer at google just dropped a 400-page free book on docs for review: agentic design patterns.
the table of contents looks like everything you need to know about agents + code: > advanced prompt techniques > multi-agent patterns > tool use and MCP > you name it
longer context doesn't generate better responses. it can even hurt your llm/agent. 1M context window doesn't automatically make models smarter as it's not about the size; it's how you use it.
here are 4 types of context failure and why each one happens:
1. context poisoning: if hallucination finds its way into your context, the agent will rely on that false information to make its future moves. for example if the agent hallucinates about the "task description", all of its planning to solve the task would also be corrupt.
2. context distraction: when the context becomes too bloated, the model focuses too much on it rather than come up with novel ideas or to follow what it has learned during training. as Gemini 2.5 Pro technical report points out, as context grows significantly from 100K tokens, "the agent showed a tendency toward favoring repeating actions from its vast history rather than synthesizing novel plans".
3. context confusion: everyone lost it when MCPs became popular, it seemed like AGI was achieved. I suspected there is something wrong and there was: it's not just about providing tools, bloating the context with tool use derails the model from selecting the right one! even if you can fit all your tool metadata in the context, as their number grows, the model gets confused over which one to pick.
4. Context Clash: if you exchange conversation with a model step by step and provide information as you go along, chances are you get worse performance rather than providing all the useful information at once. one the model's context fills with wrong information, it's more difficult to guide it to embrace the right info. agents pull information from tools, documents, user queries, etc. and there is a chance that some of these information contradict each other, and it's not good new for agentic applications.
60+ Generative AI projects for your resume. grind this GitHub repo if you want to level up: > LLM fine-tuning and applications > advanced RAG apps > Agentic AI projects > MCP and A2A (new)
this book actually exists for free, “the little book of deep learning”. best to refresh your mind about DL basics: > foundations of machine learning > how models train > common layers (dropout, pooling…) > basic intro to LLMs actually optimized for mobile.
The best researchers from DeepSeek, OpenAI, Microsoft, and ByteDance explored RL and Reasoning in LLMs,
Here's some of their key findings:
1/ RL can further improve distilled models. These models are essentially SFT fine-tuned with the data generated by larger models, and the SFT+RL combo does not disappoint.
This is verified in the DeepSeek-R1 paper.
2/ both GRPO and PPO algorithms suffer from length bias; they encourage longer responses. This can be tackled by introducing explicit rewards based on the length of the answer.
3/Most reasoning research is focused on code and math. But training models on logic puzzles improves them for mathematical tasks too.
This shows the RL reasoning is generalized beyond the specific domain knowledge.
Previous research also shows RL can be a great generalizer.
4/The reasoning might not be only induced by RL; it might already be hidden in the base models due to the pre-training and CoT data they were trained on.
So while RL does wake up the reasoning beast, maybe it's not the only solution (e.g. other methods such as distillation)
5/ back to the length bias; reasoning models tend to generate longer responses for wrong answers. RL might be the culprit.
RL favours longer answers when the reward is negative, to dilute the penalty per individual token and lower the loss.
This might explain the "aha" moments!
6/ OpenAI's competitive programming paper showed an interesting finding:
o3 can learn its own test-time strategies (like writing an inefficient but correct solution to verify the answer of an optimized solution)
RL helps LLMs develop their own reasoning & verification methods. The recent article by @rasbt helped me a lot in getting a broad view of the recent research on reasoning models.
He also lists more influential papers on this topic, It's a must-read if you're interested.
OpenAI just released a 34-page practical guide to building agents,
Here's 10 things it teaches us:
1➜ agents are different from workflows: they are complete autonomous systems that perform tasks on your behalf. many applications use LLMs for workflows, but this is not an agent.
2➜ use them for tricky stuff: complex decision making, dynamic rules, unstructured data
3➜ core recipe: each agent has three main components: Model (the brain), Tools, Instructions on how to behave
4➜ choose the right brain: set up evals to get a baseline performance, use a smart model to see what's possible, gradually downgrade the model for cost and speed
5➜ tools are key: choose well-defined and tested tools. an agent needs tools to retrieve data and context, and take actions.
6➜ instruction matters A LOT: be super clear telling the agent its goals, steps, and rules. Vague instructions = unpredictable agent. Be explicit.
7➜ start simple, then scale: often a single agent with several tools is ok. don't jump to complex multi-agent systems immediately.
8➜ if you use multi-agents: you can have a "manager" agent directing traffic to specialist agents, or have agents hand off tasks to each other.
9➜ gaurdrails are a MUST: check user input for weird stuff, make sure the agent isn't about to do something risky, filter out private info, block harmful content. Don't let it run wild.
10➜ build and plan for humans: start small, test, improve. always have a plan for when the agent gets stuck or is about to do something high-risk.
Google published a 69-page whitepaper on Prompt Engineering and its best practices, a must-read if you are using LLMs in production: > zero-shot, one-shot, few-shot > system prompting > chain-of-thought (CoT) > ReAct
The best researchers from Yale, Stanford, Google DeepMind, and Microsoft laid out all we know about Agents in a 264-page paper [book],
Here are some of their key findings:
They build a mapping of different agent components, such as perception, memory, and world modelling, to different regions of the human brain and compare them:
- brain is much more energy-efficient - no genuine experience in agents - brain learns continuously, agent is static
An agent is broken down to: - Perception: the agent's input mechanism. can be improved with multi-modality, feedback mechanisms (e.g., human corrections), etc. - Cognition: learning, reasoning, planning, memory. LLMs are key in this part. - Action: agent's output and tool use.
Agentic memory is represented as: - Sensory memory or short-term holding of inputs which is not emphasized much in agents. - Short-term memory which is the LLM context window - Long-term memory which is the external storage such as RAG or knowledge graphs.
The memory in agents can be improved and researched in terms of: - increasing the amount of stored information - how to retrieve the most relevant info - combining context-window memory with external memory - deciding what to forget or update in memory
The agent must simulate or predict the future states of the environment for planning and decision-making.
ai world models are much simpler than the humans' with their causal reasoning (cause-and-effect) or physical intuition.
LLM world models are mostly implicit and embedded.
EMOTIONS are a deep aspect of humans, helping them with social interactions, decision-making, or learning.
Agents must understand emotions to better interact with us.
But rather than encoding the feeling of emotions, they have a surface-level modelling of emotions.
Perception is the process by which an agent receives and interprets raw data from its surroundings.
What, How, Where, and How Well? This paper reviews test-time scaling methods and all you need to know about them: > parallel, sequential, hybrid, internal scaling > how to scale (SFT, RL, search, verification) > metrics and evals of test-time scaling
