Trending ML Papers

What it is. Orca is an attempt to build a single foundation model that learns how the world changes over time, rather than one model for text, one for images, and another for robot actions. Its core idea is 'next-state prediction': feeding the model 125,000 hours of video plus 160 million event descriptions and asking it to represent the world as a continuously evolving internal state. Once that shared representation is trained, small task-specific heads plug in on top to produce text, images, or robot actions from the same frozen brain.

Where it fits. Today's AI headlines are dominated by three separate species of model: language models that predict the next token, video/image generators that predict the next frame, and robot policies that predict the next action. Yann LeCun and others have argued for years that a real 'world model' — one that understands cause, physics and dynamics — should sit under all three. Orca, from the Beijing Academy of AI, is one of the most ambitious public attempts yet to unify them, riding the same wave as Meta's V-JEPA line and various embodied-AI initiatives.

Why it matters. If a unified world model actually works, it changes the economics of building AI products: instead of stitching together a chat model, an image model, and a robot controller, teams could share one backbone and swap tiny decoders per use case. Orca already reports beating similarly sized specialist models on text, image prediction, and embodied action tasks — while keeping the backbone frozen — which hints at a real re-use dividend. The caveats are big: the paper itself flags limitations, we do not yet know how it holds up outside curated benchmarks, and 'general world model' claims have historically been oversold.

What it is. This paper studies a very practical question: when an AI agent is asked to do something impossible or under-specified — 'buy me a pink living-room pillow' in a store that has none — does it know to stop, or does it just keep clicking? The authors build a benchmark of over 28,000 tasks across web shopping, terminal (command-line) work, and question answering, and test 13 models plus two popular agent frameworks. They also introduce a technique called CONVOLVE that writes short 'stopping rules' distilled from past trajectories and drops them into the agent's context — no retraining required.

Where it fits. Agentic AI — models that browse, click, run shell commands, or use tools on your behalf — is the hottest product category of 2026, from AI assistants that book travel to coding copilots that ship pull requests. Almost all published benchmarks measure success rates on tasks that can be solved. Real users, on the other hand, routinely ask for things that are ambiguous, contradictory, or simply not doable. The research community is only beginning to measure that failure mode.

Why it matters. For anyone shipping an agent product, this is the difference between a tool that fails gracefully and one that racks up API calls, blows through a budget, and eventually returns nonsense. The results are unflattering: even top models only abstain in time on 26.7% of impossible web-shopping tasks, and bigger models are sometimes worse at knowing when to stop. Meanwhile, CONVOLVE more than doubles that number on Llama-3.3-70B without any weight updates, suggesting there is low-hanging fruit for teams to pick up with pure prompt/context engineering.

What it is. Instead of calling a giant hosted LLM every time you need to classify a log line or repair broken JSON, this paper proposes a 'compile once, run cheap' pattern. You write the function you want in plain English; a 4-billion-parameter 'compiler' model turns that description into a tiny 23 MB file (a small set of extra weights called a LoRA adapter — think of it as a plug-in for a base model); and then a fixed, tiny 0.6B model on your laptop or phone runs the resulting 'neural program' locally. The compiled programs behave like traditional software artifacts: you can name them, version them, share them, and call them offline.

Where it fits. Most production teams outsource fuzzy, hard-to-specify tasks — filtering, ranking, extracting, cleaning — to OpenAI or Anthropic APIs because writing rules is brittle and slow. That comes with cost, latency, privacy concerns, and the well-known risk that a vendor silently changes the model. There is a parallel push in the field toward 'small model' futures and toward compiling behavior into weights (SakanaAI's Text-to-LoRA, hypernetworks, adapters). PAW pushes this direction into a full developer experience: a compiler, an interpreter, a package model.

Why it matters. The demonstration numbers are striking: a 0.6B model running compiled PAW programs beats prompting a 32B model on their benchmark while using about 1/50th the memory and hitting 30 tokens/second on a MacBook M3. If this generalizes, it becomes economically reasonable to bake LLM-style behavior directly into apps, browser extensions, and edge devices — with no per-call API bill and no dependency on a remote provider. The open questions are how far you can push accuracy on truly complex tasks, and whether the compiler itself becomes the new bottleneck to update and secure.

What it is. To train a coding agent, you need a way to check whether its proposed fix actually works. Today that means spinning up a Docker container for each project, installing every dependency, and running its test suite — enormously expensive at scale. Dockerless replaces that with an AI 'judge' that reads the code repository directly: it generates specific verification questions ('what tests would confirm this fix?', 'is this function even used?'), sends sub-agents to hunt for answers in the code, then rules the patch correct or not based on the collected evidence. No container, no test run.

Where it fits. The last two years produced SWE-bench, SWE-Gym, Multi-SWE-RL and a wave of coding agents from OpenAI, Anthropic, Cognition, ByteDance, and others. The uncomfortable secret of that ecosystem is that scaling training data is bottlenecked by devops: building reproducible environments for thousands of open-source repos is brutal, and for private or legacy code it is often impossible. Prior 'lightweight' verifiers looked only at surface diffs and were too shallow. Dockerless, from Shanghai Jiao Tong and ByteDance's Douyin group, is the first strong attempt to make the verifier itself an agent that reads code.

Why it matters. If verifiers can go environment-free without losing accuracy, the pool of code that can be used to train coding agents grows dramatically — including messy private codebases at enterprises. The paper reports matching a Docker-based post-training pipeline on SWE-bench Verified (62.0% resolve rate), Multilingual (50.0%) and Pro (35.2%), while beating the strongest open-source verifier by 14.3 AUC points. The caveat is that a learned judge can only be as trustworthy as the training signal it distilled from; teams betting on it should still spot-check with real execution.

What it is. This paper is about how to make a small model as smart as possible by learning from a bigger one — a process called distillation. The authors identify a subtle bug they call 'privilege illusion': when you give the teacher extra hints (like a hidden answer or a labeled bounding box) it looks smarter, but a lot of that apparent superiority comes from information the student can never actually access, so the student ends up faking it rather than getting genuinely better. Their fix, DOPD, decides on a per-token basis whether to imitate the privileged teacher, imitate a version of the student itself, or lean back — routing each little decision to the right supervisor.

Where it fits. Distillation is the workhorse behind almost every 'small, fast, cheap' model shipping today — from Qwen and Gemma variants to on-device assistants. 'On-policy distillation' is the current best-practice flavor, where the small model tries something, the big model comments on every token, and the small model updates. Practitioners have been quietly noticing that dumping privileged context into the teacher gives short-term wins but destabilizes training later. DOPD, from a NUS/CUHK/PKU/JD collaboration, is one of the first papers to name and quantify that pathology.

Why it matters. Distillation quality directly determines how good the small model in your product can be for a given inference budget. DOPD reports 7.5 and 6.0 point average improvements over vanilla on-policy distillation on language and vision-language settings respectively, plus better stability and out-of-distribution robustness. For product teams building compressed variants of frontier models, the practical takeaway is more important than the technique itself: be suspicious of teachers that look better because they had extra information — those gains may not survive deployment.