News

Daily AI-curated picks from arxiv and lobste.rs.

• May 12, 2026 arxiv Git for AI Agents: A Runtime That Snapshots Every Decision

  What if you could `git checkout` to any point in an AI agent's execution history? Shepherd builds exactly that — a typed, forkable execution trace that turns meta-agent operations into formally verified functions.

• May 11, 2026 arxiv Escaping the Memory Wall: Stencil Computations on a 850,000-Core Wafer-Scale Chip

  Cerebras's wafer-scale engine rewrites the rules for memory-bound scientific kernels — what happens when your chip has more on-die SRAM than a server rack has L3 cache?

• May 10, 2026 arxiv Why Transformers Obsess Over the First Token: The Mechanical Origin of Attention Sinks

  A mechanistic dissection of one of LLM inference's most mysterious phenomena reveals that attention sinks aren't a bug or an artifact — they're a structural inevitability, with concrete implications for optimizing KV caches and long-context inference.

• May 9, 2026 arxiv From Patch to Exploit: AI Reconstructs Vulnerabilities from Binary Linux Updates

  An LLM-powered agent analyzes binary package diffs to reconstruct the underlying vulnerability before a CVE advisory is even published — closing the dangerous window between patch release and public disclosure.

• May 8, 2026 arxiv SIMD All the Way Down: Running LLMs on Your CPU with Ternary Weights

  Litespark uses custom SIMD intrinsics and ternary weight quantization to bring LLM inference to ordinary x86 CPUs — no GPU, no cloud, just aggressive low-level optimization.

• May 6, 2026 arxiv Stop Re-Prefilling: Quantized KV-Cache Handoff for Multi-Agent Edge LLMs

  When chaining LLM agents on constrained edge hardware, QKVShare shows you can hand off compressed KV caches between models — skipping expensive re-computation — using token-level mixed-precision allocation that preserves context fidelity without blowing memory budgets.

• May 5, 2026 arxiv The Same Model, Different Machine: How Hosting Transforms Open-Weight LLMs

  When you call a hosted 'open-weight' LLM API, you're not just running model weights — you're getting an opaque service layer that silently shapes outputs through batching strategies, quantization choices, and infrastructure decisions that vary wildly between providers.

• May 4, 2026 arxiv GPU Schedulers Are Blind to AI Agents — Here's the Fix

  Current GPU clusters treat each LLM call in a multi-step agent workflow as independent, discarding gigabytes of KV state between steps; SAGA makes the entire agent workflow the scheduling atom, slashing latency by up to 8x.

• May 3, 2026 arxiv Your LLM's Activations Betray Every Step of a Social Engineering Attack

  Multi-turn prompt injection follows a predictable attack trajectory that leaves a readable fingerprint in the model's internal activation space — even when every individual message looks completely benign.

• May 2, 2026 arxiv SpMM on Wafer-Scale Silicon: Cerebras CS-3 vs. the GPU Orthodoxy

  Sparse matrix multiplication kernels on the Cerebras CS-3 expose surprising performance trade-offs that challenge GPU-centric assumptions about AI inference acceleration.

• May 1, 2026 arxiv Checkpoint/Restore for AI Agent Sandboxes That Actually Understands What's Running

  Crab brings semantics-aware C/R to agent microVMs — enabling RL rollout branching, spot execution, and safe rollback without treating every byte of state as equally important.

• Apr 30, 2026 arxiv Inside the Black Box: Reverse-Engineering NVIDIA's GPU Command Streams

  Researchers cracked open NVIDIA's closed-source userspace driver to reveal the actual low-level hardware commands generated from CUDA API calls — exposing the hidden translation layer that every GPU workload silently depends on.

• Apr 29, 2026 arxiv Two Clever Algorithms for Faster Parallel Prefix Scans Over Message-Passing Networks

  New exclusive scan algorithms that squeeze better performance out of bounded, one-ported distributed systems — the kind of low-level primitive that every high-performance collective operation depends on.

• Apr 28, 2026 arxiv Your LLM's Layers Are a Lie Detector: Runtime Fingerprinting for Backdoors and Jailbreaks

  Instead of patching prompts or retraining models, this paper shows that malicious inputs leave a distinctive signature in *how* layer activations converge — a cheap, architecture-agnostic probe you can wire into any inference stack.

• Apr 27, 2026 arxiv One Fast Path for All: The Cloud Network Stack That Lets Tenants Define Their Own Protocols

  Chamelio collapses the redundant guest/host packet processing layers in cloud networking into a single optimized path while still letting each tenant run custom protocol logic — slashing CPU cost and tail latency without sacrificing isolation.

• Apr 26, 2026 arxiv RAG for Reasoning: Storing Reusable Thinking Skills to Cut LLM Token Waste

  What if LLMs could cache hard-won reasoning patterns instead of rediscovering them from scratch every time — fewer tokens, better answers?

• Apr 25, 2026 arxiv Scaling Vector Search: Distributed Product Quantization with Dask

  When your ANN index outgrows a single machine, here's how to parallelize product quantization and inverted indexing across a cluster — with real throughput numbers.

• Apr 24, 2026 arxiv The Hidden Token Tax Killing Your Agentic Workflows

  Every MCP-connected LLM agent silently burns 10,000–100,000 tokens per turn injecting tool schemas it will never use — here's the dynamic gating approach that eliminates that overhead.

• Apr 23, 2026 arxiv Teaching LLMs to Write Their Own Regex: A Smarter Way to Parse Distributed System Logs

  Rather than choosing between brittle regex and slow LLMs, DeepParse uses LLMs to synthesize regex masks offline, giving distributed systems fast, structured log parsing that adapts to evolving formats without sacrificing throughput.

• Apr 21, 2026 arxiv Mid-Flight Course Correction: How LLMs Can Catch and Fix Their Own Reasoning Errors During Generation

  By watching the residual stream at each generation step and steering the KV-cache when a 'phase shift' is detected, LPSR lets a model recover from wrong reasoning paths without restarting — treating inference as a controllable dynamical system rather than a one-way stream.

• Apr 20, 2026 arxiv Logarithmic-Time Geometry in Self-Organizing Matter: Distributed Algorithms at the Physical Layer

  Researchers cracked a key structural decomposition problem in programmable matter — nanoscale agents with only local communication achieving O(log n) time complexity — blurring the line between distributed algorithms and physical computing substrates.

• Apr 19, 2026 arxiv Rethinking SmartNICs: Stream Computation as a First-Class Primitive

  SCENIC breaks the programmability-vs-performance deadlock in SmartNICs by embedding stream computation directly into the device, enabling the kind of flexible, high-throughput in-network processing that modern AI datacenters desperately need.

• Apr 18, 2026 arxiv Breaking the KVCache Barrier: Cross-Datacenter LLM Inference Is Coming

  By exploiting sparse attention patterns in next-generation models, prefill could move to entirely different datacenters from decode — upending the core constraint that has kept LLM serving tightly coupled within a single cluster.

• Apr 17, 2026 arxiv Symbolic Superoptimization Comes to Tensor Programs

  Prism treats tensor program optimization as a symbolic search problem, compactly representing entire families of programs at once — a fundamentally different approach from today's pattern-matching compilers like XLA or TVM.

• Apr 16, 2026 arxiv Git Merge Conflicts Are a Line-Based Lie — Token Merging Fixes That

  A universal merge strategy that operates on tokens instead of lines could silently eliminate the spurious conflicts that have frustrated developers every time someone reformats a file or refactors a function.

• Apr 14, 2026 arxiv Multikernel OS Design: Rethinking Serverless Density from First Principles

  Can a multikernel architecture give serverless providers the holy trinity of high deployment density, strong isolation, and low latency — without sacrificing any of the three?

• Apr 13, 2026 arxiv What It Actually Takes to Run the World's Fastest Supercomputer at Full Speed

  Three campaigns running the HPL benchmark on Aurora's exascale hardware expose the unglamorous engineering — cross-layer coordination, operational discipline, and hard-won lessons — that separates a machine that benchmarks fast from one that sustains it.

• Apr 12, 2026 arxiv Beating the KV Cache Bottleneck: Smarter Offloading for Long-Context LLM Inference

  As context windows grow to millions of tokens, KV cache memory becomes the silent killer of inference throughput — this paper shows how to offload it without wrecking your latency.

• Apr 11, 2026 arxiv City-Scale Space Syntax: GPU-Accelerated Graph Analysis for Urban Movement Prediction

  Researchers cracked the O(n²) wall of visibility graph analysis using GPU-parallelized HyperBall sketches, scaling a classical architectural tool from single buildings to entire cities for the first time.

• Apr 10, 2026 arxiv Stop Wasting Your GPU: Static Partitioning and CPU Offloading as the Fix for Chronic Underutilization

  When diverse workloads leave your GPU half-idle, the fix isn't more hardware — it's smarter partitioning and knowing exactly what to push back to the CPU.

• Apr 9, 2026 arxiv The Self-Healing Inference Server: LLM Serving Systems That Rewrite Themselves

  Autopoiesis proposes an LLM serving architecture that autonomously evolves its own scheduling and resource policies in response to runtime dynamics — no human operator required.

• Apr 6, 2026 arxiv The Hidden Redundancy Killing Multi-Agent LLM Throughput (And How to Fix It)

  When every agent in a multi-agent system receives the same shared context, they're all silently recomputing the same KV cache — TokenDance collapses this redundancy at the serving layer to scale multi-agent inference without wasting memory bandwidth.

• Apr 5, 2026 arxiv Teaching LLMs to Read Assembly: Idiomatic Decompilation for Modern Languages

  What if instead of reconstructing C from machine code, you could get clean, readable Dart back from x86-64 binaries — using a small, specialized LLM?

• Apr 4, 2026 arxiv Asymmetric Draft Trees: A Training-Free Trick That Makes Speculative Decoding Smarter

  Instead of treating speculative decoding trees as symmetric structures, Goose dynamically shapes them based on per-position acceptance probabilities — squeezing out more accepted tokens per forward pass without touching model weights.

• Apr 1, 2026 arxiv Cheating Double Precision: How INT8 Emulation Unlocks GPU Speedups for HPC Workloads

  Researchers used INT8-based matrix multiplication emulation to accelerate FP64 scientific computing on GPUs — turning the AI hardware boom into a win for traditional high-precision HPC simulations.

• Mar 31, 2026 arxiv Hierarchical Indexing Makes Sparse Attention Actually Fast

  Instead of scoring every historical token for sparse attention, HISA builds a multi-level index over the KV cache so you can skip irrelevant tokens entirely — turning an O(n) scan into something much faster without sacrificing selection quality.

• Mar 30, 2026 arxiv Generation IS Compression: Rethinking Video Codecs as Generative Latents

  What if you stopped compressing video frames and instead transmitted a seed for a generative model to hallucinate them back — turning inference itself into the decompression step?

• Mar 28, 2026 arxiv Your 'Private' Local AI Has a Side Channel Built Into Its Architecture

  On-device VLMs promise privacy, but the clever tiling trick that makes them handle high-res images efficiently turns out to be a perfect side-channel that leaks what you fed it — without ever touching the model weights.

• Mar 27, 2026 arxiv Your RAG Knowledge Base Shouldn't Be Read-Only

  Instead of assembling a knowledge base once and calling it done, this paper argues KB construction should be a continuous training loop — synthesizing evidence across fragmented documents and writing enriched facts back in, making retrieval dramatically smarter over time.

• Mar 26, 2026 arxiv Plasma at Exascale: Rethinking Particle-in-Cell Simulations for Multi-GPU Systems

  How do you simulate billions of plasma particles across heterogeneous GPU clusters without drowning in data movement and sync overhead? This paper shows how.

• Mar 25, 2026 arxiv AI That Tunes Your OS: Automated Kernel Specialization for Maximum Application Performance

  Instead of hand-tuning OS parameters for years, Wayfinder automatically searches the vast configuration space of modern operating systems to find the optimal setup for your specific workload.

• Mar 24, 2026 arxiv Kill the Overthinking Loop: Real-Time Inference Intervention for Reasoning Models

  ROM streams reasoning model output, detects when the answer has already converged, and cuts generation early — reducing latency and compute cost without touching the model weights.

• Mar 22, 2026 arxiv Synchronized Time is a Lie: The Hidden Chaos Beneath Your Distributed System's Clocks

  A provocative examination of how daylight saving time, leap seconds, and GPS time corrections reveal that the synchronized time your distributed systems rely on is a carefully maintained fiction with real failure modes.

• Mar 21, 2026 arxiv Are Your GPU Kernels Actually Fast? Benchmarking Against the Laws of Physics

  SOL-ExecBench reframes GPU kernel optimization by measuring not speedup over a baseline, but closeness to the hardware's theoretical ceiling — revealing that most 'optimized' kernels have far more headroom than we thought.

• Mar 20, 2026 arxiv Speculative Tool Execution: Stealing Time from the LLM-Tool Serial Loop

  Just like CPUs speculatively execute future instructions before branching, this framework predicts which tools an LLM agent will call next and fires them in parallel while the model is still thinking — slashing the latency of agentic pipelines without changing the model at all.

• Mar 19, 2026 arxiv Your SSDs Are the New DRAM: Co-Activation-Aware KV Cache Offloading Across Multiple Drives

  Instead of naively spilling KV cache to SSDs when DRAM fills up, Swarm tracks which cache entries are accessed together at inference time and co-locates them across drives — turning NVMe SSDs into a viable, intelligent memory tier for persistent LLM serving.

• Mar 18, 2026 arxiv Breaking the Sequential Curse: Parallel Newton Methods for Dynamical Systems

  A unified mathematical framework showing that recurrent computations and MCMC chains don't have to run step-by-step — turning sequential time into a parallelizable optimization problem that GPUs can actually exploit.

• Mar 17, 2026 arxiv Split to Win: Why Hybrid LLM Inference Needs Disaggregated Prefill and Decode Stacks

  Hybrid Mamba-Transformer models have a dirty secret — their prefill and decode phases are so computationally unlike each other that running them on the same infrastructure is quietly destroying your throughput.

• Mar 16, 2026 arxiv Free Normalization: Piggybacking Layer Norms onto Matrix Multiplication at No Extra Cost

  By reusing the block-scale exponents that MXFP accelerators already compute during matmul, MXNorm eliminates tensor normalization overhead entirely — turning a separate bottleneck into a zero-cost side effect.

• Mar 15, 2026 arxiv Cornserve: Rethinking Distributed Inference When Every Request Takes a Different Path

  When an AI model can take text in and spit audio out, or video in and text out, the entire serving stack breaks — Cornserve is the first distributed system built from scratch for any-to-any multimodal inference.

• Mar 14, 2026 arxiv Training-Free LLM Inference Acceleration by Exploiting Attention Stability Within Sentences

  By observing that dominant attention patterns freeze mid-sentence, this paper halves decoding work without touching model weights — a pure systems optimization hiding in plain sight.

• Mar 13, 2026 arxiv Stack-Spanning Attacks: How Compound AI Systems Create New Hardware-Software Attack Surfaces

  When you chain LLMs, vector databases, and software tools on shared hardware, you're not just adding risks — you're composing them in ways that amplify threats across the entire software-hardware stack.

• Mar 12, 2026 arxiv Your LLM Serving Optimization Is Also a Spy: The Prefix Caching Side-Channel

  When multiple tenants share an LLM inference server with prefix caching enabled, an attacker can reconstruct other users' prompts by measuring response latency — a cache hit is measurably faster, turning a throughput win into an information leak.

• Mar 11, 2026 arxiv Your IAM Revocation Lag Is a Memory Consistency Problem

  A formal proof that authorization revocation in agentic systems is structurally equivalent to memory consistency models — meaning distributed systems theory gives us the vocabulary to reason about how many unauthorized API calls leak through before access is actually cut off.

• Mar 10, 2026 arxiv Your LLM Is Already an Embedding Model: Kill the Second Model in Your RAG Pipeline

  Why run a separate retriever when the LLM's internal activations already encode richer contextual representations than any standalone embedding model you'd bolt on?

• Mar 9, 2026 arxiv The Optimizer's Paradox: When JIT Compilers Make Your Code Slower

  JIT compilers are supposed to make code faster — so what happens when their own optimizations are the bug?

• Mar 8, 2026 arxiv Particle Filtering for Language Models: Ensembling LLMs with Sequential Monte Carlo

  Instead of naively averaging model outputs, this paper borrows Sequential Monte Carlo from Bayesian statistics to combine multiple LLMs and prompting strategies — treating generation as a particle filtering problem where better hypotheses survive.

• Mar 7, 2026 arxiv Beyond MPI: Asynchronous Many-Task Runtimes Take On Radiation Hydrodynamics

  A head-to-head comparison of MPI vs. async many-task runtimes for large-scale radiation hydrodynamics simulations reveals whether the HPC world's sacred cow is actually the best tool for the job.

• Mar 6, 2026 arxiv The Hidden Geometry of Broken Attention: Spikes, Sinks, and What They Cost You

  A systematic anatomical study of why certain transformer tokens develop extreme activation outliers and become attention black holes — and what that means for every inference optimization technique built on top of them.

• Mar 5, 2026 arxiv The GPU Workload That Rewrites Molecular Physics: Profiling ML Force Fields

  As neural network potentials replace classical force fields in molecular dynamics, they're spawning a new category of GPU workload — and this paper is the first to systematically profile what that means for hardware utilization, memory bandwidth, and compute patterns.

• Mar 4, 2026 arxiv Stream Functions: A New Abstraction Where Serverless Meets Stateful Stream Processing

  When your workload is too stateful for serverless and too short-lived for Kafka, neither paradigm serves you well — this paper proposes 'stream functions' to fill the gap.

• Mar 3, 2026 arxiv The Database Engine That Writes Itself: Synthesizing Query Processing Instead of Engineering It

  What if instead of spending decades hand-engineering a query optimizer, you just synthesized a custom query processor at runtime — and it still performed competitively?

• Mar 2, 2026 arxiv Teaching AI to Write CUDA Kernels That Beat the Compiler

  Researchers built an LLM agent trained via large-scale RL to generate optimized CUDA kernels, closing the gap with compiler-based systems like torch.compile that LLMs have historically been unable to challenge.

• Mar 1, 2026 arxiv LLMs Can Learn to Hide: The Steganography Problem in AI Monitoring

  If a misaligned model can encode covert messages in its outputs that evade detection, every monitoring system we build is potentially compromised — and this paper proves the threat is real and formalizable.

• Feb 28, 2026 arxiv LLMs as Storage Whisperers: Auto-Tuning Parallel File Systems with Autonomous Reasoning

  STELLAR uses an LLM agent to autonomously navigate the nightmarish configuration space of large-scale parallel file systems, eliminating the need for the rare human experts who currently gatekeep HPC I/O performance.

• Feb 27, 2026 arxiv The Parallel Decoding Illusion: Why Diffusion LLMs Keep Reinventing Autoregression

  Diffusion language models promise to break autoregression's sequential bottleneck, but something keeps pulling them back — this paper explains the structural reason why truly parallel decoding remains elusive and what it would actually take to escape that gravity.

• Feb 26, 2026 arxiv The I/O Expert in a Box: LLMs That Diagnose HPC Storage Bottlenecks

  When your supercomputer job crawls because of I/O contention, IOAgent reads your traces and tells you exactly why — no storage systems PhD required.