LLMs as Storage Whisperers: Auto-Tuning Parallel File Systems with Autonomous Reasoning

The Hidden Bottleneck in Scientific Computing

Most HPC performance conversations focus on CPUs, GPUs, and network interconnects. But in data-intensive workloads — genomics pipelines, climate simulations, particle physics experiments — the real bottleneck is often I/O. A single misconfigured stripe count on a Lustre file system can cause an application that should max out bandwidth to crawl at a fraction of its potential throughput. The problem isn’t exotic: it’s that tuning parallel file systems is genuinely hard, deeply contextual, and historically requires years of specialist knowledge that most domain scientists simply don’t have.

STELLAR (arxiv.org/abs/2602.23220v1) takes a direct swing at this problem by replacing the human expert with an LLM-powered autonomous reasoning loop.

Why Parallel File System Tuning Is a Nightmare

Systems like Lustre, GPFS, and BeeGFS expose hundreds of tunable parameters. Stripe width, stripe count, OST (Object Storage Target) layout, read-ahead size, lock contention settings, client cache behavior — each interacts with the others in non-linear ways, and the optimal configuration is tightly coupled to the access pattern of the specific application running at that moment.

Traditional autotuners attempt to navigate this space with Bayesian optimization, genetic algorithms, or random search. These approaches have a structural weakness: they treat the parameter space as a black box, relying purely on observed performance to guide search. With hundreds of parameters and expensive benchmark runs, convergence is slow and sample-inefficient. You might burn dozens of runs before approaching anything near-optimal.

What a senior storage engineer does differently is reason about the problem. They look at an application’s access pattern — sequential vs. random, read-heavy vs. write-heavy, large contiguous blocks vs. small scattered writes — and immediately eliminate whole regions of the configuration space based on principles. They have a mental model of why certain settings help or hurt.

What STELLAR Actually Does

STELLAR encodes that reasoning capability by using an LLM not as a generator of configurations, but as an agent that reasons about the problem domain before proposing a configuration. The system feeds the LLM context about the target workload’s I/O characteristics and the file system’s current state, then asks it to reason through which parameters are most likely to matter and why.

The key architectural insight is autonomous iteration. Rather than a single-shot prompt (“give me a good Lustre config”), STELLAR runs a closed feedback loop: propose a configuration, measure performance, reason about the results, update beliefs, propose again. According to the paper’s evaluation results, this loop converges — it almost always selects a near-optimal parameter configuration within the first five attempts.

Five attempts is a significant number in this context. A single benchmarking run on an HPC system can take minutes to hours depending on the workload. Traditional black-box optimizers might require 50–200 samples to converge on the same result. Cutting that by an order of magnitude has real operational value.

The “Pre-Trained on Expertise” Advantage

What makes LLMs particularly suited for this class of problem is that they arrive pre-loaded with domain knowledge. Lustre tuning guides, HPC system administration documentation, and performance analysis papers are well-represented in training data. The model doesn’t need to learn from scratch that larger stripe sizes generally benefit sequential bandwidth-heavy workloads — it already knows this and can apply it immediately as a prior.

This positions STELLAR differently from pure ML-based autotuners. It’s not learning a mapping from configuration space to performance space via sample-intensive exploration. It’s doing something closer to what an expert does: applying structured domain reasoning to prune the search space dramatically, then using observed performance feedback to refine within that narrowed space.

Practical Implications for HPC Infrastructure Teams

The implications for storage administrators are significant in a few directions.

First, it lowers the expertise barrier. Domain scientists running workloads on institutional clusters typically can’t adjust file system parameters at all — that’s gated behind sysadmin access and institutional process. A system like STELLAR could operate as a service layer: the scientist describes their workload, the tuner proposes an optimal per-job configuration, and the system applies it automatically.

Second, it changes the economics of performance tuning. The current model treats storage optimization as a labor-intensive consulting engagement. STELLAR suggests a future where that expertise is deployed programmatically, at scale, across heterogeneous workloads simultaneously.

Third — and this is worth watching carefully — there’s a question of how the LLM reasoning quality degrades with novel or unusual storage systems. STELLAR’s advantage comes partly from the LLM’s pre-existing knowledge of well-documented systems like Lustre. For proprietary or highly customized storage stacks, that prior knowledge may be weaker, and the system may behave more like a standard autotuner than an expert-augmented one.

What to Watch

STELLAR represents an early but compelling example of LLMs applied not just to code generation or text tasks, but to systems reasoning — where the output is a configuration rather than prose. The pattern is generalizable: database query planner tuning, network configuration optimization, Kubernetes resource scheduling. Any domain where the search space is large, the feedback loop is expensive, and deep documentation of best practices already exists is a candidate for the same approach.

The five-attempt convergence result is the number to remember. If that holds across diverse production workloads, it moves STELLAR from an academic curiosity into a tool with genuine operational value for the institutions running the world’s largest scientific computing infrastructure.