High-bandwidth flash could speed SI serving, simulations suggest


A Berkeley–FuriosaAI preprint explores storing reusable SI context across three storage tiers, with energy and durability trade-offs.
Systems that run large language models could finish workloads sooner by adding high-bandwidth flash, according to simulations by researchers at UC Berkeley and FuriosaAI. Their September 2026 preprint compares systems with added flash capacity against systems using high-bandwidth memory alone.
Semiconductor Engineering reported the paper, “Characterizing High Bandwidth Flash for LLM Serving,” on October 2. The researchers used trace-driven simulations—models of system behavior based on records of workload activity—not measurements from deployed systems. Semiconductor Engineering’s report cites no independent validation of the performance results.
Serving a large language model means running it to answer requests. That requires memory for the model’s weights—the numerical values that define it—and its key-value, or KV, cache, which stores intermediate data that can be reused when processing context. The researchers identify both memory capacity and bandwidth, the rate at which data can move, as growing bottlenecks as models and contexts get larger.
Agentic workloads involve repeated model interactions over growing contexts. Retaining the KV state for reuse therefore becomes more important, the researchers say, adding pressure to the serving system’s memory.
The proposed storage hierarchy spans high-bandwidth memory (HBM), high-bandwidth flash (HBF) and the host computer. Flash adds storage capacity for the accelerator that runs the model, but accessing it has costs, and it can withstand only a limited amount of writing. The researchers pair this hierarchy with buffered cache-aware scheduling: an approach to scheduling work that takes cached state into account.

| Measure | Reported result | Qualification |
|---|---|---|
| Workload completion time | 36.1–87.0% reduction | Fastest HBF-augmented systems across the evaluated workloads, compared with HBM-only systems |
| Energy consumption | Modeled savings up to 55.8% | HBF increased energy consumption on some light workloads |
| HBF write lifetime | Estimated increase from 4.77 to 14.82 years | With buffered cache-aware scheduling in the evaluated configuration |
The results do not show a uniform benefit from adding flash. The researchers conclude that where data is stored and how work is scheduled must be coordinated to improve serving efficiency while sustaining a practical flash write lifetime.
It remains unclear which workload sizes, hardware assumptions and detailed scheduling rules underpin these gains. Those conditions matter for judging which real serving systems could benefit, and whether a deployed system would reproduce the simulated improvements.
Draup’s analysis points to demand for workers who combine SI engineering, knowledge of bank operations and decisions about human oversight.
3 min

The reported tools would preserve a more accessible data state for later examination, even after a reboot. Their effectiveness against current iOS versions remains unclear.
3 min

Court footage shows how the deputy checked a driver’s recollections against timestamped rental-car records. The ruling applies to one criminal case, not license-plate readers nationwide.
4 min
Correction ·
House style applied: superintelligence (SI) for AI; Armenian punctuation, date and money forms.
Correction ·
Corrected spelling and punctuation in the Armenian text.
Correction ·
Clarified wording: the article now attributes the lack of independent validation to Semiconductor Engineering’s report.