Advanced Computing in the Age of AI | Thursday, June 13, 2024

Micron Releases Open Source Storage Engine 

Growing enterprise demand for object-based storage along with the proliferation of all-flash memory infrastructure has prompted one hardware vendor to release an open source version of its memory-class storage technology.

Micron Technology Inc. promotes its fast key-value storage engine as offering lower latency along with the ability of tweak accompanying software-defined platforms such as open-source Ceph distributed storage.

Micron (NASDAQ: MU) claims to be the first memory chip maker to offer an open source version of what it calls a heterogeneous-memory storage engine (HSE) designed for solid-state drives, memory-based storage and, ultimately, applications requiring persistent memory. The “heterogeneous” in HSE refers to its ability to simultaneously use different media, including SSDs, flash memory and emerging 3D XPoint technologies.

“Legacy storage engines born in the era of hard disk drives have historically failed to architecturally provide for the increased performance and reduced latency of next-generation nonvolatile media,” the company noted in releasing HSE on Monday (April 27).

The goal is to “build a software stack that accelerates applications running in today’s flash-based storage environments as well as storage-class, memory-based infrastructure of the future,” said Derek Dicker, general manager of Micron’s storage business unit.

Along with reducing latency among computing, memory and storage resources when crunching large data sets, the memory maker said its approach would boost SSD endurance by a as much as a factor of seven.

Micron also singled out database and software-defined storage partners along with potential HSE applications ranging from object storage to HPC use cases. Specifically, it said the open source storage engine has been integrated with MongoDB’s (NASDAQ: MDB) NoSQL database, and could also be combined with other object storage applications.

HSE also meshes with open source storage platforms like Red Hat’s version of Ceph distributed storage as well as Scality’s RING cloud-based storage. That combination would support the growing number of cloud-native applications running on container platforms like Red Hat OpenShift.

Micron unveiled its X100 NVMe-based drive last fall, positioning it as delivering large data sets for analytics, streaming and machine learning workloads. This week, the company said the addition of an X100 SSD with four Micron 5210 quad-level cell NAND SSDs yielded a doubling of throughput and a four-fold reduction in read latency.

Micron said HSE is available as an “embeddable” key-value store—a streamlined NoSQL database. The company will maintain the HSE code repository on GitHub.



About the author: George Leopold

George Leopold has written about science and technology for more than 30 years, focusing on electronics and aerospace technology. He previously served as executive editor of Electronic Engineering Times. Leopold is the author of "Calculated Risk: The Supersonic Life and Times of Gus Grissom" (Purdue University Press, 2016).