Advanced Computing in the Age of AI | Wednesday, July 24, 2024

Facebook Crafts Cold Storage Library From Blu-ray Discs 

Social media giant Facebook has been pushing the limits of cold storage with disk technology, and as it hinted it might be doing a year ago, the company has taken cold storage to even lower temperatures with a new robotic library it designed based on Blu-ray discs.

The setup can cram an amazing 1 PB of data into a single 48U rack (seven feet high), and there are plans to jack that up by a factor of five, explained Jay Parikh, vice president of infrastructure at Facebook. It will be interesting to see what Spectra Logic, IBM, and Oracle – all of whom have made substantial investments in petabyte-scale tape libraries – will think of this development at the Social Network. There are a number of Blu-ray archiving solutions out there, but none of them at this scale.

The Open Compute Summit is happening in San Jose this week, and Parikh, vice president of infrastructure at Facebook, walked down memory lane, showing the old Facebook datacenters in co-location facilities and the napkin on which the first plans for homemade servers and datacenters were sketched out five years ago when the company decided to not only build its own infrastructure, including datacenters, but also to open source its designs. The Open Compute Project was formally announced in July 2011, and has since that time gathered momentum with a number of cloud suppliers and big banks, all of whom want cheaper, more efficient, and precisely tailored server, storage, and networking infrastructure.

facebook-dvd-cold-storageThere is a possibility that a new cold storage vendor could emerge to compete with Spectra Logic, IBM, and Oracle based on this design. Facebook, unlike Amazon Web Services, seems to be allergic to tape drives and libraries, and has expressed a similar interest of creating massive archiving systems based on the cheapest flash in the market – the kind you see in thumb drives. A combination of flash and Blu-ray libraries as well as the Open Vault cold storage arrays that Facebook recently put into production could be commercialized by Open Compute resellers. Parikh said that Facebook already has 30 PB of data, mostly old photos that are rarely if ever accessed, stored in the cold storage facility in its Prineville, Oregon datacenter, and that it will be ramping up to around 150 PB in the next month or two. The two cold storage facilities are able to store up to 3 EB – that's exabytes – of data, which should give Facebook plenty of room for capacity expansion. But, the company is always looking for the cheaper, more efficient option, which is necessary if it is to grow its user base from the current 1.2 billion to reach the other 5 billion who don't have access to the Internet but will at some point.

Each rack of Open Vault cold storage can house up to 2 PB of disk capacity, and it consumes only 2 kilowatts of power. This is accomplished by turning disk drives off except when a particular file is being accessed, and by spreading a file over multiple disk drives across multiple arrays to get parallel access to the bits in the old picture you haven't looked at in five years.

At the Open Compute Summit, Parikh said that the as-yet-unnamed Blu-ray library would burn 80 percent less energy than the Open Vault cold storage, and would be 50 percent less expensive than that disk-based setup as well.

The machine has trays that house 10,368 discs, stacked in magazine carriers on shelves much as is done for tape cartridges in tape libraries. The library adheres to Open Rack physical size standards, so it can roll right into any datacenter that uses Open Rack, and it also uses Open Rack's power shelves to juice up the robot arm and 16 disc burners in the library. It has 24 disc magazines, a dozen per side. Each magazine has 36 cartridges, and each cartridge has a dozen 100 GB Blu-ray discs using the BDXL format. The discs that Facebook is testing are certified for up to 50 years of operation, and some Blu-ray media is certified for up to 1,000 years of storage. (Facebook will probably not splurge on these, though.)

Facebook did not say when this prototype Blu-ray library would be put into production, when the design would be opened up and contributed to the Open Compute Project, or how it would be able to quintuple capacity. The next stop on the Blu-ray roadmap seems to be a mere doubling to 200 GB or so per disc, so Facebook has something else up its sleeve.