Advanced Computing in the Age of AI | Saturday, December 9, 2023

Yahoo, NASCAR Intrigued by Spectra DS3 Object Storage for Tape 

The media and entertainment business drives $1.5 trillion in revenues each year according to PriceWaterhouseCoopers and is growing at a 5 to 6 percent compound annual growth rate. To put that into perspective for you, that is a little bit larger than the global IT market absent telecommunications and data transmission services. The core players in the market will consume on the order of 30 exabytes of storage media in 2013, and that turns out to be a $6 billion business that is growing at around 12 percent compounded annually, according to Coughlin & Associates, which tracks the media market. Part of the reason the media and entertainment industry is spending more and more is that filmmakers and broadcasters are doing higher resolution filming, and they are doing more of it. The ratio of shooting to finished product is approaching 100 to 1 in reality TV shows, for instance.

The same holds true for sports. During a NASCAR race, there might be two dozen cameras in the racecars and then three dozen filming from different vantage points of the track. You never know where the interesting bit is going to be, so you try to record it all and sort it out later.

NASCAR Productions does the broadcasting for 24 racetracks in the United States and Canada that are part of the NASCAR circuit. The organization has races 40 weeks out of the year for a total of 360 races. And it generates a lot of video that needs to be archived.

Chris Whitmayer, director of broadcast, production, and new media technology at the company, says that NASCAR currently has over 160,000 hours of footage stored on tape cartridges, for a total of around 7 PB. Each month, another 1,200 hours of footage are added to the archive. It would be very costly to put all of this data on spinning disk. And so, like other media companies, NASCAR relies on tape.

NASCAR had an eight-frame Spectra T950 tape library with LTO4 drives housing most of its data in Charlotte, North Carolina. There is a second backup facility with a replica archive at an undisclosed location in Kansas.

To make room for its digital archive to expand, NASCAR recently moved to an eight-frame T-Finity tape library, which can grow by a factor of five to have 40 frames and hold around 125 PB of uncompressed data. Whitmayer tells EnterpriseTech that the plan is to move to LTO6 drives and to gradually convert the archive of LTO4 drives over to the denser format. An LTO4 cartridge can hold 800 GB of uncompressed data, while an LTO6 drive can do 2.5 TB uncompressed. So in the long run, NASCAR will be able to cram a lot more data in the same space. And it is going to need to with recording being done in ever-higher resolution, thus requiring more capacity. More importantly, LTO6 media is a few pennies per gigabyte cheaper than LTO4 media, so over the long haul, this will save the organization money.

Because the editors who make footage available from NASCAR races for various media can't wait to restore that data from the tape archive at the full resolution, a proxy copy of the entire archive is stored in an H.264 codec format. The represents about 1 PB of the 7 PB in total storage in the tape library. This proxy and the full archive are wrapped in metadata – over 8 million data points describe the archive – to make it searchable.

Editors bring back a snippet they want out of the proxy archive, a process called scrubbing in the broadcasting lingo. A two minute snippet encoded in H.264 may only be 2 GB in size where it is on the order of 2 TB in a 4K high-def format that is increasingly used for filming today. (4K is the resolution of four HD screens all in one image.) Once editors locate the snippet of footage they want, if they need a higher resolution version, the NASCAR media asset management system knows how to locate that section of the file on a specific tape and only send that piece back. This footage may have to be transcoded for other formats (for television, tablets, smartphones, and various other devices with different screen aspect ratios), and that requires a lot of I/O. So NASCAR has a 4 TB flash-based storage array to boost transcoding speeds. It also has a fast disk array to hold the prior 10 to 14 days of footage and a nearline disk array with slower performance that stores the prior three to six months of racing footage. This array may be slower, but it is faster than pulling from the tape archive.

Like many organizations, NASCAR is cryptic about how it may use the new DS3 specification and the BlackPearl appliance; the company's techies are not allowed to talk about products before they are in production and a lot of the time they aren't allowed to say anything even after they are. The most that Whitmayer would say at the Spectra conference about BlackPearl and its tape-headed extended S3 protocol is that NASCAR would definitely take look at it.

And then he quickly added: "When there is a good idea, people will find it and follow it."

This is particularly true in a world where, as Spectra CEO Nathan Thompson explained, hierarchical storage management software licenses scale linearly with the size of the disk and tape archives, as is the case in the media and entertainment industry. Anything that might make storing and retrieving footage easier is something that media companies are going to take a hard look at, perhaps to the chagrin of those who make a tidy living selling archiving and retrieval software that interfaces between disk and tape systems. And if it also saves them money on software licenses, that is a big bonus.

Another interesting possibility for the use of the DS3 protocol and libraries front-ended by the BlackPearl appliance in the media and entertainment industry is to link web caches at content delivery networks directly into the tape libraries.

Today, media companies have to try to guess what viewers will want to look at and dump their footage archives into the content delivery networks of Akamai, Amazon Web Services, and others who provide such services for Web applications. Filling the CDN could now be done directly out of the archive using applications written using REST commands. The first user, no doubt, would experience a bit of a delay, as a multi-gigabyte file is pulled into cache. But from the perspective of media companies, this is better than paying Akamai or AWS a whole lot of money.