Advanced Computing in the Age of AI | Monday, July 22, 2024

Container Vendors Downplay ‘Memory Pressure’ Concerns 

Leading purveyors of application container infrastructure are downplaying a recent analysis that identifies a resource management feature in the Linux kernel as a potential "performance killer."

LinkedIn (NYSE: LNKD) released an application container analysis last week after conducting several months of internal "pressure testing" of container infrastructure. It found that many container projects running on Docker or rival CoreOS platforms rely on the Linux kernel feature called control groups, or "cgroups," to manage resources like computing and memory.

The analysis found that cgroups "do not totally isolate resources, but rather limit resource usage so that applications running in memory-limited cgroups do not starve other cgroups," concluded Zhenyun Zhuang, a LinkedIn software engineer.

One consequence is "memory pressure," which Zhuang warned could raise issues affecting the performance of applications running in cgroups.

While Docker had no comment on the analysis, CoreOS noted that a Linux group is already working to resolve resource management glitches. "Many of the issues noted in this paper from LinkedIn are well-known issues and are simply complex problems that the Linux memory management community has been attempting to fix over the last few years as the use of cgroups grows," CoreOS CTO Brandon Philips noted in an email. A Linux memory management body has been meeting in recent months to iron out cgroup issues.

Rather than reserving memory as with virtual machines, cgroups impose only an upper limit on memory usage for applications in the control group. Hence, the LinkedIn engineer found that memory is allocated on demand, and applications deployed in cgroups must compete for free memory from the operating system.

Among the implications is that the OS must reclaim memory from the page cache or from "anonymous" memory if it does not have enough free memory to meet the cgroup request. "Memory reclamation by the OS could be a performance killer, affecting the performance of other cgroups," Zhuang warned.

Docker had no comment on the LinkedIn analysis. A user perspective on how Docker containers use cgroups to manage resources is here.

For its part, CoreOS noted that its platform is built around the Kubernetes container orchestrator, which according to Philips provides the opportunity to "right size" cgroups on the fly "based on real usage."

Philips nevertheless praised LinkedIn's experimental approach to wringing out container infrastructure that has identified "tricky problems" in Linux memory management and its interactions with cgroups. "Having early adopters identify, and make recommendations like this will help all organizations as they move along their journey of containerization over the coming years," Philips said.

A possible remedy to the memory pressure issue called Control Group v2 containing performance improvements was released earlier this year. CoreOS said it intends to use cgroups v2, noting that its Linux distribution supports the new version. "However, it will take some time for the tooling around cgroups to move over to v2, and we won’t make it the default until the primary tools have the ability to use it," said Philips.

About the author: George Leopold

George Leopold has written about science and technology for more than 30 years, focusing on electronics and aerospace technology. He previously served as executive editor of Electronic Engineering Times. Leopold is the author of "Calculated Risk: The Supersonic Life and Times of Gus Grissom" (Purdue University Press, 2016).