OS Support for the Multi-Tenant, Real-Time Edge
Open Access DepositedEmbedded and real-time devices in many domains increasingly depend on network connectivity. The ability to offload computations encourages Cost, Size, Weight, and Power (C-SWaP) optimizations. Coordinating with the cloud, on the other hand, enables the system to gather data from various sensors, thus guiding global collaborations or understanding of the surrounding environment. The promise is significant: Autonomous Vehicles (AVs) coordinating with each other and better understanding the environment, Industry 4.0 factories aggregating data for global optimization, and power or resource-constrained devices leveraging resources on the edge cloud to make machine learning-based inferences. The proliferation of low-latency wireless technologies, such as 5G, coupled with the integration of edge cloud computing, is playing a pivotal role in advancing and reinforcing these trends. Unfortunately, it is challenging for today’s operating system designs to process high-density multi-tenant workloads while maintaining strong isolation with limited resources on the edge. This thesis focuses on providing real-time OS support on the edge cloud, optimizing throughput and latency of requests while still maintaining strong isolation. Given the varied tenants with differing trust levels on the edge cloud, robust isolation is imperative for ensuring secure and independent environments. Our previous work on EdgeOS emphasizes system-wide isolation as fine-grained as per-client, thus isolating tenant and device computations. EdgeOS also provides fast start and re-initialization of such fine-grained isolation, allowing separate chains of computations per client request. Instead of sharing data between computations or services, EdgeOS introduces a software DMA engine that performs copy-based data movement between computations. Therefore, EdgeOS can enforce strong isolation without performance penalties. Beyond EdgeOS, this thesis introduces Edge-RT, an OS infrastructure for the edge designed to meet end-to-end deadlines across chains of computations. Despite the focus on deadline and isolation, Edge-RT maintains high system efficiency. To achieve this, Edge-RT focuses on per-packet deadlines inherited by the computation operating on it. Edge-RT introduces mechanisms to avoid per-packet system overhead while trading bounded impact on predictable scheduling. According to evaluations, compared to Linux and EdgeOS, Edge-RT can maintain high throughput and meet significantly more deadlines, even in bimodal workload scenarios with global utilization above 60%. Beyond Edge-RT’s primary focus on meeting end-to-end request deadlines, it has become evident that enabling low-latency software requires more than just data-plane optimizations for fast request processing. Unfortunately, these efforts alone are insufficient: effective low-latency service also requires advances in the control-plane. However, control-plane operations require strong spatial and temporal isolation between tenant computations, which can introduce significant overhead. This thesis introduces Janus, an OS abstraction for a flexible control plane, which provides strong isolation and managed latency through the use of pervasive kernel-bypass. Janus leverages hardware Memory Protection Key (MPK) to enable low-cost control operations for Protected Procedure Call (PPC) and thread dispatch—two essential building blocks of spatial and temporal isolation. By transparently improving these fundamental control-plane operations, Janus enables efficient and predictable user-defined and customizable system control policies and mechanisms while maintaining strong isolation. We evaluate Janus’s ability to define new control operations, increase the efficiency of an existing RTOS, and support low-latency services in a memcached server. Compared to a Linux-based approach, a specialized latency-sensitive control plane using Janus provides over a 6x improvement in throughput while providing 99th percentile tail latencies almost 3x lower. In a multi-tenant system, tail latency improves by orders of magnitude. By integrating EdgeOS, which focuses on lightweight isolation abstractions, Edge-RT, which emphasizes end-to-end packet scheduling, and Janus, which optimizes operation-plane abstraction, this thesis establishes a robust foundation for real-time, multi-tenant edge infrastructures.
- All rights reserved
Notice to Authors
If you are the author of this work and you have any questions about the information on this page, please use the Contact form to get in touch with us.