Energy Efficient Embedded Video Processing Systems

High-density, nanoscale fabrication technologies have enabled the chip designers to assemble billions of transistors on a single die. This has in turn provided the capability to realize high-complexity systems like video systems, which have universally penetrated into communication, security, education, entertainment, navigation, and robotics domains. The advancement in the fabrication technology has also driven the user expectations of the next-generation video processing systems. A prime example is high-resolution video capture and playback. Therefore, video applications like the latest video encoders [1] now target high-resolution video content compression beyond full-HD (like 4K ultrahigh definition, 3840 × 2160 pixels) at high frame rates (>120 frames per second). On the contrary, emergence and evolution of next-generation video systems (with increasing throughput and connectivity requirements and adaptability to application, battery life, etc.) require high processing capabilities and efficient utilization of the resources, which might be prohibitive on a resource- and power-constraint hardware platform. Though high-end systems can meet the throughput requirements, efficient and long-term deployment of such applications on small, battery-driven, autonomous systems is challenging, due to the high computational and power requirements while addressing the throughput constraints. Coupled with the high-throughput demands, a video system must be capable of responding in real time to changes in the workload of the application. Further, modern nano-era fabrication technologies have their own associated challenges (like power wall [2] and reliability [3]) which must be accounted for forging energy-efficient multimedia systems. This suggests that new design methodologies for next-generation video systems are needed, to address the abovementioned challenges on modern systems. This book presents some of these methodologies, both at the software and hardware layers of the system.

This chapter discusses the basics of video processing in general, while specifically targeting the video coding applications. General video system design and its memory access patterns and resource utilization are deliberated. Fundamentals of HEVC and H.264/AVC video encoding are followed by their associated challenges when designing computationally efficient video processing systems. Modern technological challenges that arise in deploying video systems are also presented in this chapter. Afterwards, the state-of-the-art techniques to meet these design challenges are discussed, with details targeting video processing system’s software and hardware layers.

This chapter provides an overview of designing a video system to meet the challenges outlined in Chap. 2. Details are given about the architectural aspects, and the complexity and power control knobs of the system. By examining these knobs, motivational analysis is carried out which forms the foundation of the algorithmic- and architectural-design decisions presented in this book.

This chapter provides details about the runtime supervision of the video processing system at the software layer. The main responsibilities addressed in this layer are to allocate compute nodes, realize power efficiency and budget power to the video system. In order to parallelize the execution of a video application, resources are allocated to the application at runtime, by considering the user demands, and application and hardware attributes of the system. Further, the workload is distributed among the compute nodes in a way that throughput-per-watt is increased. Video application properties are also exploited at runtime, and these properties are used to adjust the configuration knobs, which leverage power/complexity with the output video quality. Moreover, resource and power allocation to multiple applications running concurrently on multi/many-core homogeneous and heterogeneous systems are also discussed.

The techniques based in the software layer for computation- and power-efficient video processing system given in Chap. 4 do not necessitate any custom hardware. However, custom hardware architectures for video processing systems are in wide use because they produce higher throughput and have higher complexity and power reduction potential compared to the software-only solutions. This chapter outlines some of the hardware architectural enhancements and custom accelerators for highly efficient video processing systems. Efficient I/O and internode communications for video processing system are discussed. Hardware architectures of the complete system and accelerators are also given, specifically pertaining to H.264/AVC and HEVC encoders. Furthermore, the hardware accelerator allocation or workload administration (whereby the accelerator provides its services to multiple nodes) is also discussed, which can be useful in shared hardware accelerator paradigms. Targeting the memory subsystem, power-efficient hybrid memory architectures and SRAM aging mitigation are also presented.

The experimental evaluation of the techniques presented in Chaps. 4 and 5 are discussed in this chapter. In the previous chapters, we have already included the sensitivity analysis of the individual parts within the algorithmic and architectural details, whenever deemed useful. Here, the main results and comparison with other state-of-the-art techniques are presented, to provide an overview to the reader about gains and drawbacks of these techniques. Major emphasis of the results is video encoding, specifically H.264/AVC and HEVC video encoders. It must also be noted that these encoders have much more modules and higher complexity than many benchmark applications available in Parsec [1], MediaBench [2], Cosmic [3], and MiBench [4] benchmark suites.

Targeting multimedia systems under high throughput, resource and power constraints, this book discusses efficient software-/application-level techniques and hardware-/architectural-level designs for the multimedia (specifically video) systems. Mainly, the aim of the techniques discussed in this book is to maximize the throughput-per-watt metric of the system while considering some modern design challenges and methodologies. The challenges addressed in this book include parallelization of multimedia applications on possibly heterogeneous systems, load balancing on many-core and customized nodes, resource (number of cores and power) budgeting, and efficient design of the multimedia system’s memory architecture. In a broader perspective, these problems can collectively represent the power wall or dark silicon challenge for the next-generation video processing systems.