Methods for Characterization and Analysis
of Voltage Margins in Multicore Processors

Conservative design margins in modern multicore CPU chips aim to guarantee correct execution of the software layers of computing system under various operating conditions, such as worst-case voltage noise (Ldi/dt), and accounting for the inherent variability among different cores of the same CPU chip, among different manufactured chips and among different workloads. However, guard-banding the main operational parameters of CPU chips (voltage, frequency), leads to limited energy efficiency.

In this tutorial, we will present recent methods and studies on design-time voltage-margins characterization and identification in modern multicore CPUs; such methods aim to improve energy efficiency while guaranteeing the correctness of software execution.

• We will discuss key power-delivery-network (PDN) challenges and present an on-chip dedicated circuitry for PDN voltage noise characterization. The presentation will include various analysis conducted with real hardware using this circuitry for characterizing voltage noise caused by Ldi/dt viruses, system-call intensive benchmarks and scan-debug activity.
• We will present the main challenges and how they can be addressed for the characterization and identification of different types of variability of modern multicore CPUs (across cores, across chips and across workloads) and to analyze the system behavior in scaled conditions (what types of malfunctions are observed). Both single-thread and multi-thread workloads will be discussed.
• We will present the main challenges and how they can be addressed for the characterization and identification of different types of variability of modern multicore CPUs (across cores, across chips and across workloads) and to analyze the system behavior in scaled conditions (what types of malfunctions are observed). Both single-thread and multi-thread workloads will be discussed.
• We will present a novel non-intrusive, zero-overhead, cross-platform approach for post-silicon dI/dt voltage noise monitoring based on sensing CPU electromagnetic emanations using an antenna and a spectrum analyzer. The approach is based on the observation that high amplitude electromagnetic emanations are correlated with high voltage noise. We leverage this observation to automatically generate voltage noise (dI/dt) stress tests and measure PDN resonance frequency.
• The tutorial analysis is based on real system measurements using client chips as well as on different multicore server CPU chips mainly based on ARMv8 architecture (such as ARM Cortex-A72 and Cortex-A53 CPUs, AppliedMicro's - now Ampere Computing - multicore X-Gene 2 and X-Gene 3 CPUs). Discussion and comparison among the implementations and also with different architectures (mainly Intel and AMD x86 multicore CPU chips) will also take place.

The purpose of the tutorial is to summarize recent characterization and exploitation findings on multicore CPUs in server machines, emphasize on the potential of energy saving through identification and exploitation of design margins and to discuss our reports and findings to other machines similarly studied in the past.

Available upon request

The target audience of the tutorial includes researchers and practitioners interested in energy efficiency for microprocessors through voltage margins identification and exploitation and the corresponding reliability considerations.

Dimitris Gizopoulos (dgizop@di.uoa.gr) is Professor at the Department of Informatics & Telecommunications of the National & Kapodistrian University of Athens in Greece where he leads the Computer Architecture Laboratory (http://cal.di.uoa.gr/). Gizopoulos’ research focuses on the dependability, the energy-efficiency and the performance of computer architectures built around CPUs, GPUs and other accelerators. Gizopoulos has published more than 170 papers in top-tier conferences and journals (including best paper awards and nominations), has served as Associate Editor for several IEEE Transactions and Magazines (TC, TVLSI, D&T, TSUSC; currently on the Editorial Board of TC, TSUSC, TETC), as Guest Editor in several Transactions Special Issues and as member of several Program, Organizing and Steering Committees of major IEEE and ACM conferences. Gizopoulos is an IEEE Fellow, a Golden Core member of the IEEE Computer Society and a Senior ACM member (http://www.di.uoa.gr/~dgizop). Gizopoulos has served as Tutorials and Education Chair for the IEEE Computer Society TTTC Council, as Editor for the IEEE Design & Test Magazine Tutorial articles department, a member of major IEEE Educational Awards evaluation committees and has presented several conference tutorials including MICRO 2018, ISCA 2018, and MICRO 2017.

Shidhartha Das (Shidhartha.Das@arm.com) is a Senior Principal Engineer with Arm Research, based in Cambridge, UK, where he conducts research in several aspects of circuits and systems for emerging technologies, low power and variation-tolerant circuits, and microarchitectural design. His current research interests include emerging nonvolatile memory technologies, micro-architectural and circuit design for variation measurement and mitigation, on-chip power delivery, and VLSI architectures for digital signal processing accelerators. Dr. Das was a recipient of the multiple best paper awards, including the IEEE/ACM International Symposium on Low-Power Electronic Design Best Paper Award in 2015, the Sophia Antipolis Micro-electronics Conference 2010, the IEEE/ACM International Symposium on Microarchitecture 2003, and the Microprocessor Review Analysts’ Choice Award in Innovation 2007. He is a recipient of the Arm Inventor of the Year Award in 2016, for his contributions to emerging non-volatile memory technologies. His research has been featured in the IEEE Spectrum and has been invited to several top-tier journals and conferences. He serves on the Technical Program Committee at the European Solid-State Circuits Conference, the International Symposium on Low-Power Engineering and Design, and the International On-Line Testing Symposium.

Yiannakis Sazeides (yanos@cs.ucy.ac.cy) is an Associate Professor in the Department of Computer Science at the University of Cyprus. He has worked and contributed towards the development and design of high performance processors with Compaq and Intel. During a HiPEAC sponsored mini-sabbatical at ARM in 2011 he explored the convergence of reliability and security. Recently (2015-2016) he spend a sabbatical at Intel, IDC center in Haifa Israel. He was the leader of the Task Force on Reliability and Availability in FP7 HiPEAC2 Network of Excellence and served in the advisory board of HiPEAC3. He is actively contributing in the organization of many workshops related to fault-tolerance. He was also responsible for the organization of the HiPEAC 2009 conference in Cyprus and the tutorials and workshops chair of ISCA2010. He serves regularly in Program Committees of several international conferences (such as ISCA, MICRO and HPCA). His research interests lie in the area of Computer Architecture with particular emphasis in fault-tolerance, data-center modelling, memory hierarchy and dynamic program behavior. He has over 70 publications, including three best paper awards (MICRO 1997, DATE 2016, CAL 2017) three granted and three patent applications.

George Papadimitriou (georgepap@di.uoa.gr) is a PhD candidate at the Department of Informatics & Telecommunications of the University of Athens in Greece, since March 2014. He graduated from the Department of Electronic Computer Systems of the Piraeus University of Applied Sciences as valedictorian, where he received his BSc degree in Computer Engineering in 2011. He received his MSc with honors in Computer Systems Technology from the Department of Informatics & Telecommunications of the University of Athens. His research focuses on dependability and energy-efficient computer architectures, functional correctness of hardware designs and design validation of microprocessors and microprocessor-based systems. He is particularly interested in investigating different methods on silicon validation of microprocessor architectures to ensure efficient bug detection. His current research focuses primarily on energy-efficient architectures and on investigating methods for reducing energy in the microprocessors. He has presented several conference tutorials including MICRO 2018, and ISCA 2018.

Athanasios Chatzidimitriou (achatz@di.uoa.gr) is a PhD candidate at the Department of Informatics & Telecommunications of the University of Athens in Greece. He holds a BSc in Computer Engineering from the Technological Educational Institute of Athens and an MSc in Advanced Information Systems - Technology of Embedded Computing Systems from University of Piraeus. His research focuses on methods and tools for microarchitecture level reliability assessment as well as energy-efficient computing, where he has published more than 25 papers in international conferences and journals. In the recent years he has co-organised/participated in Tutorials around these research areas (MICRO 2017, MICRO 2018, and ISCA 2018). He is also the lead developer of GeFIN fault injection framework.

Zacharias Hadjilambrou (zhadji01@cs.ucy.ac.cy) is pursuing a Ph.D. degree in Computer Science at the University of Cyprus. He has received M.Sc. (2015) and B.Sc. (2013) degrees in Computer Science from the same university. He has worked at ARM Research multiple times as an intern. His current research interests include voltage-noise, workload characterization of online-search applications and datacenters.