Thursday, December 9, 2021 | 1:30pm – 3:00pm PT
Speakers: Yujun Lin and Song Han, MIT

Data-driven, AI-based design space exploration of neural network accelerator and neural network architecture is desirable for specialization and productivity. Previous frameworks focus on sizing the numerical architectural hyper-parameters while neglect searching the PE connectivities and compiler mappings. We push beyond searching only hardware hyper-parameters and propose the Neural Accelerator Architecture Search (NAAS), which fully exploits the hardware design space and compiler mapping strategies at the same time. Thanks to the low search cost, NAAS can be easily integrated with hardware-aware NAS algorithm, achieving the joint searching for neural network architecture, accelerator architecture and compiler mapping. As a data-driven approach, NAAS rivals the human design by 4.4x EDP reduction with 2.7% accuracy improvement on ImageNet under the same computation resource, and offers 1.4x to 3.5x EDP reduction than only sizing the architectural hyper-parameters.

Speaker Bio: Yujun Lin is a 4th year Ph.D. student at MIT, advised by Prof.Song Han. He received B.Eng from Tsinghua University. His research is at the intersection of computer architecture and machine learning, especially software and hardware co-design for deep learning and its applications.

Speaker Bio: Song Han is an assistant professor in MIT’s Department of Electrical Engineering and Computer Science. His research focuses on efficient deep learning computing. He has proposed “deep compression” as a way to reduce neural network size by an order of magnitude, and the hardware implementation “efficient inference engine” that first exploited model compression and weight sparsity in deep learning accelerators. His team’s work on hardware-aware neural architecture search has been integrated by PyTorch and AutoGluon, and received six low-power computer vision contest awards in flagship AI conferences. He has received best paper awards at the International Conference on Learning Representations and Field-Programmable Gate Arrays symposium. He is also a recipient of an NSF Career Award and MIT Tech Review’s 35 Innovators Under 35 award. Many of his pruning, compression, and acceleration techniques have been integrated into commercial artificial intelligence chips. He earned a PhD in electrical engineering from Stanford University.

To appear at the Design Automation Conference 2021 December 5-9, view Session Information and Project Details.