Tianhao Huang, Xiaozhi Zhu, Mo Niu

An End-to-End Benchmarking Tool for Analyzing the Hardware-Software Implications of Multi-modal DNNs

  • Computer Networks and Communications
  • Hardware and Architecture
  • Software
Save to CiteDrive Show PDF

Abstract-Multi-modal deep neural networks (DNNs) have become increasingly pervasive in many machine learning application domains due to their superior accuracy by fusing various modalities together. However, multi-modal DNNs present many unique characteristics such as multi-stage execution, frequent synchronization and high heterogeneity, which are not well understood in the system and architecture community. In this article, we first present and characterize a set of multi-modal DNN workloads of different sizes from five domains and measure metrics like accuracy to ensure the availability of these applications from the algorithm perspective. We then explore their important hardwaresoftware implications from system and architecture aspects by conducting an in-depth analysis on the unique hardware-software characteristics of multimodal DNNs. We hope that our work can help guide future hardware-software design and optimization for efficient inference of multi-modal DNN applications on both cloud and edge computing platforms.

Need a simple solution for managing your BibTeX entries? Explore CiteDrive!

  • Web-based, modern reference management
  • Collaborate and share with fellow researchers
  • Integration with Overleaf
  • Comprehensive BibTeX/BibLaTeX support
  • Save articles and websites directly from your browser
  • Search for new articles from a database of tens of millions of references
Try out CiteDrive

More from our Archive