A Big Data Processing System Accelerated by Heterogeneous Processors

Abstract/Technology Overview

With the explosive growth of data, companies have to invest a lot of money in hardware and software to extract valuable information from unbounded data. The increasingly complex applications impose another challenge to modern analytical systems to effectively and efficiently process data in real-time. We designed a big data analytics system that incorporates three different types of processors and provides access to many native analytical algorithms and libraries. Users working on large and complex data (video, audio, images, texts and so on) can easily design, program and deploy it at-scale to obtain real-time analysis results.

Technology Features, Specifications and Advantages

Our systems have the following features and advantages.

  • Heterogeneity awareness: our system supports three different processors (CPUs, GPUs, and FPGAs) to provide larger design and optimization space in performance, power consumption and monetary cost;
  • Cross-domain support: with a Native Access Layer (NAL), users can easily access a wide range of native analytical algorithms and libraries, such as Caffe, TensorFlow, and OpenCV;
  • Easy deployment: built on top of the open source distributed realtime computation system Apache Storm, our system can be easily scaled to tens and hundreds of nodes to obtain high processing capability;​​​​
  • Programmability: we provide a set of Java APIs for users to easily utilize the power of large-scale heterogeneous processors.

Potential Applications

Our system has integrated various analytical algorithms and libraries. Thus, it can be easily applied to those streaming and Machine Learning (ML) / Deep Learning (DL) related applications. For example:

  • Video analytics
  • Face recognition
  • Traffic monitoring and smart planning
  • Terrorism prediction and alert system
  • Online store recommendation system
  • Any other computation or memory intensive analytical applications

Customer Benefit

Corporate users with a lot of data to analyze may significantly reduce large-scale infrastructure and manpower costs, to maintain a large team of experts with cross-domain knowledge.

Technology Owner

Jiong He


ADSC (Illinois at Singapore Pte Ltd)

Technology Category
  • Big Data, Data Analytics, Data Mining & Data Visualisation
Technology Status
  • Available for Licensing
Technology Readiness Level
  • TRL 4

Large-scale data analytics, heterogeneous computing