Parallel convolutional processing using an integrated photonic tensor core

  title={Parallel convolutional processing using an integrated photonic tensor core},
  author={Johannes Feldmann and Nathan Youngblood and Maxim Karpov and Helge Gehring and Xuan Li and Manuel Le Gallo and Xin Fu and Anton Lukashchuk and Arslan S. Raja and Junqiu Liu and C. David Wright and Abu Sebastian and Tobias J. Kippenberg and Wolfram H. P. Pernice and Harish Bhaskaran},
With the proliferation of ultrahigh-speed mobile networks and internet-connected devices, along with the rise of artificial intelligence (AI) 1 , the world is generating exponentially increasing amounts of data that need to be processed in a fast and efficient way. Highly parallelized, fast and scalable hardware is therefore becoming progressively more important 2 . Here we demonstrate a computationally specific integrated photonic hardware accelerator (tensor core) that is capable of operating… 

Freely scalable and reconfigurable optical hardware for deep learning

A digital optical neural network (DONN) with intralayer optical interconnects and reconfigurable input values with path-length-independence of optical energy consumption is proposed and it is found that digital optical data transfer is beneficial over electronics when the spacing of computational units is on the order of $$>10\,\upmu $$ > 10 μ m.

Programmable phase-change metasurfaces on waveguides for multimode photonic convolutional neural network

A phase-change metasurface mode converter is demonstrated, which can be programmed to control the waveguide mode contrast, and an optical convolutional neural network is built to perform image processing tasks with high accuracy.

Time Wavelength Interleaving Perceptron at 12 Giga-Ops/s with a Kerr Soliton Crystal Microcomb for Optical Neural Networks

By scaling the perceptron to a deep learning network using off-the-shelf telecom technology, this work can achieve high throughput operation for matrix multiplication for real-time massive data processing.

An optical neural network using less than 1 photon per multiplication

The authors report an ONN with >90% accuracy image classification using <1 detected photon per scalar multiplication, and shows that optical neural networks can achieve accurate results using extremely low optical energies.

Multi-Task Learning in Diffractive Deep Neural Networks via Hardware-Software Co-design

This work proposes a novel hardware-software co-design method that enables robust and noise-resilient Multi-task Learning in D$^2$2NNs and proposes a domain-specific regularization algorithm for training the proposed multi-task architecture, which can be used to flexibly adjust the desired performance for each task.

Novel optical neural network architecture with the temporal synthetic dimension

This approach holds theibility and easiness of reconfiguration with potentially complex functionality in achieving desired optical tasks, pointing towards promisingly perform on-chip optical computations with further miniaturization.

Optical Stochastic Computing Architectures Using Photonic Crystal Nanocavities

This report proposes a transmission model considering key nanocavity device parameters, such as Quality factors, resonance wavelength and switching efficiency, and proposes the design of XOR gate and multiplexer and illustrates the use of the gates to design an edge detection filter.

Radio-Frequency Multiply-and-Accumulate Operations with Spintronic Synapses

This work demonstrates through physical simulations with parameters extracted from exper-imental devices that frequency-multiplexed assemblies of resonators implement the corner-stone operation of artificial neural networks, the Multiply-And-Accumulate (MAC), directly on microwave inputs.

Arbitrary linear transformations for photons in the frequency synthetic dimension

This work presents a photonic architecture to achieve arbitrary linear transformations by harnessing the synthetic frequency dimension of photons and shows that the same physical structure can be reconfigured to implement a wide variety of manipulations including single-frequency conversion, nonreciprocal frequency translations, and unitary as well as non-unitary transformations.

Robust and Efficient Single-Pixel Image Classificationwith Nonlinear Optics

Nonlinear optics is poised to enable even richer and more complex operations and lift the processing capability of artificial intelligence to yet another level.



Large-Scale Optical Neural Networks based on Photoelectric Multiplication

Simulations of the network using models for digit- and image-classification reveal a "standard quantum limit" for optical neural networks, set by photodetector shot noise, which suggests performance below the thermodynamic limit for digital irreversible computation is theoretically possible in this device.

Digital Electronics and Analog Photonics for Convolutional Neural Networks (DEAP-CNNs)

A Digital Electronic and Analog Photonic (DEAP) CNN hardware architecture that has potential to be 2.8 to 14 times faster while using almost 25% less energy than current state-of-the-art graphical processing units (GPUs).

In-memory computing on a photonic platform

This work shows that integrated optics with collocated data storage and processing can be combined to enable all-photonic in-memory computations, and sets the stage for development of entirely photonic computers.

Photonic Multiply-Accumulate Operations for Neural Networks

This work describes the performance of photonic and electronic hardware underlying neural network models using multiply-accumulate operations, and investigates the limits of analog electronic crossbar arrays and on-chip photonic linear computing systems.

An Optical Frontend for a Convolutional Neural Network

An architecture that utilizes a single electrical to optical conversion by designing a free-space optical frontend unit that implements the linear operations of the first layer with the subsequent layers realized electronically.

Calculating with light using a chip-scale all-optical abacus

The central element of an all-optical calculator is demonstrated, a photonic abacus, which provides multistate compute-and-store operation by integrating functional phase-change materials with nanophotonic chips.

Multipurpose silicon photonics signal processor core

A reconfigurable but simple silicon waveguide mesh with different functionalities with a simple seven hexagonal cell structure is demonstrated, which can be applied to different fields including communications, chemical and biomedical sensing, signal processing, multiprocessor networks, and quantum information systems.

ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars

This work explores an in-situ processing approach, where memristor crossbar arrays not only store input weights, but are also used to perform dot-product operations in an analog manner.

Dot-product engine for neuromorphic computing: Programming 1T1M crossbar to accelerate matrix-vector multiplication

The Dot-Product Engine (DPE) is developed as a high density, high power efficiency accelerator for approximate matrix-vector multiplication, invented a conversion algorithm to map arbitrary matrix values appropriately to memristor conductances in a realistic crossbar array.

Fast and reliable storage using a 5  bit, nonvolatile photonic memory cell

This work demonstrates an optically addressed, multilevel memory capable of storing up to 34 nonvolatile reliable and repeatable levels (over 5 bits) using the phase change material Ge2Sb2Te5 integrated on a photonic waveguide and investigates the influence of write-and-erase pulse parameters on the single-pulse recrystallization, amorphization, and readout error in the memory, thus tailoring pulse properties for optimum performance.