This article looks at an AI inference processing technology using light instead of electrons from LIghtmatter and combined with traditional CMOS including SRAM memory. This article is based on an interview with Lightmatter CEO Nick Harris. The company sees this product useful for data center inference and perhaps eventually in some AI computation-intensive industrial and consumer applications (such as autonomous vehicles).
There are widely cited predictions that energy consumption from projects accelerating information and communications technology (ICT) will increase over the 2020s, with a 2018 Nature paper estimating that if current trends continue, it will 2030 will consume more than 20% of electricity demand. At several industry events I’ve heard conversations that one of the major limitations on data center performance will be the amount of energy consumed. NVIDIA’s latest GPU solutions use 400+ W processors, and this power consumption could more than double in future AI processor chips. Solutions that can accelerate key computational functions while consuming less energy will be important for more sustainable and economical data centers.
Lightmatter’s Envise chip (shown below) is a general-purpose machine learning accelerator that combines photonics (PIC) and CMOS transistor-based devices (ASIC) into a single compact module. Using silicon photonics for high-performance AI inference tasks, the device consumes far less energy than CMOS solutions alone, helping to reduce the expected power load of data centers.
The figure below shows the CMOS and photonics chips combined in the Envise compute module. 500 MB of SRAM is used to store weighting levels from a trained machine learning (ML) model.
Envise’s optical chip performs analog processing, so it doesn’t have the precision of floating point calculations used in conventional computers. Thus, Envise processors are more suitable for applications where this lack of precision is not an issue, such as AI inference. Thus, Envise provides a specialized computer that excels at certain types of problems. With the slowdown of traditional CPU scaling, specialized computing devices such as Envise will play an important role in computing specific applications.
Envise works similar to the Google tensorflow devices for general purpose AI applications, except it uses an optical AI processor engine. Any application that uses linear algebra can run on the Envise modules, including AI inference, natural language processing, financial modeling, and ray tracing.
Lightmatter will offer its Envise processors in an Envise server that combines 16 Envise modules with AMD EPYC processors, and SSD and DDR4 DRAM, see diagram below.
Lightmatter has a roadmap for even faster processing by using more colors for parallel processing channels, with each color acting as a separate virtual computer.
Nick said that in addition to data center applications for Envise, he could see the technology being used to enable autonomous electric vehicles that require powerful AI but are limited by battery power, making it easier to provide attractive range per vehicle charge. In addition to the Envise module, Lightmatter also offers optical interconnect technology that it calls Passage.
Lightmatter makes optical AI processors that can deliver fast results with less power consumption than conventional CMOS products. Their compute module combines CMOS logic and memory with optical analog processing units useful for AI inference, natural language processing, financial modeling, and ray tracing.