DesignCon is a trade show focused on electronic product design, electronic components as well as applications that drive demand in electronics. It’s been going on for decades in Silicon Valley. The in-person conference included three keynote speeches. John Bowers, Frad Kavli Chair of Nanotechnology, UCSB, discussed how photonics could be used in high-capacity co-packaged electronics. Laurence Moroney, Head of Artificial Intelligence at Google, talked about practical applications for AI and machine learning. Jose Morey, consultant for NASA, IBM, Hyperloop Transportation and Liberty BioSecurity; gave an inspiring talk on the future of humanity in space, healing from old age and a future made possible by robots.
John Bowers showed the future evolution of co-packaged optics and electronic chips for data center communication, as shown below. True co-packaging will require chip stacking and heterogeneous integration of various chip types, including optical engines. The PIPES project in which UCSB is involved is developing technology for 10 Tbit/s links with an efficiency of 0.5 pJ/bit which includes technologies such as quantum dot lasers.
Electronic products need memory and storage to function and there were several sessions at DesignCon that explored how storage and memory are evolving to meet the needs of current and future products. As shown in the Rambus Talks image below, memory technology is evolving to provide higher bandwidth and capacity and new, more efficient and secure computing architectures, thanks to new interconnects (e.g. CXL) and data center disaggregation.
Memory represents a large portion of server costs and must be used efficiently to provide the best total cost of ownership (see figure below). Processors, memory, and storage have different life cycles and must be replaced separately. This resulted in the use of similar resource pools, such as a memory pool using CXL.
Additionally, accessing data and moving data on-chip is extremely expensive in terms of power (see below). This is causing system and data center designers to rethink architectures to emphasize data locality and minimize data movement.
CXL enables memory disaggregation with short-term memory access modifications shown below. CXL offers memory bandwidth and capacity expansion with “remote memory” providing additional memory levels which can include non-volatile memories.
Conventional memory systems for AI applications include on-chip memory (with highest bandwidth, but limited capacity), HBM (with very high bandwidth and density but high cost), and GDDR (which offers a good compromise between bandwidth, energy efficiency, cost and reliability).
Memory also plays an important role in edge computing, which also reduces potential power consumption by processing data close to where it is generated. While data centers play an important role in ML training, edge computing plays an important role in ML inference. The figure below shows Rambus’ view of memory types for servers, ML training, and inference. The sweet spot for inference favors GDDR6. Accelerator cards seem to play an important role in AI edge computing and automotive applications.
Rambus also offers root of trust solutions for automotive design to prevent the hacking of vehicles that increasingly power computer systems. One of their discussions focused on advanced packaging options, including UCIe (chip specs) and HBM solutions that approach 1TB/s bandwidth.
DesignCon 2022 covered electronic design and integration, including photonic communication in chips. Rambus gave presentations on the need to process data closer to where it is stored and discussed the use of various xDDR and HBM memories for various applications including Edge AI training, Edge inference and ADAS.