Greg Martin is Strategic Marketing Director at Xilinx
In dynamic and evolving markets – such as 5G, data center, automotive and industrial – applications require ever-increasing computational acceleration while staying within tight power limits. A key factor in the demand for higher computational density is artificial intelligence (AI), whose adoption is accelerating rapidly.
AI inference needs high processing performance with tight energy budgets, whether deployed in the cloud, edge or endpoint. Dedicated AI hardware is often needed to accelerate AI inference workloads.
At the same time, AI algorithms are evolving much faster than the speed of traditional silicon development cycles. Solid silicon chips, such as ASIC implementations of AI networks, are at risk of obsolescence very quickly due to the rapid innovation in advanced AI models.
Adaptive computing is the answer to these challenges.
Adaptive computing is unique in that it consists of silicon hardware that can be optimized for specific applications after manufacture. Because the optimization occurs after the hardware is manufactured, it can be configured with the latest AI models, models that did not exist when competing ASICs were designed. This optimization can also be run and rerun an almost infinite number of times, providing unique flexibility. Hardware changes can be made even after the device is fully deployed in a production environment. Just as a production CPU can be given a new program to run, an adaptive platform can be given a new hardware configuration to adapt, even in a live production environment.
Adaptive hardware versus alternatives
CPUs and GPUs have unique capabilities and are well suited to certain tasks. CPUs are optimal for decision making functions where complex logic needs to be evaluated. GPUs are optimal for offline data processing where high throughput is required, but latency is not an issue. Adaptive computing is optimal when high throughput with low latency is required, such as real-time video streaming, 5G communications and sensor fusion in the automotive sector.
The reason adaptive computing can deliver high performance with low latency is its ability to enable domain-specific architectures (DSAs) that optimally deploy specific applications within specific domains. In contrast, CPUs and GPUs have fixed von-Neumann-based architectures that do not allow domain optimization of their underlying architecture.
DSAs can also be built using a special (solid) silicon device, commonly referred to as an application-specific standard product or ASSP. While there are advantages to implementing a DSA in a fixed ASSP, there are also disadvantages.
First, there is the pace of innovation. To keep up, manufacturers are expected to create and deliver new services in shorter time frames than ever before. More specifically, the timeframes are shorter than the time it takes to design and build a new solid silicon DSA. This creates a fundamental mismatch in the market between the market demands for innovation and the time it takes companies to design and manufacture ASSPs. Changes in industry standards or other fluctuating requirements can quickly make such devices obsolete.
The second consideration is the cost of custom silicon. The one-time cost of designing and manufacturing a unique silicon design, such as a complex 7nm ASIC, can be several hundred million dollars in one-time engineering costs (NRE). Costs are expected to increase further as device geometries shrink to 5nm and below. The cost increase is slowing the adoption of advanced nodes for ASSPs, leaving their users with outdated and less efficient technology.
Introducing adaptive platforms
Adaptive platforms are all built on the same fundamental adaptive hardware foundation; however, they include much more than just the silicone hardware or device. Adaptive platforms include a comprehensive set of runtime software. In combination, the hardware and software provide a unique opportunity from which to build highly flexible, yet efficient applications.
These devices make adaptive computing accessible to a wide range of software and system developers. These platforms can be used as the basis for many products, including the following benefits:
Shorter time to market. An application built using a platform such as the Alveo data center accelerator card can use accelerated hardware for a specific application, but requires no hardware customization. A PCIe card is added to the server and accelerated libraries are called directly from an existing software application.
Lower operating costs. Optimized applications based on an adaptive platform can provide significantly higher efficiency per node than CPU-only solutions due to increased compute density.
Flexible and dynamic workloads. Adaptive platforms can be reconfigured depending on current needs. Developers can easily switch between the applications deployed within an adaptive platform, using the same equipment to meet changing workload needs.
Future-proof designs. Adaptive platforms can be continuously adapted. If new features are required in an existing application, the hardware can be reprogrammed to best implement these features, reducing the need for hardware upgrades and extending the life of the system.
Acceleration of the whole application. Rarely does AI inference exist on its own. It is part of a larger chain of data analysis and processing, often with multiple pre- and post-stages using a traditional (non-AI) implementation. The embedded AI parts of these systems benefit from AI acceleration. The non-AI parts also benefit from acceleration. The flexible nature of adaptive computing is suitable for accelerating both the AI and non-AI processing tasks. This is called “application-wide acceleration” and it has become increasingly important as compute-intensive AI inference permeates more applications.
Adaptive Platform Accessibility
In the past, to take advantage of FPGA technology, developers had to build their own hardware boards and use a hardware description language (HDL) to configure the FPGA. In contrast, adaptive platforms allow developers to take advantage of adaptive computing directly from their familiar software frameworks and languages such as C++, Python, TensorFlow, etc. Software and AI developers can now use adaptive computing without having to build a board or hire hardware experts. to be.
Different types of adaptive platforms
There are many types of adaptive platforms based on the application and need, including data center acceleration maps and standardized edge modules. Multiple platforms exist to give the best possible starting point for the desired application. Applications range widely from latency-sensitive applications, such as autonomous driving and real-time streaming video, to the high complexity of 5G signal processing and the data processing of unstructured databases.
Adaptive computing can be deployed across the cloud, network, edge, and even at the endpoint, bringing the latest architectural innovations to discrete and end-to-end applications. The range of deployment locations is enabled by a variety of adaptive platforms — from high-capacity devices on PCIe accelerator cards in the data center to small, power-efficient devices suitable for endpoint processing required by IoT devices.
Introducing the AI Engine
One of the biggest recent innovations in adaptive computing was the introduction of the AI engine by Xilinx. The AI engine is essentially still a configurable block, but it’s also programmable as a CPU. Rather than being formed from standard FPGA processing hardware, an AI engine contains high-performance scalar and SIMD (single instruction) vector processors. These processors are optimized for efficiently implementing math functions typically found in AI inference and wireless communications.
Arrays of AI engines are still connected with FPGA-like, customizable data interconnects that allow building efficient, optimized data paths for the target application. This combination of computationally dense (mathematical), CPU-like processing elements connected with FPGA-like interconnection heralds a new generation of AI and communication products.
Prepare for a more connected and intelligent world
Essentially, adaptive computing builds on existing FPGA technology but makes it more accessible to a wider range of developers and applications than ever before. Software and AI developers can now build optimized applications using adaptive computing, a technology previously unavailable to them.
The ability to adapt hardware to a specific application is a unique differentiator of CPUs, GPUs and ASSPs, which have fixed hardware architectures. Adaptive computing allows the hardware to adapt to an application, providing high efficiency, yet allowing for future adaptations as workloads or standards evolve.
As the world becomes more connected and intelligent, adaptive computing will continue to lead the way in optimized, accelerated applications, empowering all developers to build a brighter future.