The calculation is at an inflection point. Moore’s Law, which predicts that the number of transistors on an electronic chip will double every year, is slowing because of the physical limits to fitting more transistors onto affordable microchips. These increases in computing power are slowing as demand increases for high-performance computers that can support increasingly complex artificial intelligence models. This inconvenience has prompted engineers to explore new methods to expand the computing power of their machines, but a solution remains unclear.
Photonic computing is a potential remedy for the growing computational demands on machine learning models. Instead of using transistors and wires, these systems use photons (microscopic particles of light) to perform computational operations in the analog domain. Lasers produce these tiny bundles of energy, which move at the speed of light like a spaceship flying at warp speed in a science fiction movie. When photonic computing cores are added to programmable accelerators like a network interface card (NIC, and its extended counterpart, SmartNICs), the resulting hardware can be plugged in to power a regular computer.
MIT researchers have now harnessed the potential of photonics to accelerate modern computing by demonstrating its capabilities in machine learning. Their photonic-electronic reconfigurable SmartNIC, called “Lightning,” helps deep neural networks — machine learning models that mimic how brains process information — complete inference tasks like image recognition and language generation in chatbots like ChatGPT. The prototype’s new design enables impressive speeds and creates the first photonic computing system to serve machine learning requests in real time.
Despite its potential, a major challenge in implementing photonic computing devices is that they are passive, meaning they lack memory or instructions to control data flows, unlike their electronic counterparts. Previous photonic computing systems faced this bottleneck, but Lightning removes this barrier to ensure data movement between electronic and photonic components runs smoothly.
“Photonic computing has shown significant advantages in accelerating bulky linear computational tasks such as matrix multiplication, while requiring electronics to take care of the rest: memory access, nonlinear computations, and conditional logic. This creates a significant amount of data to be exchanged between photonics and electronics to complete real computing tasks, such as a machine learning request, says Zhizhen Zhong, a postdoc in the group of MIT Associate Professor Manya Ghobadi at the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). “Controlling this flow of data between photonics and electronics was the Achilles’ heel of the past state-of-the-art photonic computing works. Even if you have a super-fast photonic computer, you need enough data to run it without stalling. Otherwise, you have a supercomputer that is just idle without doing any reasonable calculation.”
Ghobadi, an associate professor in MIT’s Department of Electrical Engineering and Computer Science (EECS) and a CSAIL member, and her group colleagues are the first to identify and solve this problem. To accomplish this feat, they combined the speed of photonics and the data flow control capabilities of electronic computers.
Before lightning, photonic and electronic computer systems operated independently and spoke different languages. The team’s hybrid system tracks the necessary computational operations on the data path using a reconfigurable count-action abstraction, which connects photonics to the electronic components of a computer. This programming abstraction acts as a unified language between the two and controls access to the data flows that pass through. Information carried by electrons is translated into light in the form of photons, which operate at the speed of light to help complete an inference task. Then the photons are converted back into electrons to relay the information to the computer.
By seamlessly coupling photonics with electronics, the new count-action abstraction makes Lightning’s fast real-time computation rate possible. Previous attempts used a stop-and-go approach, meaning the data would be hindered by a much slower control software that made all the decisions about its movements. “Building a photonic computing system without an abstraction of count-action programming is like trying to drive a Lamborghini without knowing how to drive,” said Ghobadi, who is a senior author of the paper. “What would you do? You probably have a driver’s manual in one hand, then push the clutch, then check the manual, then release the brake, then check the manual, etc. This is a stop-and-go operation because for every decision you have to consult some higher level to tell you what to do. But that’s not how we drive; we learn to drive and then use muscle memory without checking the manual or driving rules behind the wheel. Our count-action programming abstraction works like the muscle memory in Lightning . It seamlessly drives the electrons and photons in the system while driving.”
An environmentally friendly solution
Machine learning services that complete inference-based tasks, such as ChatGPT and BERT, currently require heavy computing resources. Not only are they expensive – some estimates show that ChatGPT demands 3 million dollars per month to drive — but they are also harmful to the environment, potentially emitting more than twice as much carbon dioxide as the average person. Lightning uses photons that move faster than electrons do in wires, while generating less heatwhich allows it to compute at a faster rate while being more energy efficient.
To measure this, the Ghobadi group compared their device to standard graphics processing units, computing units, SmartNICs and other accelerators by synthesizing a Lightning chip. The team observed that Lightning was more energy efficient when completing inference requests. “Our synthesis and simulation studies show that Lightning reduces machine learning power consumption by orders of magnitude compared to state-of-the-art accelerators,” said Mingran Yang, a PhD student in Ghobadi’s lab and co-author of the paper. By being a more cost-effective and faster option, Lightning presents a potential upgrade for data centers to reduce the carbon footprint of its machine learning model while speeding inference response time for users.
Additional authors on the paper are MIT CSAIL postdoc Homa Esfahanizadeh and graduate student Liam Kronman, as well as MIT EECS associate professor Dirk Englund and three recent graduates within the department: Jay Lang ’22, MEng ’23; Christian Williams ’22, MEng ’23; and Alexander Sludds ’18, MEng ’19, PhD ’23. Their research was supported in part by the DARPA FastNICs program, the ARPA-E ENLITENED program, the DAF-MIT AI Accelerator, the United States Army Research Office through the Institute for Soldier Nanotechnologies, National Science Foundation (NSF) grants, the NSF Center for Quantum Networks, and a Sloan Fellowship.
The group will present its findings at the Association for Computing Machinery’s Special Interest Group on Data Communication (SIGCOMM) this month.
#system #combines #light #electrons #unlock #faster #greener #computing
