IBM Research has developed a new technology that brings optical connections directly to chips. Co-packaged optics should improve energy efficiency and increase bandwidth for training and deploying large language models in data centers.
IBM has developed new technology in which optical connectors are integrated on chips and circuit boards. That technology is called co-packaged optics. It should make data centers more economical, but also faster. Like many innovations today, IBM mentions this one in the same breath as AI.
Traditional data centers use electrical signals for communication between components. That process is energy-intensive and limited in speed. IBM’s co-packaged optics technology replaces the electrical connections with optical connections, allowing data to be transmitted more efficiently and faster. IBM makes that possible by using polymer optical waveguides. According to the company, the breakthrough is a world first.
The new method increases beachfront density – the number of optical fibers that can be connected at the edge of a chip – by a factor of six. As a result, data centers can count on energy consumption of less than one picojoule per bit, compared with five picojoules with traditional electrical connections. The result is energy savings of more than 80 percent.
Application in generative AI
IBM specifically frames the technology within the growing need for bandwidth in generative AI. Large language models require ever faster connections between chips and data centers. Integrating optical links directly on circuit boards and chips can speed up training processes for AI models. In addition, GPU accelerators can work more effectively because they waste less time waiting for data.
IBM conducted extensive testing of the polymer optical waveguides, including stress tests at temperatures ranging from -40°C to 125°C, to ensure reliability and durability. These tests were conducted at IBM facilities in New York and Quebec.
The technology is designed to be compatible with standard manufacturing methods, enabling production at scale. IBM is now working on further development and collaboration with partners to make co-packaged optics available for commercial applications.