What is next for LLMs? Pushing the boundaries of next-gen AI computing hardware with photonic chips.

Large language models (LLMs) are rapidly pushing the limits of contemporary computing hardware. For example, training GPT-3 has been estimated to consume around 1,300 MWh of electricity, and projections suggest future models may require city-scale (gigawatt) power budgets. These demands motivate exploration of computing paradigms beyond conventional von Neumann architectures. This review surveys emerging photonic hardware optimized for next-generation generative AI computing. We discuss integrated photonic neural network architectures (e.g. Mach-Zehnder interferometer meshes, lasers, wavelength-multiplexed microring-resonators) that perform ultrafast matrix operations. We also examine promising alternative neuromorphic devices and platforms, including 2D materials and hybrid spintronic-photonic synapses, which combine memory and processing. The integration of two-dimensional materials (graphene, TMDCs) into silicon photonic platforms is reviewed for tunable modulators and on-chip synaptic elements. Transformer-based LLM architectures (self-attention and feed-forward layers) are analyzed in this context, introducing the mathematical operations associated with the transformers and identifying strategies and challenges for mapping dynamic matrix multiplications onto these novel photonic hardware systems. Overall, we broadly introduce state-of-the-art photonic components, AI algorithms, and system integration methods, highlighting key advances and open issues in scaling such photonic systems to mega-sized LLM models. We find that photonic computing systems could potentially surpass electronic processors by orders of magnitude in throughput and energy efficiency, but require breakthroughs in memory especially for long-context windows and long token sequences and in storage of ultra-large datasets, among others. This survey provides a comprehensive roadmap for AI hardware development, emphasizing the role of cutting-edge photonic components and technologies in supporting future LLMs.