Insatiable demand for AI services is fueling a new industry of custom chips

Custom silicon chips and artificial intelligence are driving a fundamental shift in data center computing, with custom semiconductor solutions emerging as a critical strategy for managing escalating power consumption and performance demands.

Current state and challenges: The rapid growth of AI services is creating unprecedented demands on data center infrastructure, with AI power consumption expected to increase by 44.7% annually through 2028.

• Data center power consumption is projected to reach 857 terawatt hours by 2028, equivalent to the electricity usage of a nation ranking just behind Japan
• Traditional approaches to chip design and power management are struggling to keep pace with these demands
• Custom silicon solutions are expected to represent 25% of AI accelerators by 2028, according to Marvell estimates

The evolution of data centers: Modern AI facilities are increasingly resembling manufacturing plants, with operational excellence becoming the key determinant of success.

• Providers must optimize for energy consumption per token, equipment reliability, and operational effectiveness
• Competition will center on process design and infrastructure efficiency
• Service providers with the most efficient infrastructure will gain competitive advantages in the market

Custom chip innovation: Semiconductor customization is expanding beyond traditional boundaries to encompass various chip categories and use cases.

• Custom designs range from complete AI accelerators to modified merchant chips with specialized IP and firmware
• Meta has developed custom network interface controllers to improve reliability
• High bandwidth memory interfaces are being redesigned to reduce power consumption by up to 70% and increase memory capacity by 33%

Technical challenges and solutions: Creating custom semiconductors requires significant investment and innovation in fundamental technologies.

• SerDes (Serializer-deserializer) circuits serve as essential building blocks for data movement between chips
• Advanced chip packaging techniques are crucial for power delivery and data path optimization
• AI systems will require hundreds of accelerators connected through specialized optical engines and controllers

Industry transformation: The semiconductor industry is adapting its business model to accommodate the shift toward customization.

• Companies are specializing in different aspects of the custom chip development process
• AI tools are reducing design time and costs from months to minutes
• New service categories are emerging for chip design, manufacturing, and IP development

Looking ahead: The semiconductor industry faces complex challenges but appears positioned to successfully navigate the transition to custom silicon solutions, driven by necessity and enabled by technological innovation.

• The industry’s proven ability to overcome technical challenges suggests these obstacles will be surmounted
• Specialization and collaboration among semiconductor companies will be key to managing costs and complexity
• AI’s role in chip design will continue to expand, making custom solutions more accessible and economical