Korea’s KAIST develops next-gen ultra-low power Gen AI LLM accelerator
Researchers in South Korea have developed the world’s first artificial intelligence (AI) semiconductor chip that operates at ultra-high speeds with minimal power consumption for processing large language models (LLMs), based on principles that mimic the structure and function of the human brain.
The research team was from the Korea Advanced Institute of Science and Technology (KAIST) PIM Semiconductor Research Center and the Graduate School of AI Semiconductor (led by Professor Yu Hoi-jun). This ultra-low power “complementary transformer” semiconductor using Samsung Electronics’ 28 nm process as announced by Korea’s Ministry of Science and ICT on Feb. 6th. The chip is 41 times smaller in area than the Nvidia AI processor, enabling it to be used on devices like mobile phones.
The new AI chip successfully ran GPT 2 using only 1/625 of the power consumption and at 1/41 the size of Nvidia’s A100 graphics processing unit (GPU). This breakthrough is considered a key development in the escalating global AI semiconductor war.
Previously, the technology was less accurate than deep neural networks (DNNs) and mainly capable of simple image classifications, but the research team succeeded in improving the accuracy of the technology to match that of DNNs to apply it to LLMs.
The team said its new AI chip optimizes computational energy consumption while maintaining accuracy by using unique neural network architecture that fuses DNNs and SNNs, and effectively compresses the large parameters of LLMs.
A photo describing an artificial intelligence chip which processes a large language model with neuromorphic computing technology provided by the Ministry of Science and ICT on March 6, 2024.
References:
The ministry said the utilization of neuromorphic computing technology, specifically spiking neural networks (SNNs), is essential to the achievement.
Previously, the technology was less accurate than deep neural networks (DNNs) and mainly capable of simple image classifications, but the research team succeeded in improving the accuracy of the technology to match that of DNNs to apply it to LLMs.
The team said its new AI chip optimizes computational energy consumption while maintaining accuracy by using unique neural network architecture that fuses DNNs and SNNs, and effectively compresses the large parameters of LLMs. (Yonhap)
https://www.koreatimes.co.kr/www2/common/viewpage.asp?newsIdx=370119&categoryCode=113