Unlocking the Power of Etched ASIC: Revolutionizing AI Hardware

Etched is making waves in the artificial intelligence hardware space with its revolutionary new AI accelerator chip. The Silicon Valley startup, founded in 2022 by Harvard dropouts Gavin Uberti and Chris Zhu, has developed a custom application-specific integrated circuit (ASIC) called Sohu that is purpose-built to run transformer models – the architecture behind today’s most advanced AI systems.

Etched transformer ASICS for LLMs

Etched claims its Sohu chip can process AI workloads up to 20 times faster than Nvidia’s top-of-the-line GPUs while using significantly less power. With $120 million in fresh funding and partnerships with major cloud providers, Etched is positioning itself as a formidable challenger to Nvidia’s dominance in AI chips.

Primary Venture Partners and Positive Sum Ventures led the funding round, which included participation from high-profile investors like Peter Thiel, Github CEO Thomas Dohmke, and former Coinbase CTO Balaji Srinivasan. As transformer models continue to drive breakthroughs in generative AI, Etched’s specialized hardware could reshape the landscape of AI computing.

Etched’s approach targets the complexities of GPUs and TPUs, particularly the need to handle arbitrary CUDA and PyTorch code, which demands sophisticated compilers. While other AI chip developers like AMD, Intel, and AWS have invested billions into software development with limited success, Etched is narrowing its focus. By exclusively running transformers, Etched can streamline software development for these models.

Most AI companies use transformer-specific inference libraries such as TensorRT-LLM, vLLM, or HuggingFace’s TGI. Although somewhat inflexible, these frameworks suffice for most needs because transformer models across different applications—text, image, or video—are fundamentally similar. This allows users to adjust model hyperparameters without altering the core model code. However, the most prominent AI labs often require custom solutions, employing engineers to optimize GPU kernels meticulously.

Etched aims to eliminate the need for reverse engineering by making its entire software stack open source, from drivers to kernels. This openness allows engineers to implement custom transformer layers as needed, enhancing flexibility and innovation.

Etched’s approach to AI hardware is comparable to the advancements seen with Groq’s LPU Inference Engine. Groq’s LPU, a dedicated Language Processing Unit, has set new benchmarks in…