What is an open-source LLM by EPFL and ETH Zurich
ETH Zurich and EPFL’s open-weight LLM offers a transparent alternative to black-box AI built on green compute and set for public release.
Large language models (LLMs), which are neural networks that predict the next word in a sentence, are powering today’s generative AI. Most remain closed, usable by the public, yet inaccessible for inspection or improvement. This lack of transparency conflicts with Web3’s principles of openness and permissionless innovation.
So everyone took notice when ETH Zurich and Swiss Federal Institute of Technology in Lausanne (EPFL) announced a fully public model, trained on Switzerland’s carbon‑neutral “Alps” supercomputer and slated for release under Apache 2.0 later this year.
It is generally referred to as “Switzerland’s open LLM,” “a language model built for the public good,” or “the Swiss large language model,” but no specific brand or project name has been shared in public statements so far.
Open‑weight LLM is a model whose parameters can be downloaded, audited and fine‑tuned locally, unlike API‑only “black‑box” systems.
Anatomy of the Swiss public LLM
- Scale: Two configurations, 8 billion and 70 billion parameters, trained on 15 trillion tokens.
- Languages: Coverage in 1,500 languages thanks to a 60 / 40 English–non‑English data set.
- Infrastructure: 10,000 Nvidia Grace‑Hopper chips on “Alps,” powered entirely by renewable energy.
- Licence: Open code and weights, enabling fork‑and‑modify rights for researchers and startups alike.
What makes Switzerland’s LLM stand out
Switzerland’s LLM blends openness, multilingual scale and green infrastructure to offer a radically transparent LLM.
- Open-by-design architecture: Unlike GPT‑4, which offers only API access, this Swiss LLM will provide all its neural-network parameters (weights), training code and data set references under an Apache 2.0 license, empowering developers to fine‑tune, audit and deploy without restrictions.
- Dual model sizes: Will be released in 8 billion and 70 billion parameter versions. The initiative spans lightweight to large-scale usage with consistent openness, something GPT‑4, estimated at 1.7 trillion parameters, does not offer publicly.
- Massive multilingual reach: Trained on 15 trillion tokens…
Click Here to Read the Full Original Article at Cointelegraph.com News…