14.88 trillion parameter neural network weights -- open source. 55,653 named tensors. 70-layer MoE transformer with Mamba SSM, RetNet, Hyena, GQA, MLA. 1M vocab, 50M context. Compressed to 358 MB. Free for any use.
By chatting or signing in you agree to the Terms and chat-message logging (revocable in History).