Join leaders in Boston on March 27 for an exclusive night of networking, insights, and conversation. Request an invite here.
True to his word, billionaire multi-company leader Elon Musk’s startup xAI today made its first large language model (LLM) Grok open source.
The move, which Musk had previously proclaimed would happen this week, now enables any other entrepreneur, programmer, company, or individual to take Grok’s weights — the strength of connections between the model’s artificial “neurons,” or software modules that allow the model to make decisions and accept inputs and provide outputs in the form of text — and other associated documentation and use a copy of the model for whatever they’d like, including for commercial applications.
“We are releasing the base model weights and network architecture of Grok-1, our large language model,” the company announced in a blog post. “Grok-1 is a 314 billion parameter Mixture-of-Experts model trained from scratch by xAI.”
Those interested can download the code for Grok on its Github page or via a torrent link.
VB Event
The AI Impact Tour – Atlanta
Continuing our tour, we’re headed to Atlanta for the AI Impact Tour stop on April 10th. This exclusive, invite-only event, in partnership with Microsoft, will feature discussions on how generative AI is transforming the security workforce. Space is limited, so request an invite today.
Request an invite
What Grok’s open sourcing means
Parameters refers to the weights and biases that govern the model — the more parameters, generally the more advanced, complex and performant the model is. At 314 billion parameters, Grok is well ahead of open source competitors such as Meta’s Llama 2 (70 billion parameters) and Mistral 8x7B (12 billion parameters).
Grok was open sourced under an Apache License 2.0, which enables commercial use, modifications, and distribution, though it cannot be trademarked and there is no liability or warranty that users receive with it. In addition, they must reproduce the original license and copyright notice, and state the changes they’ve made.
Grok’s architecture, developed using a custom training stack atop JAX and Rust in October 2023, incorporates innovative approaches to neural network design. The model utilizes 25% of its weights for a given token, a strategy that enhances its efficiency and effectiveness
Grok was initially released as a proprietary or “closed source” model back in November 2023 and it was, until now, accessible only on Musk’s separate but related social network X (formerly Twitter), specifically through the X Premium+ paid subscription service, which costs $16 per month or $168 per year.
However, Grok’s release does not include the full corpus of its training data. This doesn’t really matter for using the model, since it has already been trained, but it does not allow for users to see what it learned from — presumably user text posts on X (the xAI blog post states it opaquely as “Base model trained on a large amount of text data, not fine-tuned for any particular task.”)
It also does not include any hookup to the realtime information available on X, which Musk initially tou …
Article Attribution | Read More at Article Source
Join leaders in Boston on March 27 for an exclusive night of networking, insights, and conversation. Request an invite here.
True to his word, billionaire multi-company leader Elon Musk’s startup xAI today made its first large language model (LLM) Grok open source.
The move, which Musk had previously proclaimed would happen this week, now enables any other entrepreneur, programmer, company, or individual to take Grok’s weights — the strength of connections between the model’s artificial “neurons,” or software modules that allow the model to make decisions and accept inputs and provide outputs in the form of text — and other associated documentation and use a copy of the model for whatever they’d like, including for commercial applications.
“We are releasing the base model weights and network architecture of Grok-1, our large language model,” the company announced in a blog post. “Grok-1 is a 314 billion parameter Mixture-of-Experts model trained from scratch by xAI.”
Those interested can download the code for Grok on its Github page or via a torrent link.
VB Event
The AI Impact Tour – Atlanta
Continuing our tour, we’re headed to Atlanta for the AI Impact Tour stop on April 10th. This exclusive, invite-only event, in partnership with Microsoft, will feature discussions on how generative AI is transforming the security workforce. Space is limited, so request an invite today.
Request an invite
What Grok’s open sourcing means
Parameters refers to the weights and biases that govern the model — the more parameters, generally the more advanced, complex and performant the model is. At 314 billion parameters, Grok is well ahead of open source competitors such as Meta’s Llama 2 (70 billion parameters) and Mistral 8x7B (12 billion parameters).
Grok was open sourced under an Apache License 2.0, which enables commercial use, modifications, and distribution, though it cannot be trademarked and there is no liability or warranty that users receive with it. In addition, they must reproduce the original license and copyright notice, and state the changes they’ve made.
Grok’s architecture, developed using a custom training stack atop JAX and Rust in October 2023, incorporates innovative approaches to neural network design. The model utilizes 25% of its weights for a given token, a strategy that enhances its efficiency and effectiveness
Grok was initially released as a proprietary or “closed source” model back in November 2023 and it was, until now, accessible only on Musk’s separate but related social network X (formerly Twitter), specifically through the X Premium+ paid subscription service, which costs $16 per month or $168 per year.
However, Grok’s release does not include the full corpus of its training data. This doesn’t really matter for using the model, since it has already been trained, but it does not allow for users to see what it learned from — presumably user text posts on X (the xAI blog post states it opaquely as “Base model trained on a large amount of text data, not fine-tuned for any particular task.”)
It also does not include any hookup to the realtime information available on X, which Musk initially tou …nnDiscussion:nn” ai_name=”RocketNews AI: ” start_sentence=”Can I tell you more about this article?” text_input_placeholder=”Type ‘Yes'”]