Join us in Atlanta on April 10th and explore the landscape of security workforce. We will explore the vision, benefits, and use cases of AI for security teams. Request an invite here.
Ever since the groundbreaking research paper “Attention is All You Need” debuted in 2017, the concept of transformers has dominated the generative AI landscape.
Transformers however are not the only path forward with generative AI. A new approach from AI21 Labs dubbed “Jamba” looks to go beyond transformers. Jamba combines the Mamba model, which is based on the Structured State Space model (SSM), along with a transformer architecture to create an optimized gen AI model. Jamba is an acronym that stands for Joint Attention and Mamba (Jamba) architecture, and it aims to bring the best attributes of SSM and transformers together. Jamba is being released as an open-source model under the Apache 2.0 license.
To be clear, it’s not likely that Jamba will replace current transformer-based large language models (LLM) today, but it will likely be a supplement in certain areas. According to AI21 Labs Jamba can outperform traditional transformer-based models on generative reasoning tasks as measured by benchmarks such as HellaSwag. However, it currently does not outperform transformer-based models on other critical benchmarks such as the Massive Multitask Language Understanding (MMLU) for problem-solving.
Jamba isn’t just a new Jurassic take from AI21 Labs
AI21 Labs has a particular focus on gen AI for enterprise use cases. The company raised $155 million in Aug. 2023 to support it’s growing efforts.
VB Event
The AI Impact Tour – Atlanta
Continuing our tour, we’re headed to Atlanta for the AI Impact Tour stop on April 10th. This exclusive, invite-only event, in partnership with Microsoft, will feature discussions on how generative AI is transforming the security workforce. Space is limite …
Article Attribution | Read More at Article Source
Join us in Atlanta on April 10th and explore the landscape of security workforce. We will explore the vision, benefits, and use cases of AI for security teams. Request an invite here.
Ever since the groundbreaking research paper “Attention is All You Need” debuted in 2017, the concept of transformers has dominated the generative AI landscape.
Transformers however are not the only path forward with generative AI. A new approach from AI21 Labs dubbed “Jamba” looks to go beyond transformers. Jamba combines the Mamba model, which is based on the Structured State Space model (SSM), along with a transformer architecture to create an optimized gen AI model. Jamba is an acronym that stands for Joint Attention and Mamba (Jamba) architecture, and it aims to bring the best attributes of SSM and transformers together. Jamba is being released as an open-source model under the Apache 2.0 license.
To be clear, it’s not likely that Jamba will replace current transformer-based large language models (LLM) today, but it will likely be a supplement in certain areas. According to AI21 Labs Jamba can outperform traditional transformer-based models on generative reasoning tasks as measured by benchmarks such as HellaSwag. However, it currently does not outperform transformer-based models on other critical benchmarks such as the Massive Multitask Language Understanding (MMLU) for problem-solving.
Jamba isn’t just a new Jurassic take from AI21 Labs
AI21 Labs has a particular focus on gen AI for enterprise use cases. The company raised $155 million in Aug. 2023 to support it’s growing efforts.
VB Event
The AI Impact Tour – Atlanta
Continuing our tour, we’re headed to Atlanta for the AI Impact Tour stop on April 10th. This exclusive, invite-only event, in partnership with Microsoft, will feature discussions on how generative AI is transforming the security workforce. Space is limite …nnDiscussion:nn” ai_name=”RocketNews AI: ” start_sentence=”Can I tell you more about this article?” text_input_placeholder=”Type ‘Yes'”]