TECH

Open-source AI models released by Tokyo lab Sakana founded by former Google researchers

SUMMARY

Tokyo's Sakana AI, founded by ex-Google researchers, unveils a new method for creating AI models inspired by evolution. They use a technique called "model merging" to combine existing models and create new ones.

Then, like natural selection, the most successful models are chosen to inform the next generation. Sakana AI is releasing three Japanese language models, two being open-sourced. This follows a trend of talented researchers leaving Google to pursue their own AI ventures, with funding flowing into these startups. Sakana AI hopes to make Tokyo a global AI hub, similar to OpenAI and DeepMind in other cities.

Tokyo, Japan - March 21, 2024 - In a move that could reshape the landscape of artificial intelligence, a Tokyo-based startup called Sakana AI has unveiled a novel method for developing AI models, inspired by the very foundation of life itself: evolution.

Founded by two prominent ex-Google researchers, David Ha and Llion Jones, Sakana AI departs from traditional AI development techniques. Instead of focusing on meticulously handcrafted algorithms, they leverage a technique called "model merging." This approach combines existing AI models to generate entirely new ones, akin to breeding in the natural world.

But Sakana AI doesn't stop there. They incorporate an "evolutionary" twist. Hundreds of these merged models are created, and the most successful performers from each generation are then selected – just like natural selection. These "parent" models then inform the creation of the next generation, leading to a continuous cycle of improvement.

The results are promising. Sakana AI is releasing three new Japanese language models, with two being made open-source. This transparency allows the broader AI community to build upon their work and accelerate advancements in the field.

The pedigree of Sakana AI's founders adds further weight to their innovation. Both Ha and Jones were instrumental figures at Google, with Jones co-authoring the seminal 2017 paper "Attention Is All You Need," which introduced the transformer architecture – the backbone of the now-viral chatbot ChatGPT. This paper ignited the current race to develop powerful generative AI tools. Ha, with his experience as a Google Brain researcher and head of research at Stability AI, further strengthens Sakana AI's expertise.

This exodus of talent from Google is fueling a wave of new AI ventures. Venture capitalists are lining up to invest in these ventures, recognizing the potential of these revolutionary approaches. Noam Shazeer, another former Google researcher, heads Character.AI, a startup building advanced chatbots. Aidan Gomez, yet another ex-Googler, founded Cohere, a company specializing in large language models

Sakana AI harbors a bold ambition – to position Tokyo as a global hub for AI innovation, mirroring the success of OpenAI (San Francisco) and DeepMind (London). Their recent $30 million seed funding round led by Lux Capital provides the resources to fuel their ambitious goals.

The future of AI is likely to be shaped by a multitude of approaches. Sakana AI's bio-inspired method stands as a testament to the ingenuity and potential of alternative development strategies. By harnessing the power of evolution, this Tokyo startup is poised to play a significant role in the next chapter of artificial intelligence.