Home » Away from OpenAI: Independent language models show what is possible

Away from OpenAI: Independent language models show what is possible

by admin
Away from OpenAI: Independent language models show what is possible

If many celebrities from the tech industry are now calling for a break in researching large language models in an open letter, you can do it like computational linguist Emily Bender gigantic hype criticize.

But you can also simply shrug your shoulders, because the genie has long been out of the bottle: Even if OpenAI, Meta and Google decided overnight to freeze all development work on large language models and take their models offline, it would be possible not stop further development. Because a large number of models are available online – and their number is constantly growing. Stanford University’s Center for Research on Foundation Models has published an interactive list of technical information on large language models, listing over 50 models. And these are only the officially available ones – leaked models like Alpaca are not listed here.

Because what is often forgotten in all the excitement about the AI ​​race between OpenAI, Meta and Google is the fact that the big Silicon Valley companies have a development advantage and can throw a lot of resources on the problem. However, since the architecture of the large language models is essentially known, the training of such large models is primarily a technical and organizational problem. Even though the vast majority of papers and patents in this area now come from industry, academic research still plays a role – which may even be growing again, as Stanford University’s alpaca model shows, training for which only costs around $600 has cost.

See also  Abe: Japan should cooperate with AUKUS in the field of network and AI | Shinzo Abe | Artificial Intelligence | Cyber ​​Security

Last but not least, the Eleuther AI developer group has shown that independent groups can also get involved. The team – originally a loose international association of software developers organized via Discord – was able to present an open source version of GPT-3 with GPT-Neo in March 2021. The developers are now in the process of becoming more professional and want to employ up to 20 researchers in a non-profit organization. Finally, the international cooperation project BigScience was able to publish Bloom 2022, an open-source language model with 176 billion parameters. The recently published Open Flamingo by the LAION group shows that development does not stop at language models, but that multimodal models are also freely published.

If you want to get an impression of the performance of large language models without registering with every platform or even installing models yourself, you now have a whole range of different options, depending on your technical interest and level of knowledge: The Poe app from the online service Quora for example combines limited access to GPT-4, Claude+ by Antrophic (founded by former OpenAI people), ChatGPT and the smaller models Sage and Dragonfly.

There are more comparison options with currently 20 models at the online service Open Playground. Former Github CEO Nathaniel Friedman recently released the platform to make it easier to compare models. If you want to get really deep, you can choose from an intimidatingly large collection of models on cloud development platforms like Huggingface, or train your own models on Google Colab.

See also  ChatGPT4 has its say on crypto and Bitcoin


(wst)

To home page

You may also like

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Accept Read More

Privacy & Cookies Policy