Meta has trained and will release a new major language model to researchers CEO Mark Zuckerberg notified on Friday.
the model, called LLaMAis intended to help researchers and engineers explore applications for AI such as answering questions and summarizing documents.
Meta’s launch of its new model, which was developed by its Fundamental AI Research (FAIR) team, comes as major tech companies and well-capitalized startups race to present advances in artificial intelligence and integrate the technology into commercial products.
Large language models underpin applications such as OpenAI’s ChatGPT, Microsoft Bing AI and Googleis not released Bard.
In his post, Zuckerberg said LLM technology could eventually solve mathematical problems or conduct scientific research.
“LLMs have shown great promise in generating text, conducting conversations, summarizing written material, and more complex tasks such as solving mathematical theorems or predicting protein structures,” Zuckerberg wrote on Friday.
Here is an example of the system output, from Meta’s paper:
Prompt: Here is the recommendation letter I wrote for an application for a dragon feeder position at Magic Unicorn Corporation: Dear Recruiter,
Response: I’ve known ___ for two years, and I think she would make an excellent dragon feeder for the Magic Unicorn Corporation. ___ has an ability to remember and process large amounts of information, which is an important skill for a dragon feeder. ___, as a skilled knight, has a deep understanding of how to kill dragons and how to use each dragon’s weaknesses against it. This means she knows what kind of food each dragon likes and what kind of food is dangerous to each dragon. This knowledge and experience will be invaluable as she feeds the dragons. I am confident that ___’s competence, skill and experience will make her an excellent employee. Contact me at (___) ___-___ if you have any questions. I look forward to hearing from you.
Here’s the model’s production for a fictional scene from “Seinfeld”:
Prompt: (A funny scene between Jerry and George where George introduces his new AI watch)
Response: George: “But you see, Jerry, it’s more than just a clock. It’s a mini computer. You program it however you want. It has a world clock, alarm, calculator, a database and a language translator. It also has a word processor , a spell checker, a full ten function calculator, PDA and a modem. I think it even makes cappuccino.”
Jerry: “Where do you get this?”
George: “From my ad text. Look, Jerry, it’s not a watch. It’s an organizer, a personal assistant, and a voice recorder. You can play chess against it. And you know, if you want, you can program it to speak foreign language. It’s a microprocessor, Jerry.”
Jerry: “George, if you had one of these, would you wear it?”
George: “I’m wearing it right now.”
Jerry: “I don’t see anything on your wrist.”
George: “That’s because it’s a microprocessor.”
Meta says its LLM differs from competitive models in several ways.
First, it says that it comes in several sizes, from 7 billion parameters to 65 billion parameters. Larger models have been successful in recent years in expanding the technology’s capabilities, but they cost more to operate, a phase researchers call “inference.”
For example, OpenAI’s Chat-GPT 3 has 175 billion parameters.
Meta also said it will make its models available to the research public and is accepting applications from researchers. The underlying models for Google’s LaMDA and OpenAI’s ChatGPT are not public.
“Meta is committed to this open research model and we will make our new model available to the AI research community,” Zuckerberg wrote.