Apple Releases OpenELM: Small, Open Source AI Models Designed To Run On-device (venturebeat.com) 15
Just as Google, Samsung and Microsoft continue to push their efforts with generative AI on PCs and mobile devices, Apple is moving to join the party with OpenELM, a new family of open source large language models (LLMs) that can run entirely on a single device rather than having to connect to cloud servers. From a report: Released a few hours ago on AI code community Hugging Face, OpenELM consists of small models designed to perform efficiently at text generation tasks. There are eight OpenELM models in total -- four pre-trained and four instruction-tuned -- covering different parameter sizes between 270 million and 3 billion parameters (referring to the connections between artificial neurons in an LLM, and more parameters typically denote greater performance and more capabilities, though not always).
[...] Apple is offering the weights of its OpenELM models under what it deems a "sample code license," along with different checkpoints from training, stats on how the models perform as well as instructions for pre-training, evaluation, instruction tuning and parameter-efficient fine tuning. The sample code license does not prohibit commercial usage or modification, only mandating that "if you redistribute the Apple Software in its entirety and without modifications, you must retain this notice and the following text and disclaimers in all such redistributions of the Apple Software." The company further notes that the models "are made available without any safety guarantees. Consequently, there exists the possibility of these models producing outputs that are inaccurate, harmful, biased, or objectionable in response to user prompts."
[...] Apple is offering the weights of its OpenELM models under what it deems a "sample code license," along with different checkpoints from training, stats on how the models perform as well as instructions for pre-training, evaluation, instruction tuning and parameter-efficient fine tuning. The sample code license does not prohibit commercial usage or modification, only mandating that "if you redistribute the Apple Software in its entirety and without modifications, you must retain this notice and the following text and disclaimers in all such redistributions of the Apple Software." The company further notes that the models "are made available without any safety guarantees. Consequently, there exists the possibility of these models producing outputs that are inaccurate, harmful, biased, or objectionable in response to user prompts."
Small models (Score:3)
In my experience, the only thing to challenge the frontrunner model (ChatGPT 4, April 2024) is LLaMA-3 70B. Anything smaller is brainless in comparison, and since even the best is still "oh wow..that's mostly right," I don't think anyone will want to settle for "eh...it kind of works."
It wouldn't surprise me to see silicon dedicated to a specific transformers architecture, but until then I think computation is quite intensive and slow. I'm running L3 70B on 2x 3090s and it's still just a few tokens per second, and that requires, what, 500-750W of power for ~60s?
Re: Small models (Score:2)
I mean, generic AI hardware is good at accelerating by far the majority of a transformer. Itâ(TM)s still all just a bunch of matrix multiplies, just⦠a *bunch* of them. That said, it surprises me that Apple havenâ(TM)t thrown more into making their neural engine more powerful.
Re: (Score:3)
I don't really follow Apple, but my understanding is they're skipping the M3 chip and going to an M4 AI focused serious of notebooks in about a year.
https://www.pcmag.com/news/ai-... [pcmag.com]
Methinks this free software is to develop a market for the new hardware.
Re: (Score:2)
I don't really follow Apple, but my understanding is they're skipping the M3 chip and going to an M4 AI focused serious of notebooks in about a year.
https://www.pcmag.com/news/ai-... [pcmag.com]
Methinks this free software is to develop a market for the new hardware.
The M3 variants have been in all Mac laptops and the iMac (IIRC) for several months now. They are skipping the M3 in some Desktops (Mac mini, Mac Studio and Mac Pro); because the M4 is about to be ready for Production, and it would make absolutely no sense to just stuff an M3 in there, just for a few months.
Re: (Score:2)
Re: (Score:2)
A bit faster than that but yeah, it can take a bit for my 70B 4-bit quant. Definitely slower than ChatGPT 4, glacial compared to 3.5, but still fast enough to use comfortably. About the speed of having a conversation with an average typist.
What is this? (Score:2)
"AI" has no Moral Use (Score:2)
Or... our economy and society need to change? (Score:3)
Money hasn't been a real measure of value or effort in a long time. Yes, it's a way to force certain results (allocating limited resources based on the mistaken assumption of rationality and fairness), but it's not the only way.
Just because a machine can imitate a human's results doesn't make it instantly immoral. Tools can be used in immoral ways, yes... And when we're driven only by the almighty dollar is immoral action really surprising?
Is it more or less immoral to force people to do work when we cou
Re: (Score:2)
Re: (Score:2)
But I do think this time it will be somewhat different and that we'll be running open source AIs on our local devices. This will be an epic end-run around the corporate interests whose "intellectual property" propaganda you seem so heavily invested in.
Re: (Score:2)
Huh? You can extrapolate that argument to almost any machine. Sewing machine .. took jobs from seamstresses. Automobiles .. took jobs from stagecoach drivers. Printing press .. took jobs from scribes.
"sample code license"? (Score:2)
Can someone explain the licensing terms better? Is this just a way to offer something with no liability to the original releaser?
I'm not an AI expert so don't understand what all the additional information allows versus what other releases have given. Nor the additional work needed by each to actually use them.
Re: "sample code license"? (Score:1)
Type your question into an AI prompt :)
Re: (Score:3)
Apple's example code license:
https://developer.apple.com/su... [apple.com]
It appears there are basically two conditions:
1) if you distribute the thing unmodified you include the license text
2) you don't blame Apple for whatever happens, and if you're modifying it you don't stamp their name on it.
The fact that they're licensing the model weights themselves, and maybe some other stuff useful for training these things, means that you can fine tune, retrain, whatever the model (modify it to your purposes) and then it's you