By KIM BELLARD
I am a fanboy for AI; I don’t really understand the technical aspects, but I sure am excited about its potential. I’m also a sucker for a catchy phrase. So when I (belatedly) learned about TinyAI, I was hooked.
Now, as it turns out, TinyAI (also know as Tiny AI) has been around for a few years, but with the general surge of interest in AI it is now getting more attention. There is also TinyML and Edge AI, the distinctions between which I won’t attempt to parse. The point is, AI doesn’t have to involve huge datasets run on massive servers somewhere in the cloud; it can happen on about as small a device as you care to imagine. And that’s pretty exciting.
What caught my eye was a overview in Cell by Farid Nakhle, a professor at Temple University, Japan Campus: Shrinking the Giants: Paving the Way for TinyAI. “Transitioning from the landscape of large artificial intelligence (AI) models to the realm of edge computing, which finds its niche in pocket-sized devices, heralds a remarkable evolution in technological capabilities,” Professor Nakhle begins.
AI’s many successes, he believes, “…are demanding a leap in its capabilities, calling for a paradigm shift in the research landscape, from centralized cloud computing architectures to decentralized and edge-centric frameworks, where data can be processed on edge devices near to where they are being generated.” The demands for real time processing, reduced latency, and enhanced privacy make TinyAI attractive.
Accordingly: “This necessitates TinyAI, here defined as the compression and acceleration of existing AI models or the design of novel, small, yet effective AI architectures and the development of dedicated AI-accelerating hardware to seamlessly ensure their efficient deployment and operation on edge devices.”
Professor Nakhle gives an overview of those compression and acceleration techniques, as well as architecture and hardware designs, all of which I’ll leave as an exercise for the interested reader.
If all this sounds futuristic, here are some current examples of TinyAI models:
- This summer Google launched Gemma 2 2B, a 2 billion parameter model that it claims outperforms OpenAI’s GPT 3.5 and Mistral AI’s Mixtral 8X7B. VentureBeat opined: “Gemma 2 2B’s success suggests that sophisticated training techniques, efficient architectures, and high-quality datasets can compensate for raw parameter count.”
- Also this summer OpenAI introduced GPT-4o mini, “our most cost-efficient small model.” It “supports text and vision in the API, with support for text, image, video and audio inputs and outputs coming in the future.”
- Salesforce recently introduced its xLAM-1B model, which it likes to call the “Tiny Giant.” It supposedly only has 1b parameters, yet Marc Benoff claims it outperforms modelx 7x its size and boldly says: “On-device agentic AI is here”
- This spring Microsoft launched Phi-3 Mini, a 3.8 billion parameter model, which is small enough for a smartphone. It claims to compare well to GPT 3.5 as well as Meta’s Llama 3.
- H2O.ai offers Danube 2, a 1.8 b parameter model that Alan Simon of Hackernoon calls the most accurate of the open source, tiny LLM models.
A few billion parameters may not sound so “tiny,” but keep in mind that other AI models may have trillions.
TinyML even has its own foundation, “a worldwide non-profit organization empowering a community of professionals, academia and policy makers focused on low power AI at the very edge of the cloud.” Its ECO Edge workshop next month will focus on “advancing sustainable machine learning at the edge,”
Rajeshwari Ganesan, Distinguished technologist at Infosys, goes so far as to assert, in AI Business, that “Tiny AI is the future of AI.” She shares tinyML’s concern about sustainability; AI’s “associated environmental cost is worrisome. AI already has a huge carbon footprint — even larger than that of the airline industry.” With billions – that’s right, billions — of IoT devices coming online in the next few years, she warns: “the processing power requirements may explode due to the sheer amount of data generated by them. It is imperative to shift some of the compute load to edge devices. Such small AI models can be pushed to edge IoT devices that require minimal energy and processing capacity.”
European tech company Imec is big into TinyAI, and also fears AI’s ecological impact, calling current approaches to AI “economically and ecologically unsustainable.” Instead, it believes: “The era of cloud dominance is ending: future AI environments will be decentralized. Edge and extreme edge devices will do their own processing. They will send a minimum amount of data to a central hub. And they will work – and learn – together.”
The fun part, of course, is imagining what TinyAI could be used for. Professor Nakhle says: “Among the immediate and realistic applications, healthcare stands out as a domain ripe for transformation.” He goes on to describe such potential transformations:
For instance, if paired with accessible pricing tailored to specific regions and nations, wearable devices equipped with TinyAI capabilities can revolutionize patient monitoring by analyzing vital signs and detecting anomalies in real time and promptly alerting users to irregular heart rhythms or fluctuations in blood pressure, facilitating timely intervention and improving health outcomes.
Imec sees healthcare as a particular area of focus, and offers these examples for TinyAI:
Another example is one of my favorite future healthcare technologies, nanorobots. MIT just announced a tiny battery for use in cell-sized robots, which “could enable the deployment of cell-sized, autonomous robots for drug delivery within in the human body,” among other things. Now we’ll just have to get TinyAI into those robots to help achieve the many tasks we’ll be asking of them.
We’re already overflowing with great ideas for how to use AI in healthcare; we’ve barely scratched its potential. Once we get our heads around TinyAI, we’ll find even more ways to apply it. The future is vast…and may be tiny.
Exciting times indeed.
Kim is a former emarketing exec at a major Blues plan, editor of the late & lamented Tincture.io, and now regular THCB contributor
