As I peruse the thought pieces already spewing forth after the latest round of Apple’s announcements, one thought is blooming in my mind.
What cost (mostly) on-device AI?
A while ago I read something about the energy usage of an AI query, I can’t find that article now but some searches suggest the following:
- Google search: roughly 0.3 Wh
- Typical ChatGPT/Gemini text prompt: roughly 0.2–0.4 Wh
- Heavy reasoning or coding task: several Wh to tens of Wh
To put that into perspective:
- Boiling a full kettle: ~100 Wh
- Running a 10W LED bulb for 10 hours: 100 Wh
- Driving a petrol car 1 mile: roughly 250–400 g CO₂
So where does the latest Siri AI feature? It’s based on Gemini, but it’s not wholly Gemini and if the query needs ‘Private Cloud Compute’ it’ll bounce to Apple’s own data centres, which Apple says run on 100% renewable energy (I think).
I’ve not seen much about this, which I am surprised about given none of the other AI companies can claim a lower (?) environmental impact, it feels like a free hit for Apple?
Am I wrong? Anyone? Gruber? Anyone?
What are your thoughts?