Post

Introducing Phi-3 WebGPU

Posted Updated
By Fodev JEO 1 min read

Introducing Phi-3 WebGPU, a private and powerful AI chatbot that runs locally in your browser, powered by 🤗 Transformers.js and onnxruntime-web!

🔒 On-device inference: no data sent to a server ⚡️ WebGPU-accelerated (> 20 t/s) 📥 Model downloaded once and cached

Phi-3 running at 42 tokens per second 100% locally in your browser! 🤯⚡️

What speed do you get?

Try it out! 👇 ( from Xenova)

https://huggingface.co/spaces/Xenova/experimental-phi3-webgpu

Script, Phi-3
LLM Phi-3 WebGPU
This post is licensed under CC BY 4.0 by the author.