Introducing Phi-3 WebGPU

Posted May 8, 2024 Updated May 14, 2024

By Fodev Jeong 1 min read

Introducing Phi-3 WebGPU, a private and powerful AI chatbot that runs locally in your browser, powered by 🤗 Transformers.js and onnxruntime-web!

🔒 On-device inference: no data sent to a server ⚡️ WebGPU-accelerated (> 20 t/s) 📥 Model downloaded once and cached

Phi-3 running at 42 tokens per second 100% locally in your browser! 🤯⚡️

What speed do you get?

Try it out! 👇 ( from Xenova)

This post is licensed under CC BY 4.0 by the author.