The demo shows the power of using a smaller sized LLM client side in the web browser. The model I am loading for this demo is a brand new shiny model from Google's MediaPipe team that has ported the latest open research model by Google (Gemma) to run super fast on the GPU in the web browser using technologies like WebGPU.
You could use this Web AI LLM model for many things, but I decided to make this demo where you can use your mouse to select any text in this paragraph of text, right click, and then generate a summary for the selected text! You could imagine turning this into a Chrome extension if you wanted to be able to do this on any text on any website. The great thing about having the model run in the browser client side within the JavaScript ecosystem instead of on Cloud is that your privacy is preserved - none of the text you select to summarize is sent to a 3rd party to get the result! Yep, that means you could even use it offline once the model is downloaded!