←
Morgan Funtowicz319 posts
Morgan Funtowicz avatar

Morgan Funtowicz

@MorganFunto

Machine Learning Engineer - @HuggingFace / Paris 🇫🇷

◷ Joined May 2012▧ 21 Photos and videos
Llama moved to @llama_app avatar
⟳ Morgan Funtowicz retweeted
Llama moved to @llama_app@llama_cpp·Oct 30

Our picks from this week's model releases 🦙 → LFM3 by Liquid AI: small models for text, image and speech, with d1 decision models → Mistral Large 4: large, 1M context frontier → Flint by Recursive: mid-sized, built for coding agents Get them at llama.app/models

Post attachment
Morgan Funtowicz avatar
Morgan Funtowicz@MorganFunto·Oct 22

Your local AI just got more Windows-native 🦙 Llama for Windows v0.13.0 is out! 🖼️ Share images & text to Llama ⚡ Speculative decoding support 💻 See which models likely fit your hardware 🔗 llama:// deep links Give it a spin 👇 github.com/ggml-org/Llama-Wi…

Morgan Funtowicz avatar
Morgan Funtowicz@MorganFunto·Oct 12

Hey @Windows users, local AI is for you too🥳 Llama for Windows v0.12.0 is out! 🖥️ llama.cpp in your Windows 11 tray 🤗 Download models in one click 😎 Improved UX flow 👉 github.com/ggml-org/Llama-Wi…

Morgan Funtowicz avatar
Morgan Funtowicz@MorganFunto·Aug 25

Local AI has never been easier for Windows users. Open source. Truly local. Zero cloud. Llama for Windows runs llama.cpp models entirely on your hardware — download & manage models, chat from anywhere with Alt+Space Your prompts never leave your machine 🦙github.com/ggml-org/Llama-Wi…

Post attachment
Morgan Funtowicz avatar
Morgan Funtowicz@MorganFunto·15 May 2025

Serving @OpenAI Whisper, powered by @vllm_project, deployed on @huggingface Inference Endpoint, with astonishing @Gradio demo ... What else? 😎 AI community is all you need to create blazing fast whisper transcriptions 💥🚀 More in our blog 📰 huggingface.co/blog/fast-whi…

Morgan Funtowicz avatar
Morgan Funtowicz@MorganFunto·20 Jan 2025

Last but not least, vLLM and llama.cpp are currently being worked on! 🚀 Initial artifacts for testing will be made available for the community around end of Q1'25 🛠️ and follow up blog posts will pop as we start rolling them out.

Morgan Funtowicz avatar
Morgan Funtowicz@MorganFunto·20 Jan 2025

TensorRT-LLM, from our partner @nvidia, is the first backend we will be shipping. We are currently wrapping up the last parts. It'll bring cutting edge float8, quantization & sparsity support🤏 We'll soon publish a part 2 with more in-depth about this integration, stay tuned!

Morgan Funtowicz avatar
Morgan Funtowicz@MorganFunto·20 Jan 2025

Inference backends are abstracted away on Rust side as an interface every backend has to implement. It currently "only" describes the logic to schedule a request after validation by the API. In the coming weeks we will allow for customizations. github.com/huggingface/text-…

Morgan Funtowicz avatar
Morgan Funtowicz@MorganFunto·20 Jan 2025

We've made TGI more modular allowing to reuse components accross multiple inference backends. The serving REST API is now sharable with all the different backends, unifiying the UX and making it possible to focus on the core inference logic rather than transport layer.

Morgan Funtowicz avatar
Morgan Funtowicz@MorganFunto·20 Jan 2025

Anyone looking to deploy LLMs in production,🤗got something for you👀 @huggingface TGI is expanding its capabilities to bring support for different inference backends (vLLM, TensorRT-LLM, llama.cpp, etc. etc.) 🏋️ 🗞️ More info in our last blogpost huggingface.co/blog/tgi-mult…