I created MLXr because i couldn't find a lightweight OpenAI compatible server for hashtag#MLX compatible models on Apple hashtag#applesilicon.
✅ OpenAI Compatible endpoint
✅ Qwen compatible translations
✅ Tool call conversion
✅ Model management from Hugging Face
✅ Client configuration
github.com/mchenetz/mlxr#ai#mlx#inferencing#gpt#llm#openai#qwen#qwen36
Built and open-sourced SPLAI: a distributed AI execution fabric that treats AI workloads like Kubernetes treats containers.
What makes it different:
✅ Distributed-first execution across CPU/GPU/edge workers
✅ DAG-based planning + policy-aware scheduling
✅ OpenAI-compatible API layer for drop-in integration
✅ Centralized model prefetch to workers (including Hugging Face)
✅ Optional auto-install of missing models on workers at request time
✅ End-to-end observability, retries, leases, and audit trails
✅ Runs local, on VMs, or on Kubernetes with Helm
This is for teams that want production AI execution without being locked into centralized GPU stacks. If you’re building serious AI pipelines and want infra-aware orchestration, I’d love feedback from operators, platform engineers, and AI teams.
github.com/mchenetz/SPLAI#AIInfrastructure#MLOps#PlatformEngineering#Kubernetes#OpenSource#LLMOps#DistributedSystems#DevOps#SPLAI
Ollama 0.2 is here! Concurrency is now enabled by default.
ollama.com/download
This unlocks 2 major features:
Parallel requests
Ollama can now serve multiple requests at the same time, using only a little bit of additional memory for each request. This enables use cases such as:
- Handling multiple chat sessions at the same time
- Hosting code completion LLMs for your team
- Processing different parts of a document simultaneously
- Running multiple agents at the same time
Run multiple models
Ollama now supports loading different models at the same time. This improves several use cases:
- Retrieval Augmented Generation (RAG): both the embedding and text completion models can be loaded into memory simultaneously.
- Agents: multiple versions of an agent can now run simultaneously
- Running large and small models side-by-side
Models are automatically loaded and unloaded based on requests and how much GPU memory is available.
Ever wonder what the "Temperature" setting does in Generative AI? It can significantly impact your output. Check out the chart below to understand what the values mean. Essentially, it determines how creative the output should be. #AI#GenerativeAI#Technology#Innovation
I had so much fun chatting with Carlos Santana on #CloudUnfiltered! Carlos has such an amazing view on where the industry is going and always keeps his eye on the future. Check out the latest episode to here this really engaging conversation! @OutshiftByCisco#EKS@AWS#devops#platformengineering#AI
🌟 I’m thrilled to share my recent podcast interview on Cloud Unfiltered with @mchenetz
In this episode, we explored:
- The need for standardized platforms to streamline cloud-native development.
- How Platform Engineering bridges the gap between feature engineers and infrastructure teams.
- The innovative CNOE project, aiming to simplify the creation and management of internal developer platforms.
It’s a deep dive into the practical aspects and strategic importance of Platform Engineering, especially for organizations leveraging AWS and EKS.
📺 Watch to the full interview on youtube: youtube.com/watch?v=N_TcGEIh…
🎙️ Listen to the interview on your podcast player: cloudunfiltered.substack.com…
Meet the new iPad Pro: the thinnest product we’ve ever created, the most advanced display we’ve ever produced, with the incredible power of the M4 chip. Just imagine all the things it’ll be used to create.
It's a little rough so far but i commited the initial buld of my #AI tools for #audio and #podcasting to github. Feel free to beta test and let me know how it works. It is a work in progress.
github.com/mchenetz/podtools
Created a new #opensource#AI tool for converting your videos into audio, transcribing them and then auto generating whatever you want from #OpenAI. #podcast@OpenAI
This was created out of frustration when i would have to do all these things manually. Now i have a batch script that runs that does the following with this tool:
1. Converts video to audio
2. Transcribes
3. Creates Youtube Description
File Monitor: TBD
Web Interface: TBD
Docker Image: TBD
Self Hosted Transcribe: TBD
More to come! Released in the next week or so.