←
Oren Melamed3,642 posts
Oren Melamed avatar

Oren Melamed

@OrenMe

GitHub Star ⭐️ | Microsoft MVP | AI Transformation @ AT&T | Humans + agents. AI-native engineering. Rethinking how organizations build.

◷ Joined April 2009▧ 494 Photos and videos
Oren Melamed avatar
Oren Melamed@OrenMe·Nov 17

It is now clear to me more than ever that the hardest working team in @github is the one behind @GHchangelog !!! Thanks for keeping us up to date ❤️🎉 And really amazing day - Adaptive CCR - Repo insights dashboard - Cross repo review - MAI 2 and MAI 2 flash - Decision api - fusion benchmark!!! - Evals framework - Browser use And more…. Go read the changelog!!!! github.blog/changelog/

Oren Melamed avatar
Oren Melamed@OrenMe·Nov 17

Crazy day for @Microsoft and @GitHubCopilot Both MAI and also decision model! And stats are out of this world: MAI Code 2 gives you 98% of the performance of GPT 6.1 Sol at 40% of the cost, and 96% of Claude Opus 5.5’s performance at just 5% of the cost.1 MAI Code 2 Flash offers comparable performance to GPT-6 Luna and Claude Haiku 5.5 at 65% and 20% of the cost

MAI Code 2 is now available in GitHub Copilot at a 50% discount through May 18th, 2027. Give it a try and let us know what you think :) microsoft.ai/news/introducin…

Post attachment
Oren Melamed avatar
Oren Melamed@OrenMe·Nov 17

We’ve been measuring AI adoption. We’re starting to measure something more interesting: which agent customizations actually make a difference. @github just added new @GitHubCopilot Code Review metrics that give us a glimpse into this. The usage metrics API can now break down review feedback by the customization associated with it: - Custom instructions - Agent skills - MCP servers - No customization It also tracks how many review comments developers actually addressed, and whether PRs merged without a subsequent human change request. This creates an interesting new feedback loop for us. We’ve been investing in building reusable agent capabilities: skills, instructions, MCPs, custom agents and plugins. But knowing that developers installed or invoked them isn’t enough. We need to understand whether they actually improve the work. I see three layers emerging: - Distribution: APM and registries to package, version and distribute agent capabilities. - Observability: OTel to understand how agents execute, which tools they use and where resources go. - Outcomes: Metrics that help us understand whether those capabilities actually contribute to better results. There are important caveats. An addressed comment isn’t necessarily a correct finding, and attribution isn’t proof of causality. But this is a much better starting point than counting installations or invocations. Agent customizations are becoming software assets. We need to manage them, observe them and continuously evaluate their value, just like the rest of our engineering stack. We’re seeing a move from collecting prompts and building agents to engineering, operating and measuring an ecosystem of agent capabilities. We’re going to need more than a prompt library. How are you measuring the effectiveness of agent skills, instructions and MCPs in your organization? Are you still measuring adoption, or have you started measuring outcomes?

Oren Melamed avatar
Oren Melamed@OrenMe·Nov 16

Organizational context is a knowledge management problem. Operational context is a software supply chain problem. We've been talking a lot about context in the age of agents. But I think we're overloading the term, and missing an important distinction. There are two fundamentally different types of context, and they need different systems to manage them. 1. Organizational context: What the organization knows This is the knowledge accumulated across an organization, coming from different sources and evolving at different rates. It can be technical knowledge, business domain knowledge, legal knowledge, architecture, processes, decisions, and relationships between them. This is where semantic search, knowledge graphs, ontologies, GraphRAG and context services come into play. It needs to understand the state of knowledge, its sources, freshness and relationships. But it doesn't necessarily need Git-style versioning or branching. It's closer to a knowledge database. You need snapshots and recovery capabilities, but not necessarily a commit history as the primary interaction model. 2. Operational context: How agents should operate This is a different beast. It's the knowledge encoded into the standards and primitives that agent harnesses consume as first-class citizens: Skills, AGENTS.md, instructions, MCPs and other agent configurations. It can exist at repository, team, application, domain or even company level. And here, we're not just managing knowledge. We're managing executable behavior and operational standards. That brings an entirely different set of engineering requirements: - Supply chain: Clear lineage, provenance, ownership and governance. Changes should go through a registry, not propagate automatically to every consuming repository. - Security: Scanning and validation before distribution. - Evaluation: Every agent primitive should have an evaluation suite, with synthetic and real-world truth sets. A registry can even help build those truth sets by tracking where and how primitives are used. - Progressive delivery: Treat new versions like software releases. Start with a small group, expand to 10%, then 50%, then 100%. A/B testing and controlled rollouts. - Observability: Understand the impact on actual agent behavior and production workflows. This requires telemetry at the span and trace level, not just counting installations or downloads. These two worlds obviously share some infrastructure and can complement each other. But they have different lifecycles, distribution models, governance requirements and, importantly, different ways to measure success. One is about making organizational knowledge discoverable and useful. The other is about packaging, validating, distributing and continuously improving the standards agents operate with. We shouldn't confuse a knowledge retrieval system with an operational context supply chain. I'm currently spending most of my time building the second, while being deeply involved in building the first. I think both are becoming foundational infrastructure for agentic organizations. Where does APM, which I love to write bout, fit into this? I see APM (Agent Package Manager) as a potential reference implementation for the operational context supply chain. And we're starting to see the ecosystem move in this direction. @github recently added native Dependabot support for APM. It can now track agent primitive dependencies declared in apm.yml, open update PRs and regenerate the lock file. This is an important signal: Skills, prompts and instructions are starting to receive the same dependency management treatment we've had for software packages for years. But dependency management is just one piece. APM provides many of the building blocks: - Versioning and pinning of agent primitives. - Policy enforcement through policy engines. - Controlled distribution through registries. - CI integration and drift detection. - Out-of-the-box security scanning, with SARIF output. But the interesting part is what happens when you combine APM with OpenTelemetry. APM provides the lifecycle and distribution control plane. OpenTelemetry provides runtime evidence. Together, and with evaluation infrastructure and a feedback pipeline, they can form the foundation for a complete lifecycle: Build → Validate → Publish → Distribute → Observe → Evaluate → Improve. Think about the feedback loop: - Security scanners identify vulnerabilities before publication. - Policy engines enforce organizational standards. - CI ensures consuming repositories stay aligned with approved versions. - Progressive rollouts allow new versions to be tested with increasingly larger groups of consumers. - OTel spans and traces capture how agents actually use these primitives at runtime. - Evaluation infrastructure measures their impact against synthetic and real-world truth sets. - Runtime findings and evaluation results feed back into the registry, informing the next version. This closes the loop between what we publish, what we deploy, what agents actually do and what we should improve next. It's a shift from distributing markdown files between developers to operating a governed, observable and continuously improving supply chain for agent behavior. APM isn't just about installing agent primitives. It's about operating their entire lifecycle. Where to start If you're building context infrastructure for an agentic organization, I'd start by separating these two concerns. For organizational context, invest in making knowledge discoverable, connected, trustworthy and continuously updated. Think knowledge graphs, semantic retrieval, ontologies and context services. For operational context, start treating agent primitives as software dependencies rather than configuration files. Give them ownership, versioning, provenance, security policies, evaluation suites and controlled distribution. A practical starting point is to introduce a registry, pin dependencies, automate security scanning and drift detection, and establish a feedback loop using runtime telemetry and evaluations. APM is one way to start building this foundation today, while GitHub's growing support for APM, Dependabot and Copilot demonstrates how naturally this can fit into existing developer workflows. You don't need to build the entire platform upfront. Start with a few shared Skills or instructions, establish a release process, measure their adoption and behavior, and gradually introduce progressive delivery and automated evaluation. The important thing is to establish the right separation early. Don't build a knowledge platform and expect it to become your agent operations platform. And don't treat your agent supply chain as just another way to distribute knowledge. Both are foundational infrastructure for agentic organizations, and both deserve to be engineered accordingly.

Post attachment
Oren Melamed avatar
Oren Melamed@OrenMe·Nov 16

Jam packed event last night at @Arpeely for the ATG community🎉 Always a pleasure hosting amazing speakers and meeting with our community What we heard - sub second RTB ads and how to utilize AI for better decisioning, model memory(yes, yes, not agent, the real kv cache, deep stuff) and uber personal AI assistants future

Post attachmentPost attachmentPost attachment
0:07

@Arpeely with AI Transformation Guild Community. Full House 🚀🚀🚀 @OrenMe @ShohamGilad #meetups #israel #israeltech

Oren Melamed avatar
Oren Melamed@OrenMe·Nov 3

First level support for APM in Dependabot. Small changelog. Big signal. Over the last few months, I’ve been writing about APM, experimenting with it, and adopting it as part of how I think we should manage agentic primitives. Agentic primitives are becoming supply-chain dependencies, and the operational tooling we built for software dependencies is starting to catch up. JFrog added APM support. Renovate added APM dependency resolution. GitHub Agentic Workflows already uses APM for dependency management. Now GitHub Dependabot can keep APM dependencies up to date. Different tools, same direction. Skills, agents, MCPs and other agentic primitives need the same things we expect from software packages: Versioning. Dependency resolution. Lockfiles. Automated updates. Provenance. CI enforcement. This is what makes the ecosystem operational at enterprise scale, beyond demos and copy-pasted instructions. My bet is that APM is an early piece of a much bigger shift: the software supply chain is expanding to include the things that define agent behavior. And once these primitives become dependencies, managing their lifecycle stops being optional. We’re going to need the equivalent of the entire package management, security and governance stack, not another directory of prompts. This is one of those bets I’m happy to see the ecosystem validate. github.blog/changelog/2026-1…

Oren Melamed avatar
Oren Melamed@OrenMe·Nov 2

The best model is no model! This is amazing to see, IMHO. It reinforces something I keep coming back to: focus on the task, not on gaming the harness or micro-optimizing model selection. That doesn’t scale. @GitHub and other model/harness providers have way more data to optimize the model-harness combination than any individual or even most companies could ever collect. My recommendation from today onwards: mandate Auto, or at least make it the default. Disable older model options and let the vendor wars drive costs down while the models keep getting better. Stop trying to outsmart the model picker. Focus on getting the job done.

We often get asked what the best model is in GitHub Copilot. Our team spends an enormous amount of time optimizing the GitHub Copilot harness for the best, most token-efficient results. For the first time, we're publishing our data on what models perform best in Copilot: benchmarks.github.com

Oren Melamed avatar
Oren Melamed@OrenMe·Nov 1

Totally agree. My wife packs everything into one bag. Then I need to find one thing, and I’m digging through a pile of unrelated stuff. And don’t get me started when a drink spills in there. 💀 The same applies to agents. You want an orchestrator, chief of staff, whatever you want to call it, as your entry point. Work at the level of intent, ideas, and prose, not the codebase. Give it tools and access to a context repo or service with documented ADRs and the bigger picture. Ideate there, map out the plan, then let it spin up sessions across the multiple repos that make up your app, locally or remotely. I work this way with @GitHubCopilot in the desktop app, and the instrumentation is top-notch. One bag for everything is a mess. You want someone who knows what’s in the bag, what you’re looking for, and which pocket to open. 😅

I absolutely hate the idea that people think monorepos are great for agents. I still hate mono repos. When will people learn. The best way to scale is to avoid scaling.

Oren Melamed avatar
Oren Melamed@OrenMe·Oct 28

And to wrap till now - fusion is not offered with 30% discount so use it!!!

Oren Melamed avatar
Oren Melamed@OrenMe·Oct 28

And this one!!! I’ve been waiting for this Projects!!! And who else to present it then the amazing @burkeholland I’ll need a full post on this alone but if you ever worked on an app with multiple repos and wanted to set copilot on top of it with correct context and harness tools - this is it And it is available today!

Post attachmentPost attachmentPost attachment
Oren Melamed avatar
Oren Melamed@OrenMe·Oct 28

Dreaming! Copilot handles what it needs to remember and what it needs to forget!!! And the UI is so native feeling

Post attachment
Oren Melamed avatar
Oren Melamed@OrenMe·Oct 28

I wrote about fusion earlier this month Now out! This is the next era of model router but it is much more - remote model router is not enough and you need a system that operates both on client and server to really help you understand the task in hand And… it is halving prices!

Post attachmentPost attachment
Oren Melamed avatar
Oren Melamed@OrenMe·Oct 28

And @GitHubCopilot is in every part of the tool chain From signal to value If you ever adopted AI and your engineers said they work faster but your ROI didn’t move? So it is because u need a platform and not just a coding agent

Post attachment
Oren Melamed avatar
Oren Melamed@OrenMe·Oct 28

If you wonder how AI looks like from the system side!!! Let that sink!!!! This is crazy growth

Post attachmentPost attachmentPost attachment