←
Elastic Dev614 posts
Elastic Dev avatar

Elastic Dev

@elastic_devs

Where developers learn, build, and share. Elastic Dev is your source for hands-on demos, cheat sheets, explainers and more. Check out: @elastic

◷ Joined October 2025▧ 325 Photos and videos
Elastic Dev avatar
Elastic Dev@elastic_devs·20 Oct 2025

👋 Welcome to Elastic Dev — your new go-to for hands-on demos, cheat sheets, concept explainers and more. Get the latest on search, data, and GenAI straight from the source. Expect: - Practical demos and how-tos - Posts and technical deep dives from our Labs - Open source updates & more For anyone who loves building, learning and experimenting: let's get started. Hello, world.

0:29
Elastic Dev avatar
Elastic Dev@elastic_devs·Sep 9

The hard parts of hybrid retrieval, already done. Elasticsearch Vector Database is a new serverless offering where expert-level tuning is the default: - bfloat16 storage: half the disk footprint before quantization even starts - BBQ: up to 32x vector compression, 95% less memory - Auto-calibration re-tunes quantization on every merge as your data drifts - Filtered vector search at up to 8x higher throughput than OpenSearch - Jina AI embeddings and reranking on managed GPU inference, or bring your own models You bring documents and queries. We handle the embeddings, tuning, and infrastructure. Full breakdown, including the semantic_text quickstart and what ships in vectorDB index mode: go.es.io/3Vk5Wcm

1:23
Elastic avatar
⟳ Elastic Dev retweeted
Elastic@elastic·Sep 4

Do you know why BM25 is still useful when we have vector search? BM25 stops rewarding the 20th "cat" in a document. Plain TF-IDF lets term frequency grow without limit. A keyword-stuffed page outranks a good one. Two parameters fix that. k1 caps repetition. The term frequency factor climbs toward an asymptote of k1 + 1, so the 20th hit adds almost nothing over the 5th. b handles length. |D| / avgdl compares a document against the collection average and penalises the long ones.

Post attachment
Elastic avatar
⟳ Elastic Dev retweeted
Elastic@elastic·Sep 3

Before: look up the field name, check the metric type, pick the right aggregation, build the chart. Per metric. Now: type TS metrics-* in Kibana Discover. Discover appends METRICS_INFO to your query behind the scenes. One row per metric: name, type, unit, dimensions, and the data stream it lives in.. Gauges get AVG. Counters get SUM(RATE()). Histograms get PERCENTILE(p). You never choose. Break the charts down by any dimension your data exposes.. Switch data streams and dimensions that don't exist there get cleared automatically. The inventory is your metrics, charted correctly, from one line of ES|QL.

Post attachment
Elastic avatar
⟳ Elastic Dev retweeted
Elastic@elastic·Sep 2

Write load hotspots can hide inside a single weighted score. Elasticsearch sums 4 shard metrics into 1 score per node. High write load and low shard count cancel out. The node looks balanced, but the hotspot stays put. Elasticsearch Serverless stopped using a single score. Heap, write load, index shard spread: each gets its own decider. A shard only moves for a specific, named reason. One workload cut shard relocations by 50% at the same write throughput. Fleet-wide, data node OOMs dropped to near zero after the heap decider rolled out.. Full technical deep dive, including the production graphs: go.es.io/4ird54v

Post attachment
Elastic avatar
⟳ Elastic Dev retweeted
Elastic@elastic·Sep 1

1 question to an AI agent: 24 spans, 10 model calls, 160K input tokens. Elastic 9.5 records all of it out of the box. There's no collector to set up. Every Agent Builder run lands as OTel traces in your own cluster, down to each ES|QL query the agent wrote and which index it ran against. Half the model calls went to a cheaper model. You'd never know from the answer text. It's only in the trace. One thing traces don't hold by default: content. Prompts and responses stay off unless you flip the privacy toggles. And they never tell you who approved the action. That record you write yourself.

Post attachment
Elastic avatar
⟳ Elastic Dev retweeted
Elastic@elastic·Aug 28

Save these copy-paste ES|QL queries for your next Kubernetes incident diagnosis. Blog with all 9 in the reply.

Post attachment
Elastic avatar
⟳ Elastic Dev retweeted
Elastic@elastic·Aug 26

Your ES|QL query dies with Unknown column [field] because an alias got repointed and the new backing index dropped a field. Or you spot a field in your documents that never made it into the mapping. Until now, using it meant a reindex. Hours of it. In Elasticsearch 9.5, SET unmapped_fields="LOAD" reads the missing field straight from _source instead. "NULLIFY" fills the field with nulls. Queries keep working through mapping changes, without touching the data.

Post attachment
Elastic avatar
⟳ Elastic Dev retweeted
Elastic@elastic·Aug 25

470 of the 500 slowest requests share 1 log pattern. You'd never find it reading traces one at a time. At 10 traces the check is tedious. At 500 it doesn't happen. ES|QL subqueries are now on Serverless and in tech preview in 9.5 WHERE trace_id IN (subquery) The trace query runs inside Elasticsearch. The outer query pulls every log line those requests wrote, across every service, and groups them into log patterns. In this investigation it surfaced a lock wait timeout 2 hops downstream from the alert, in a service nobody was watching. 1 slow trace gives you a theory. 470 of 500 tells you which team to page.

Post attachment
Elastic avatar
⟳ Elastic Dev retweeted
Elastic@elastic·Aug 24

Elasticsearch 9.5 cuts timestamp storage by 92%. 1.03 GB down to 79 MB on a high-cardinality benchmark of 2.26 billion data points. The old time series codec applied one fixed encoding to every numeric field. Fine for timestamps and counters. Useless for floats: a 0.01 change in a CPU gauge is a jump of trillions in integer space. ES95 selects the encoding per field instead, based on what the mapping already declares: - SplitDelta: splits blocks at series boundaries, so one jump stops taxing the whole block - ALP: recovers the decimal structure in floats, compresses them as integers Per-field results: gauges down 19% to 74%, counters 20% to 30%, total doc values down 33.6%. Dimension fields like host.ip aren't covered yet; that work is ongoing. Applies to newly written data on upgrade. Existing segments keep their encoding, no migration required.

Post attachment
Elastic avatar
⟳ Elastic Dev retweeted
Elastic@elastic·Aug 21

"It feels slow" is the worst ticket a self-hosted LLM generates. vLLM starts, serves, and reports success whether it's configured brilliantly or wastefully. Nothing tells you which. But the answer is already sitting on /metrics: latency split by phase, cache hit rates, batch occupancy, completion outcomes. Almost nobody scrapes it. On a single A10G, 6 metric checks turned "it feels slow" into a verdict: TTFT: 51 ms against a 300 ms p95 target Queue time: 0.01 ms Decode: 97% of total latency Prefix cache: 32% of prefill work skipped entirely The server was over-provisioned, not under. The slowness lived somewhere in front of it: the app, the gateway, or the prompt. Tuning an LLM server is reading telemetry and reasoning about saturation. SREs have been doing that for 20 years. Full walkthrough with the ES|QL queries, the DCGM setup, and all 6 metric checks on an A10G: go.es.io/3U9DTfm

Post attachment
Elastic avatar
⟳ Elastic Dev retweeted
Elastic@elastic·Aug 21

Data stream lifecycle finally reaches the frozen tier. frozen_after sits next to data_retention, indices get snapshotted to object storage and stay queryable.

Elastic avatar
⟳ Elastic Dev retweeted
Elastic@elastic·Aug 19

Code is the only documentation that's never outdated. Our field teams get asked things like: does App A v1.2.3 work with App B v9.8.7 on Kubernetes? Docs rarely cover that. The code always does. So one of our field engineers built Sourcerer. It searches a billion lines of our code across every repo and version we support. Every answer cites the exact file and line. Built on Elasticsearch and Agent Builder. Open source, Apache 2.0. Full benchmarks and solution design: go.es.io/4xMJpmq

Post attachment
Elastic avatar
⟳ Elastic Dev retweeted
Elastic@elastic·Aug 18

Running search, analytics, metrics, logs, and vector retrieval in five systems costs more than five licenses. Here's what one platform looks like in practice.

Elastic avatar
⟳ Elastic Dev retweeted
Elastic@elastic·Aug 17

🧵 jina-embeddings-v5-omni represents the next evolution of embedding models. Literally. The vision pipeline uses the Qwen3.5 vision encoder, which itself is based on SigLIP2. Audio comes through the Qwen2.5-Omni audio encoder, built atop Whisper-large-3. Text runs through jina-embeddings-v5-text. So how do we get all of these different modalities, all running through different encoders, into one embedding space? The architecture that makes it work is called GELATO.

Post attachment