


Software Engineer @VictoriaMetrics, building VictoriaLogs


I participated in the tech talk about object storage support in VictoriaLogs - youtu.be/NCniVg1jrJU?is=Kz9Q…

@func25 published new explainer video on how VictoriaLogs stores data on disk in order to provide high query performance over hundreds of terabytes of logs. Highly recommended to view!
How does logs database like VictoriaLogs store data on disk to support fast queries over terabytes of logs? youtube.com/live/Ok4D_80QCXk

Columnar storage, bloom filters, delta-of-delta/zstd/dictionary compression, indexes, etc in a logs database will all be explained in this video.
How does logs database like VictoriaLogs store data on disk to support fast queries over terabytes of logs? youtube.com/live/Ok4D_80QCXk

VictoriaLogs data source for Grafana gained support for logs' drilldown starting from v0.33.1! Check out the following article for details - victoriametrics.com/blog/fin…

RAM prices and storage prices are so high now, so it is a good moment to save a lot of costs by migrating from Elasticsearch to VictoriaLogs! aus.social/@phs/114583927679…

I spent 3 months benchmarking and digging into the internals of Loki, Elasticsearch, OpenSearch, ClickStack, SigNoz and VictoriaLogs, and condensed it into a 20-minute read. TL;DR: I realized who ingests fastest is only one side of the story. Every fast number had a price. I abused each of them with a flood of structured, Kubernetes-style logs on identical machines, until they choked. 1. Loki ingested the most: 139k lines/s. It indexes only stream labels and keeps lines as plain text, pushing the cost to query time. Any search or aggregation on a field, the kind every dashboard runs, is expensive and can take minutes: Loki has to decompress and scan every line in matching streams. 2. Elasticsearch finds a single word almost instantly, because it indexes every word of every line. The price: all that indexing is CPU-heavy, so it topped out at 48k lines/s. And it needed almost 5x the disk of VictoriaLogs. 3. SigNoz and ClickStack store logs in ClickHouse, a columnar database like VictoriaLogs. I expected them to be close. They only reached 10k and 46k lines/s. The bottleneck wasn't ClickHouse. It was the OpenTelemetry collector in front of it, which used more CPU than ClickHouse itself. 4. VictoriaLogs (I work on it) ingested 120k lines/s with the least memory and disk. It stores each field in its own column and reads only the columns a query needs, so every test query returned within seconds. The catch: it shines on structured logs, where columns help most. On plain-text logs (from older apps or network devices), it still works, but loses most of its edge.
@cuongleqq published data ingestion benchmark results for open-source databases for logs: VictoriaLogs, Loki, Elasticsearch, OpenSearch, SigNoz and ClickStack (the last two are based on ClickHouse). The benchmark measures CPU usage, memory usage, disk space usage and disk IO usage for different ingestion rates of Kubernetes-like logs. It also measures performance for production-like queries. Hint: VictoriaLogs isn't the fastest one for data ingestion. victoriametrics.com/blog/str…

How the empty interface in Go works under the hood youtu.be/_BxdvkLLnDc?si=Xlot…

@cuongleqq published data ingestion benchmark results for open-source databases for logs: VictoriaLogs, Loki, Elasticsearch, OpenSearch, SigNoz and ClickStack (the last two are based on ClickHouse). The benchmark measures CPU usage, memory usage, disk space usage and disk IO usage for different ingestion rates of Kubernetes-like logs. It also measures performance for production-like queries. Hint: VictoriaLogs isn't the fastest one for data ingestion. victoriametrics.com/blog/str…

VictoriaMetrics v1.55.0 highlights

Excellent animations AND tech content! #golang
Escape analysis in Go: stack or heap? youtube.com/watch?v=BanG_wLn… Tbh, every second is worth it, even if you don't know Go. I wish someone had taught me this when I was learning Go.

Logchef gets a mention! If you’re looking for a really good log explore tool paired with #VictoriaLogs, def check it out!
What’s Changed in VictoriaLogs v1.53.0?

In 2017, I started learning Go, mostly from curiosity. After working some years for Mattermost, I learnt enough to start sharing that knowledge with the community. Today I’m honored to share that I’ve been recognized as a @GoogleDevExpert in Go. This wouldn’t have happened without a lot of people involved in my own process, from the extremely welcoming Go community, to my companies, @kaleidosnet, @Mattermost, @ona_hq, and now @VictoriaMetrics, that has been always supportive and helping me to contribute back to the community. The GDE program is about sharing knowledge, so, there is not going to change much from my side, except that I'll have even more support to keep sharing with you what I learn in Go. #golang #GoogleDeveloperExpert #GDE #Go

VictoriaMetrics now has a new account just for the technical side. If you're interested in database techniques, observability, and products explained visually, give it a follow:
What’s Changed in VictoriaLogs v1.53.0?

Very good visualisation on how stack and heap memory works in Go
Escape analysis in Go: stack or heap? youtube.com/watch?v=BanG_wLn… Tbh, every second is worth it, even if you don't know Go. I wish someone had taught me this when I was learning Go.

My article about SIMD and Go is out! In this one I explain how one CPU instruction can work on 8 numbers at once, and how you can now use that from plain Go with the experimental simd/archsimd and simd packages. My intention with this post is to explain in simple terms how SIMD works at CPU level, to give you the mental model on how to use it, and then give you enough information about how you use it from Go to start working with it. You can read it here: internals-for-interns.com/po… #Go #SIMD