←
Phuong Le1,111 posts
Phuong Le avatar

Phuong Le

@func25

Software Engineer @VictoriaMetrics, building VictoriaLogs

◷ Joined August 2014▧ 277 Photos and videos
Aliaksandr Valialkin avatar
⟳ Phuong Le retweeted
Aliaksandr Valialkin@valyala·Nov 1

RAM prices and storage prices are so high now, so it is a good moment to save a lot of costs by migrating from Elasticsearch to VictoriaLogs! aus.social/@phs/114583927679…

Cuong Le avatar
⟳ Phuong Le retweeted
Cuong Le@cuongleqq·Nov 3

I spent 3 months benchmarking and digging into the internals of Loki, Elasticsearch, OpenSearch, ClickStack, SigNoz and VictoriaLogs, and condensed it into a 20-minute read. TL;DR: I realized who ingests fastest is only one side of the story. Every fast number had a price. I abused each of them with a flood of structured, Kubernetes-style logs on identical machines, until they choked. 1. Loki ingested the most: 139k lines/s. It indexes only stream labels and keeps lines as plain text, pushing the cost to query time. Any search or aggregation on a field, the kind every dashboard runs, is expensive and can take minutes: Loki has to decompress and scan every line in matching streams. 2. Elasticsearch finds a single word almost instantly, because it indexes every word of every line. The price: all that indexing is CPU-heavy, so it topped out at 48k lines/s. And it needed almost 5x the disk of VictoriaLogs. 3. SigNoz and ClickStack store logs in ClickHouse, a columnar database like VictoriaLogs. I expected them to be close. They only reached 10k and 46k lines/s. The bottleneck wasn't ClickHouse. It was the OpenTelemetry collector in front of it, which used more CPU than ClickHouse itself. 4. VictoriaLogs (I work on it) ingested 120k lines/s with the least memory and disk. It stores each field in its own column and reads only the columns a query needs, so every test query returned within seconds. The catch: it shines on structured logs, where columns help most. On plain-text logs (from older apps or network devices), it still works, but loses most of its edge.

@cuongleqq published data ingestion benchmark results for open-source databases for logs: VictoriaLogs, Loki, Elasticsearch, OpenSearch, SigNoz and ClickStack (the last two are based on ClickHouse). The benchmark measures CPU usage, memory usage, disk space usage and disk IO usage for different ingestion rates of Kubernetes-like logs. It also measures performance for production-like queries. Hint: VictoriaLogs isn't the fastest one for data ingestion. victoriametrics.com/blog/str…

Aliaksandr Valialkin avatar
⟳ Phuong Le retweeted
Aliaksandr Valialkin@valyala·Oct 29

@cuongleqq published data ingestion benchmark results for open-source databases for logs: VictoriaLogs, Loki, Elasticsearch, OpenSearch, SigNoz and ClickStack (the last two are based on ClickHouse). The benchmark measures CPU usage, memory usage, disk space usage and disk IO usage for different ingestion rates of Kubernetes-like logs. It also measures performance for production-like queries. Hint: VictoriaLogs isn't the fastest one for data ingestion. victoriametrics.com/blog/str…

Jesús Espino avatar
⟳ Phuong Le retweeted
Jesús Espino@jespinog·Oct 22

In 2017, I started learning Go, mostly from curiosity. After working some years for Mattermost, I learnt enough to start sharing that knowledge with the community. Today I’m honored to share that I’ve been recognized as a @GoogleDevExpert in Go. This wouldn’t have happened without a lot of people involved in my own process, from the extremely welcoming Go community, to my companies, @kaleidosnet, @Mattermost, @ona_hq, and now @VictoriaMetrics, that has been always supportive and helping me to contribute back to the community. The GDE program is about sharing knowledge, so, there is not going to change much from my side, except that I'll have even more support to keep sharing with you what I learn in Go. #golang #GoogleDeveloperExpert #GDE #Go

Post attachment
Phuong Le avatar
Phuong Le@func25·Oct 22

VictoriaMetrics now has a new account just for the technical side. If you're interested in database techniques, observability, and products explained visually, give it a follow:

What’s Changed in VictoriaLogs v1.53.0?

7:01
Aliaksandr Valialkin avatar
⟳ Phuong Le retweeted
Aliaksandr Valialkin@valyala·Oct 13

Very good visualisation on how stack and heap memory works in Go

Escape analysis in Go: stack or heap? youtube.com/watch?v=BanG_wLn… Tbh, every second is worth it, even if you don't know Go. I wish someone had taught me this when I was learning Go.

0:29
Jesús Espino avatar
⟳ Phuong Le retweeted
Jesús Espino@jespinog·Oct 19

My article about SIMD and Go is out! In this one I explain how one CPU instruction can work on 8 numbers at once, and how you can now use that from plain Go with the experimental simd/archsimd and simd packages. My intention with this post is to explain in simple terms how SIMD works at CPU level, to give you the mental model on how to use it, and then give you enough information about how you use it from Go to start working with it. You can read it here: internals-for-interns.com/po… #Go #SIMD