SeaweedFS is a scalable distributed file system optimized for exabyte small files, supports FUSE, S3 API, Hadoop, k8s, replication, Cloud Drive, S3 gateway.
43.9% reduction in S3 object size with SeaweedFS Enterprise!
Custom Zstd dictionaries improve compression for small JSON, XML, and log files by reusing patterns.
* 43.9% size reduction vs plain Zstd
* Minimal latency impact
* No application changes
seaweedfs.com/blog/zstd-cust…
SeaweedFS development keeps accelerating!
569 PRs merged in October, a new monthly record and 5.7x more than a year ago.
3,334 PRs merged and 48 releases over the past 12 months.
Thanks to all the contributors making this possible!
#SeaweedFS#OpenSource
SeaweedFS 4.53: 3.15× faster S3 object listing!
Real-world benchmark with 1.7M+ objects spread across 6.5M+ directories, with deeply nested paths.
Before: 81 minutes
After: 26 minutes
Great to see the improvement confirmed by a user!
github.com/seaweedfs/seaweed…
S3 over RDMA in SeaweedFS Enterprise is ~5× faster than HTTP for large-object GETs, reaching 11.4 GB/s on a single 100 GbE link.
S3 auth stays intact; data moves directly from storage servers to client memory via RDMA.
seaweedfs.com/blog/s3-over-r…
One table. Multiple query engines.
SeaweedFS Table Buckets expose standard Iceberg REST + S3, so the query engine is your choice.
Comparing:
* Trino for distributed SQL + writes
* DuckDB for embedded analytics
seaweedfs.com/blog/databow-t…#SeaweedFS#ApacheIceberg#DuckDB#Trino
celld + alchemy + seaweedfs (s3)
using this heavy on local machines for alll sorts of crazy shit but particularly love pi-durable workflows it's unlocked github.com/joelhooks/alchemy…
For S3, I created a cheap VM with github.com/seaweedfs/seaweed… and use that for testing and development (orbs and local) in places where mocking/faking doesn't fit.
In PostgreSQL, a 1,000,000-row table starts at 152 MB in the heap. After the pgEdge ColdFront archiver runs, 958,056 of those rows sit as Parquet files in object storage and the hot heap is down to 6624 kB. The table name and the SQL stay the same.
ColdFront is open source (PostgreSQL License) data tiering for PostgreSQL 16, 17, and 18. It moves older partitions to Apache Iceberg on S3 and keeps that data queryable and writable.
In the new walkthrough, an UPDATE on an archived row lands directly in object storage, with no rehydration and no restore job. A lake-native table stores 0 bytes in Postgres and takes plain INSERT, UPDATE, and DELETE. Concurrent writers across two Postgres nodes with 0 conflicts, 0 Lakekeeper 409s, and 12 Iceberg snapshots for 12 writes.
There's no narration and every step is explained on screen, so pause anywhere to read the SQL. The demos run on a local Docker stack with SeaweedFS standing in for a cloud bucket.
Watch the walkthrough: 🎬 hubs.ly/Q04z8QhR0
ColdFront on GitHub: 🔗 hubs.ly/Q04z8QkL0#postgres#postgresql#apacheiceberg#opensource#dataengineering#database#datalake#parquet#s3#docker#pgedge#coldfront
It's time, we are switching our copy everywhere, and updating our tests, and recommendation for on prem s3 to @SeaweedFS, @Minio is history.
We are starting today with the tests in the repo.
github.com/Basekick-Labs/arc
It's, using celld and SeaweedFS for storage.
And it's a live multiplayer reaction game: a light flashes at the same instant for everyone on the page, timed by a Durable Object's own alarm. First click wins.
SeaweedFS Table Buckets can tier each part independently, automatically based on data temperature.
⚡ Metadata → replicated SSD
📦 Parquet data → HDD
🧊 Cold data → erasure coding / cloud
⚡ Parquet footers → cached back on SSD
seaweedfs.com/blog/volume-ti…
Why choose between Iceberg and Lance? You can analyze with Iceberg, and serve with Lance point lookups and vector search, with DuckDB joins them together.
SeaweedFS can host both table formats on the same object store, with built-in catalogs for each.
seaweedfs.com/blog/iceberg-l…