←
SeaweedFS365 posts
SeaweedFS avatar

SeaweedFS

@SeaweedFS

SeaweedFS is a scalable distributed file system optimized for exabyte small files, supports FUSE, S3 API, Hadoop, k8s, replication, Cloud Drive, S3 gateway.

◷ Joined March 2020▧ 15 Photos and videos
SeaweedFS avatar
SeaweedFS@SeaweedFS·Nov 16

43.9% reduction in S3 object size with SeaweedFS Enterprise! Custom Zstd dictionaries improve compression for small JSON, XML, and log files by reusing patterns. * 43.9% size reduction vs plain Zstd * Minimal latency impact * No application changes seaweedfs.com/blog/zstd-cust…

SeaweedFS avatar
SeaweedFS@SeaweedFS·Nov 11

SeaweedFS development keeps accelerating! 569 PRs merged in October, a new monthly record and 5.7x more than a year ago. 3,334 PRs merged and 48 releases over the past 12 months. Thanks to all the contributors making this possible! #SeaweedFS #OpenSource

Post attachment
SeaweedFS avatar
SeaweedFS@SeaweedFS·Nov 9

🚀 SeaweedFS supports Amazon S3 Vectors API! • AWS CLI & boto3 • Vector similarity + metadata filters • Lance storage + LanceDB interoperability S3 objects, tables, and vectors in one storage system! github.com/seaweedfs/seaweed… cc @lancedb @changhiskhan @eddyxu @xuanwo

SeaweedFS avatar
SeaweedFS@SeaweedFS·Nov 9

SeaweedFS 4.53: 3.15× faster S3 object listing! Real-world benchmark with 1.7M+ objects spread across 6.5M+ directories, with deeply nested paths. Before: 81 minutes After: 26 minutes Great to see the improvement confirmed by a user! github.com/seaweedfs/seaweed…

SeaweedFS avatar
SeaweedFS@SeaweedFS·Oct 26

S3 over RDMA in SeaweedFS Enterprise is ~5× faster than HTTP for large-object GETs, reaching 11.4 GB/s on a single 100 GbE link. S3 auth stays intact; data moves directly from storage servers to client memory via RDMA. seaweedfs.com/blog/s3-over-r…

SeaweedFS avatar
SeaweedFS@SeaweedFS·Oct 19

One table. Multiple query engines. SeaweedFS Table Buckets expose standard Iceberg REST + S3, so the query engine is your choice. Comparing: * Trino for distributed SQL + writes * DuckDB for embedded analytics seaweedfs.com/blog/databow-t… #SeaweedFS #ApacheIceberg #DuckDB #Trino

joel ⛈️ avatar
⟳ SeaweedFS retweeted
joel ⛈️@joelhooks·Oct 16

celld + alchemy + seaweedfs (s3) using this heavy on local machines for alll sorts of crazy shit but particularly love pi-durable workflows it's unlocked github.com/joelhooks/alchemy…

Scott Watermasysk avatar
⟳ SeaweedFS retweeted
Scott Watermasysk@scottw·Oct 8

For S3, I created a cheap VM with github.com/seaweedfs/seaweed… and use that for testing and development (orbs and local) in places where mocking/faking doesn't fit.

pgEdge avatar
⟳ SeaweedFS retweeted
pgEdge@pgEdgeInc·Oct 5

In PostgreSQL, a 1,000,000-row table starts at 152 MB in the heap. After the pgEdge ColdFront archiver runs, 958,056 of those rows sit as Parquet files in object storage and the hot heap is down to 6624 kB. The table name and the SQL stay the same. ColdFront is open source (PostgreSQL License) data tiering for PostgreSQL 16, 17, and 18. It moves older partitions to Apache Iceberg on S3 and keeps that data queryable and writable. In the new walkthrough, an UPDATE on an archived row lands directly in object storage, with no rehydration and no restore job. A lake-native table stores 0 bytes in Postgres and takes plain INSERT, UPDATE, and DELETE. Concurrent writers across two Postgres nodes with 0 conflicts, 0 Lakekeeper 409s, and 12 Iceberg snapshots for 12 writes. There's no narration and every step is explained on screen, so pause anywhere to read the SQL. The demos run on a local Docker stack with SeaweedFS standing in for a cloud bucket. Watch the walkthrough: 🎬 hubs.ly/Q04z8QhR0 ColdFront on GitHub: 🔗 hubs.ly/Q04z8QkL0 #postgres #postgresql #apacheiceberg #opensource #dataengineering #database #datalake #parquet #s3 #docker #pgedge #coldfront

Post attachment
Ignacio Van Droogenbroeck avatar
⟳ SeaweedFS retweeted
Ignacio Van Droogenbroeck@hectorivand·Sep 25

It's time, we are switching our copy everywhere, and updating our tests, and recommendation for on prem s3 to @SeaweedFS, @Minio is history. We are starting today with the tests in the repo. github.com/Basekick-Labs/arc

Maximilien Comlan avatar
⟳ SeaweedFS retweeted
Maximilien Comlan@_maxcomlan·Sep 19

It's, using celld and SeaweedFS for storage. And it's a live multiplayer reaction game: a light flashes at the same instant for everyone on the page, timed by a Durable Object's own alarm. First click wins.

SeaweedFS avatar
SeaweedFS@SeaweedFS·Sep 8

SeaweedFS Table Buckets can tier each part independently, automatically based on data temperature. ⚡ Metadata → replicated SSD 📦 Parquet data → HDD 🧊 Cold data → erasure coding / cloud ⚡ Parquet footers → cached back on SSD seaweedfs.com/blog/volume-ti…

SeaweedFS avatar
SeaweedFS@SeaweedFS·Sep 3

Why choose between Iceberg and Lance? You can analyze with Iceberg, and serve with Lance point lookups and vector search, with DuckDB joins them together. SeaweedFS can host both table formats on the same object store, with built-in catalogs for each. seaweedfs.com/blog/iceberg-l…