←
Common Crawl Foundation1,520 posts
Common Crawl Foundation avatar

Common Crawl Foundation

@CommonCrawl

Common Crawl is a non-profit foundation dedicated to the Open Web.

◷ Joined February 2010▧ 100 Photos and videos
Metehan Yesilyurt avatar
⟳ Common Crawl Foundation retweeted
Metehan Yesilyurt@metehan777·Nov 10

CC Rank Checker is updated with Common Crawl's Oct '26 web graph. 25.1M domains, 16 releases since 2023, harmonic centrality + PageRank. New: compare up to 20 domains in one chart, PNG export, dark mode, free API. @peec_ai: HC 6.8M to 393K since mid '25. cc: @Burnszilla @glindahl @CommonCrawl The best part is ++

Post attachment
Common Crawl Foundation avatar
Common Crawl Foundation@CommonCrawl·Nov 5

Introducing warc2zip We are pleased to introduce warc2zip, a tool that allows anyone to turn WARC archives into simpler, zipped files that can be inspected with ordinary tools.

Post attachment
Phil Hoyeck avatar
⟳ Common Crawl Foundation retweeted
Phil Hoyeck@PAHoyeck·Oct 29

Just learned that Patrick Stewart has passed away. Such a sad loss. Here he is in one of the best things ever put on television, the Star Trek episode “The Measure of a Man.” I show it to my students in Philosophy and Science Fiction every year.

4:28

Patrick Stewart has sadly passed away at the age of 86.

Post attachment
Common Crawl Foundation avatar
Common Crawl Foundation@CommonCrawl·Oct 21

October 2026 Crawl Archive Now Available We are pleased to announce the release of the October 2026 crawl archive, consisting of 2.16 billion pages (or 371.19 TiB of uncompressed content)

Post attachment
Common Crawl Foundation avatar
Common Crawl Foundation@CommonCrawl·Oct 14

statmt.org/wmt26/pdf/2026.wm… We present the results of the WMT 2026 shared task of the Open Language Data Initiative (OLDI). Participants were invited to contribute to the massively multilingual open datasets supported by OLDI, FLORES+ and OLDI Seed, or to create new resources in line with OLDI’s ethos. We accepted ten submissions: five extending FLORES+, and five extending other open and massively-parallel datasets (BOUQuET, Seed, SMOL, and WMT24+). These contributions advance the coverage and quality of multilingual datasets, especially for underserved language varieties. All contributions are released under permissive open-source licenses.

Brian Wesley avatar
⟳ Common Crawl Foundation retweeted
Brian Wesley@BrianWesley0711·Oct 13

My bet: low-resource languages would lose at the filter. Quality classifiers trained mostly on English throw out good Icelandic pages as noise. Smarter crawl budgets won't help if the filter can't read the language.