←
Polars490 posts
Polars avatar

Polars

@DataPolars

The compute engine for open data.

◷ Joined July 2022▧ 85 Photos and videos
Matt Harrison avatar
⟳ Polars retweeted
Matt Harrison@__mharrison__·Nov 12

Polars is typically one of the fastest ways to manipulate tabular data (according to popular benchmarks). If you touch data, it makes sense to learn it.

0:46
Polars avatar
Polars@DataPolars·Nov 12

Since last year we continuously improved the performance of our distributed engine by over 3x! We benchmarked it again and we are 6x faster than OSS Spark and ~30% faster than Databricks Photon. We're going to continue in this trajectory. So if you want to run fast SQL and DataFrame workloads in one platform. Reach out! pola.rs/posts/polars-nanook-…

Post attachment
Polars avatar
Polars@DataPolars·Nov 10

Query profiles in Polars Nanook can now be shared. Send a profile to a colleague when you want a second opinion on your query's performance. They see the plan, the stages and the metrics, without needing an account themselves. Blog post: pola.rs/posts/share-query-pr… Query Profile: nanook.pola.rs/portal/share/…

Polars avatar
Polars@DataPolars·Nov 5

Introducing Polars Nanook. Polars Nanook is our next generation data platform for high performance scalable workloads on cloud and on-premise for either humans or agents. Aside from the rebrand, we've been able to ship very big features: - SQL support - Huge leap in coverage distributed algorithms - GPU support - Faster than any Spark accelerator we can find pola.rs/posts/polars-cloud-b…

Matt Harrison avatar
⟳ Polars retweeted
Matt Harrison@__mharrison__·Nov 4

Working on the next version of Effective Polars. The book is more than twice as long as the second edition and now includes detailed content on extending Polars in Rust. I'm excited to get this out.

Post attachment
Polars avatar
Polars@DataPolars·Oct 27

Polars 2.1 is released! Three weeks after 2.0 we're shipping a big one. A few highlights: - Spill to disk for group-by and window functions - Recursive CTE's in SQL - Reading and writing encrypted Parquet - Better memory tracking for out-of-core - And over 90 performance improvements merged Link below

Post attachment
Polars avatar
Polars@DataPolars·Oct 15

We ran Polars 2.0 SQL, DuckDB and DataFusion on an @nvidia Vera CPU and compared it to an Intel Xeon 6, m8id.metal-96xl. On the NVIDIA Vera, Polars ran 2 to 2.5x faster compared to Xeon, and 2 to 3x faster than the runner-up. pola.rs/posts/nvidia_vera_be…

Post attachment
Polars avatar
Polars@DataPolars·Oct 6

We just released Polars 2.0. It removed many of our legacy decisions makes the streaming engine our default and promotes SQL to a first class citizen within Polars. It comes with initial out-of-core (spill to disk) support, a new Map data type and a lot of performance improvements. In fact, we think Polars is now one of the fastest analytical SQL engines on a single node. See benchmarks in the post: pola.rs/posts/release-polars…

Post attachment
Polars avatar
Polars@DataPolars·Sep 21

Polars 2.0.0rc2 has been released. We expect this to be the last release candidate and that Polars 2.0 will go live next week. Install now by running `pip install --pre polars` Link to the full release guide: docs.pola.rs/releases/upgrad…

Post attachment
Polars avatar
Polars@DataPolars·Sep 11

Come join us at the second day of Pydata Amsterdam! Learn about our new query profiler and let an agent optimize your queries!

Post attachment
Polars avatar
Polars@DataPolars·Sep 2

We are happy to announce the first pre-release of Polars 2.0. Polars 2.0 will bring the streaming engine as a default and improves overall strictness of Polars. Give it a spin and share your thoughts. We expect the final Polars 2.0 release in 3-4 weeks. pola.rs/posts/announcing-pol…

Post attachment
Polars avatar
Polars@DataPolars·Aug 28

The Polars GPU engine now runs on a new streaming backend, built with NVIDIA on their RapidsMPF library. • Queries are no longer bound by GPU memory. Data flows through the query graph in morsels that spill to host memory when device memory gets tight. • The same query scales from one GPU to many. Construct a RayEngine, pass it to .collect(), and there is no cluster to configure and no change to the query itself. Full post: pola.rs/posts/gpu-streaming-…

Post attachment