The Python community has lost one of its early contributors. 🕊️
Guido van Rossum, Python's creator, announced the passing of Tim Peters at the age of 74.
Peters was Python's second real user, creator of Timsort, author of the Zen of Python (PEP 20), and a member of the Python Software Foundation board for 13 years.
His contributions will live on in the Python ecosystem.
Rest in peace, Tim Peters. ❤️
The early design decisions for the Categorical type were under strain because of our streaming engine. Every data chunk carried its own mapping between the categories and their underlying physical values, forcing constant re-encoding. The global StringCache we built to solve it caused lock contention and wasn't designed for a distributed architecture.
The new Categories object, released in 1.31, solves this, and gives you:
• Control over the physical type (UInt8/16/32)
• Named categories with namespaces
• Parallel updates without locks
• Automatic garbage collection
When you know the categories up front you can use Enums. They're faster because of their immutability and allow you to define the sorting order of values.
The StringCache is now a no-op, but the code will keep working how it used to (with global Categories). You can also migrate by replacing it with explicit Categories where needed.
The result is a Categoricals data type that works well on the streaming engine without performance degradation, and is compatible with a distributed architecture.
Read the full deep dive: pola.rs/posts/categoricals-r…
pandas 3 has been released and marks the most significant evolution of #pandas in over ten years.
No more `copy()` everywhere, and no more `lambda` gymnastics.
Want examples? Read this hands-on article with the main changes: datapythonista.me/blog/whats…
#pandas 3 is faster, and more intuitive. I just published an article with practical examples explaining the main improvements: datapythonista.me/blog/whats…
We're happy to announce the release of #pandas 2.3.0. You can install it with `pip install pandas` or `conda install -c conda-forge pandas`. Thanks to all contributors and sponsors who made this release possible! The release notes can be found at: pandas.pydata.org/docs/whats…
Today we are launching the first open Crash Course training sessions with a limited time discount. These instructor-led sessions are open to everyone looking to get up and running with Polars.
Find a date and sign up via our Academy: pola.rs/academy/