Quick Dataframes for Huge Issues
I used to be just lately in a secret demo run by the Cuda and Polars staff. They handed me by way of a metallic detector, put a bag over my head, and drove me to a shack within the woods of rural France. They took my telephone, pockets, and passport to make sure I wouldn’t spill the beans earlier than lastly displaying off what they’ve been engaged on.
Or, that’s what it felt like. In actuality it was a zoom assembly the place they politely requested me to not say something till a specified time, however as a tech author the thriller had me feeling a bit of like James Bond.
On this article we’ll focus on the content material of that assembly: a brand new execution engine in Polars that allows GPU accelerated computation, permitting for interactive information manipulation of 100GB+ of knowledge. We’ll focus on what an information body is in polars, how GPU acceleration works with polars dataframes, and the way a lot of a lift to efficiency one can count on with the brand new CUDA powered execution engine.
Who’s this convenient for? Anybody who works with information and needs to work quicker.
How superior is that this submit? This submit comprises easy however cutting-edge information engineering ideas. It’s related to readers of all ranges.