Aspiring Data Science

#polars

Воу!

"After working hard since our Polars Cloud announcement last February, we are pleased to officially launch Polars Cloud. Polars Cloud is now Generally Available on AWS. Beyond that, we also launched our novel Distributed Engine in Open Beta on Polars Cloud.

You can immediately get started at https://cloud.pola.rs/."

import polars_cloud as pc
import polars as pl
from datetime import date

with pc.ComputeContext(
    workspace="<my-workspace>",
    cpus=2,
    memory=8,
    cluster_size=8,
) as ctx:
    in_progress = (
        pl.scan_parquet("s3://polars-cloud-samples-us-east-2-prd/pdsh/sf100/lineitem/",
            storage_options={
                "aws_request_payer": "true",
            })
        .filter(pl.col("l_shipdate") <= date(1998, 9, 2))
        .group_by("l_returnflag", "l_linestatus")
        .agg(
            count_order=pl.len()
        )
        .remote(ctx)
        .distributed()
        .execute()
    )

    print(in_progress.await_result().head)

https://pola.rs/posts/polars-cloud-launch/

Polars Cloud

Polars Cloud - Run Polars at scale, from anywhere

Scale out Polars queries on a single machine or a cluster with the new distributed engine. Zero infrastructure management. No code changes.

www.tgoop.com/AspiringDataScience/2641

88 viewsAnatoly Alekseev, edited  Sep 4 at 18:52

#polars

Воу!

"After working hard since our Polars Cloud announcement last February, we are pleased to officially launch Polars Cloud. Polars Cloud is now Generally Available on AWS. Beyond that, we also launched our novel Distributed Engine in Open Beta on Polars Cloud.

You can immediately get started at https://cloud.pola.rs/."

import polars_cloud as pc
import polars as pl
from datetime import date

with pc.ComputeContext(
    workspace="<my-workspace>",
    cpus=2,
    memory=8,
    cluster_size=8,
) as ctx:
    in_progress = (
        pl.scan_parquet("s3://polars-cloud-samples-us-east-2-prd/pdsh/sf100/lineitem/",
            storage_options={
                "aws_request_payer": "true",
            })
        .filter(pl.col("l_shipdate") <= date(1998, 9, 2))
        .group_by("l_returnflag", "l_linestatus")
        .agg(
            count_order=pl.len()
        )
        .remote(ctx)
        .distributed()
        .execute()
    )

    print(in_progress.await_result().head)

BY Aspiring Data Science