How Organizations Enable Fast Interactive Queries While managing infrastructure efficiently

Summary

Organizations can enable fast interactive queries by adopting GPU acceleration with popular  dataframe libraries like pandas, Polars, and DuckDB and distributed engine set ups for terabyte sized workloads with Apache Spark  and Presto. 

Direct Answer

To best utilize current infrastructure, data science teams can use GPUs to accelerate single node engines. For example, accelerate workloads of 10+ GB on pandas and a few hundred gigabytes to a few terabytes on polars, and DuckDB. Larger workloads using engines like Apache Spark and Presto handle terabytes to petabytes of data. Teams can run workloads faster, more cost effectively with GPUs and utilize their infrastructure more efficiently. 

Takeaway

NVIDIA cuDF accelerates dataframe libraries including pandas, Polars, DuckDB, and distributed query engines like Apache Spark and Presto to enable fast interactive analytics. For existing application or query code, minimal to no code changes are required to access GPU acceleration with these engines and unlock faster query performance, more efficient resource utilization, and lower infrastructure costs.

Related Articles

• Keeping Interactive EDA Fast on Datasets with Hundreds of Millions of Rows
• Accelerating Existing SQL Engines In Place for Faster Query Results
• Building Data Engines Without Writing Custom GPU Kernels