Posts by Matthew Rocklin
Dask Survey 2021, early anecdotes
- Jun 18, 2021
The annual Dask user survey is under way and currently accepting responses at dask.org/survey.
Stability of the Dask library
- May 21, 2021
Dask is moving fast these days. Sometimes we break things as a result.
Dask User Summit 2021
- Mar 03, 2021
Dask is organizing a user summit in mid-May. This will be a remote event focused on bringing together developers and users of Dask and the distributed PyData stack in different domains.
Co-locating a Jupyter Server and Dask Scheduler
- Sep 13, 2019
If you want, you can have Dask set up a Jupyter notebook server for you, co-located with the Dask scheduler. There are many ways to do this, but this blog post lists two.
Dask on HPC: a case study
- Aug 28, 2019
Dask is deployed on traditional HPC machines with increasing frequency. In the past week I’ve personally helped four different groups get set up. This is a surprisingly individual process, because every HPC machine has its own idiosyncrasies. Each machine uses a job scheduler like SLURM/PBS/SGE/LSF/…, a network file system, and fast interconnect, but each of those sub-systems have slightly different policies on a machine-by-machine basis, which is where things get tricky.
Dask and ITK for large scale image analysis
- Aug 09, 2019
Document headings start at H2, not H1 [myst.header]
Python and GPUs: A Status Update
- Jun 19, 2019
This blogpost was delivered in talk form at the recent PASC 2019 conference. Slides for that talk are here.
Experiments in High Performance Networking with UCX and DGX
- Jun 09, 2019
This post is about experimental and rapidly changing software. Code examples in this post should not be relied upon to work in the future.
Composing Dask Array with Numba Stencils
- Apr 09, 2019
In this post we explore four array computing technologies, and how they work together to achieve powerful results.
Building GPU Groupby-Aggregations for Dask
- Mar 04, 2019
Document headings start at H2, not H1 [myst.header]
Running Dask and MPI programs together
- Jan 31, 2019
Document headings start at H2, not H1 [myst.header]
Single-Node Multi-GPU Dataframe Joins
- Jan 29, 2019
Document headings start at H2, not H1 [myst.header]
Dask, Pandas, and GPUs: first steps
- Jan 13, 2019
Document headings start at H2, not H1 [myst.header]
GPU Dask Arrays, first steps
- Jan 03, 2019
The following code creates and manipulates 2 TB of randomly generated data.