Posts tagged HPC
Co-locating a Jupyter Server and Dask Scheduler
- Sep 13, 2019
If you want, you can have Dask set up a Jupyter notebook server for you, co-located with the Dask scheduler. There are many ways to do this, but this blog post lists two.
Dask on HPC: a case study
- Aug 28, 2019
Dask is deployed on traditional HPC machines with increasing frequency. In the past week I’ve personally helped four different groups get set up. This is a surprisingly individual process, because every HPC machine has its own idiosyncrasies. Each machine uses a job scheduler like SLURM/PBS/SGE/LSF/…, a network file system, and fast interconnect, but each of those sub-systems have slightly different policies on a machine-by-machine basis, which is where things get tricky.
Dask-jobqueue
- Oct 08, 2018
This work was done in collaboration with Matthew Rocklin (Anaconda), Jim Edwards (NCAR), Guillaume Eynard-Bontemps (CNES), and Loïc Estève (INRIA), and is supported, in part, by the US National Science Foundation Earth Cube program. The dask-jobqueue package is a spinoff of the Pangeo Project. This blogpost was previously published here