High-performance computing is handled by the CSDE pool in Hyak, the UW-Wide High Performance Computing cluster. All Hyak access is authenticated by UW NetID and requires two-factor authentication via “DUO”. If you are a UW student, you can also join the UW HPC club and access the larger STF Hyak pool. (You can still connect from the CSDE Unix systems to Hyak if you use this allocation pool.)
To use the CSDE Hyak nodes (ikt only as of 11/2017), all of the following must be done:
- CSDE Computing admins must add your UW NetID to the group “u_hyak_csde” to enable access. Request Hyak access from csde_help@u.washington.edu.
- You need an Entrust eToken/keyfob issued by UW IT. Email help@uw.edu with “Hyak: eToken Request” as the subject, and include the mailing address to which they should send your eToken. More information is available here.
- You must add the Hyak server and the Lolo server (storage system) to your UW NetID self-services pages. To do this, click here, click “Computing Services,” check the the “Hyak Server” and “Lolo Server” boxes in the “Inactive Services” section, click “Subscribe >” at the bottom of the page, and click “Finish.” After subscribing, it may take up to an hour to be fully provisioned.
Connecting to Hyak
You’ll need to SSH into Hyak using your UW NetID username and password. It will ask you to approve using DUO 2-factor. Use ssh UWNetID@hyak.washington.edu, substituting your user name in place of “UWNetID.” Please use the /gscratch/csde area and create a subdirectory there named with your UW NetID. The lolo collaboration file system is located at /lolo/collaboration/hyak/csde/.
Please subscribe to this list for Hyak status updates.
The basic gist of the Hyak cluster is this: you will SSH into the head node of the Hyak system, where you can do minor work or ask the system for an “interactive node” you can ssh directly to and work away. The “intended” way to use the cluster is to make a batch submit script and submit your job to the scheduler. Once you set up your SSH key relationship, you won’t need to use your DUO 2FA login. In a standard Hyak node on the batch system, all software is a “module,” so you’ll have to load the “R” or “Microsoft R open” (Formerly RevolutionR) module. Take a look at “Software Development Tools” here.
Additional information is available below:
Getting files in and out of Hyak using a windows share
- Mount Lolo Collaboration via CIFS from your workstation or Terminal Server. The mountpoint is \\lolo.washington.edu\collaboration\hyak\csde.
- Drag and drop files into Lolo.
- In a shell on a Hyak login node, run cp /lolo/collaboration/hyak/csde/foo /gscratch/mydir.
- NOTE: In order to Access Lolo, you must enable the service in your self-service UW NetID page. Once this is done, you should have password access to the share as long as you are logged in to the CSDE terminal servers with your UW NetID.
Backfill Queue
Have a lot of jobs you’d like to run? Using the backfill queue offers the working potential of thousands of CPU cores.
NOTE: Each job in the backfill queue can only run for ~2 hours before being shut down, so divide your jobs up accordingly and/or use “checkpointing”!
If you’re running batches of serial jobs, read the following first:
If you’re running parallel jobs, read the following first:
More information is available here:
Student High-Performance Computing Club
If you are a currently enrolled UW student paying the Student Technology Fee, you should join the UW HPC Club. This will allow you to submit jobs to be run in the student node allocation as well as the CSDE node allocation. You can then run your job on any of 3 queues:
- STF queue: qsub -W group_list=hyak-stf runsim.sh
- CSDE queue: qsub -W group_list=hyak-stf runsim.sh
- Backfill queue: qsub -W group_list=bf runsim.sh
Hyak Node Retirement
Blades are deployed for three years. Blade deployments may only be extended if there is no demand for the slots they occupy. Because Hyak currently has lots of unoccupied slots, this has meant that nodes have continued to run beyond their 3-year minimum lifespan. As long as the nodes continue to operate, they have remained in the original owner’s queues.
Click here for Hyak utilization data and here for the inventory.
Citation in Publications
Please remember to acknowledge Hyak in any media featuring results that Hyak helped generate. When citing Hyak, please use the following language:
“This work was facilitated through the use of advanced computational, storage, and networking infrastructure provided by the Hyak supercomputer system at the University of Washington.”
When you cite Hyak, please let us know by emailing help@uw.edu with “Hyak” as the first word in the subject along with a citation we can use in the body of the message. Likewise, please let us know of successful funding proposals and research collaborations to which Hyak contributed.
Hyak is a CSDE resource, so remember to cite CSDE as well! Click here for more information on acknowledging support from CSDE.
To resume a job that’s currently running, reconnect to it by logging back in directly to its node. (The My Computer icon on your session’s desktop will indicate which cluster node you’re using; see the image to the right.) For example, if you disconnect from SIM3, your session would continue running there. To reenter and check the status of jobs, you need to use Remote Desktop Connection to connect back to the Sim node with your job.