Unlocking Cyclone: A Comprehensive Guide to HPC Workflows

1. Purpose of the Tutorial Series

The "Unlocking Cyclone: A Comprehensive Guide to HPC Workflows" tutorial series is designed to equip users with the knowledge and skills necessary to access, navigate, and optimize the use of the Cyclone HPC system. Whether users are researchers running computational simulations, Python developers scaling their data analysis workflows, or scientists utilizing pre-installed software packages like GROMACS or OpenFOAM, this series provides practical guidance tailored to diverse needs. The tutorials respond to recurring challenges identified during training events and aim to demystify complex processes in HPC usage. By addressing fundamental concepts, workflow setups, and advanced optimization techniques, the series ensures users can effectively harness Cyclone’s capabilities for their research and applications.

2. Goals

  1. Enhance Accessibility: Simplify access and navigation processes to help users familiarize themselves with Cyclone’s architecture and tools.
  2. Promote Best Practices: Foster a culture of fair and efficient resource use, data management, and compliance with HPC policies.
  3. Build Proficiency: Develop user competency in job scheduling, software module management, and workflow optimization.
  4. Enable Scalability: Support users in scaling their applications using Cyclone’s compute power, whether through parallel programming or package optimization.
  5. Facilitate Troubleshooting: Provide solutions to common issues, reducing barriers to productive HPC use.

3. Anticipated Outcomes

By implementing the "Unlocking Cyclone: A Comprehensive Guide to HPC Workflows" series, the NCC aims to:
  • Improve user confidence and competence in HPC usage.
  • Enhance the productivity of Cyclone’s user base.
  • Establish Cyclone as a model of user-centric HPC systems, supporting researchers in tackling complex computational challenges.

4. Tutorials

The tutorials are structured to guide users through various aspects of using the Cyclone system:

  1. Introduction to HPC Systems: This tutorial introduces Cyclone’s architecture, including compute nodes, file systems, modules and the SLURM job scheduler. It sets the foundation for understanding the system’s components and resource management, ensuring participants are prepared for more advanced sessions.
  2. Accessing and Navigating Cyclone: Users will learn to securely access Cyclone via SSH, transfer data using tools like scp and rsync, and navigate the file system with basic Linux commands. The focus is on practical skills for accessing and managing data across Cyclone’s file systems.
  3. Setting Up and Using Development Tools: This session covers setting up tools like VS Code and MobaXTerm for remote development, managing software environments with modules, and extending functionality using tools like Conda or Python’s venv.
  4. Scaling Python Workloads on HPC: This tutorial addresses how to run Python scripts on Cyclone using SLURM, focusing on scaling Python workflows using examples with PyTorch across CPUs and GPUs, and troubleshooting common issues.
  5. Compiling and Running C/C++ Code on Cyclone with SLURM: Participants will learn to compile and run C/C++ programs, covering serial, multi-threaded, GPU-accelerated, and distributed programs using SLURM.
  6. Interactive Computing on Cyclone with Jupyter Notebooks: This tutorial demonstrates how to set up and use Jupyter Notebooks on Cyclone’s compute nodes, emphasizing resource allocation optimization, SSH tunneling, and best practices for interactive computing.