Managing AWS ParallelCluster SSH users with OpenLDAP
A common request from AWS ParallelCluster users is to have the ability to deploy multiple POSIX user accounts. The wiki on the project GitHub page documents a simple mechanism for achieving this, and a previous blog post, “AWS ParallelCluster with AWS Directory Services Authentication,” documents how to integrate AWS ParallelCluster with AWS Directory Service. However, some customers might prefer a traditional directory service hosted locally to the cluster to allow for administration that is more convenient. This eliminates a requirement for stopping and restarting the head node, or needing to learn the details of how to manage Active Directory.
A multi-user AWS ParallelCluster environment is ideally suited for cases in which a team of scientists or engineers is closely collaborating and needs to share resources, such as data or account budget. Using a single set of file systems makes cluster management easier for the administrator, particularly if they are new to AWS. When time-to-solution and total spend must be carefully balanced, having jobs from multiple users run within a single scheduler is also helpful.
In this post, we describe how to enable the OpenLDAP directory service on the cluster’s head node. This enablement can create and synchronize a local collection of users.
The process documented here applies to clusters deployed using the CentOS 7 operating system, for AWS ParallelCluster version 2.8.1. Other operating systems can follow a similar process, but will require minor changes to the commands used to install the relevant packages (e.g., using
apt-get in lieu of
Preparing a multi-user cluster
After installing and configuring the AWS ParallelCluster command-line tool, set the configuration parameters
s3_read_write_resource) to allow a script to be executed on the cluster’s head and compute nodes at boot time. The following is an example of a minimal configuration file to deploy a Slurm-based cluster using C5n.18xlarge instances, which are well suited for scaling HPC workloads:
The single post-install script contains all installation and deployment steps associated with the OpenLDAP server: