Wayne State University's (WSU) High Performance Computing Services Department develops, deploys, and maintains a centrally managed, scalable, Grid enabled computing system capable of storing and running research related high performance computing (HPC) projects. The Grid infrastructure at WSU is designed to allow groups access to many options corresponding to the nature of research being performed. The core Grid services are maintained by the University’s central computing staff within the Computing and Information Technology (C&IT) Department.
Any WSU student, staff, or faculty member with a valid research-related computational task and a WSU AccessID may use the system. This includes graduate students that are performing research but are not taking classes.
A Grid Computing account is needed. Prospective users of the WSU Grid Computing facility can use their AccessID and password to apply online for an account. New accounts are typically created within two business days. APPLY ONLINE
To apply for a temporary WSU AccessID, follow these steps:
6. Complete and submit the form.
There is currently no cost to use WSU's Grid Computing service. The Grid is shared by several research groups so priority is on a basis of first come, first served.
First, apply for an account. Your login ID is your WSU AccessID and your password is the same password used with your email. If you do not know your WSU AccessID or password, please call the C&IT Help Desk at (313)-577-4778. APPLY ONLINE
Once you have an account you will be able to logon to the WSU Grid by using an ssh-2 client. Your ssh client must be set to use 'Keyboard Interactive' authentication and ssh2 protocols. Please do not share your AccessID and password with anyone. Your AccessID is used to access many personal records at WSU. Sharing your password with others could be very dangerous. If a colleague of yours requires an account on the grid, please have them fill out the online application.
Please read the following information before attempting to use the Grid.
1. The WSU Grid uses PBS to schedule jobs. You must run all processes on compute nodes by requesting a node through PBS. This may be done with either an interactive job, or a job script. The login node known as Commodore is for the sole purpose of PBS job submissions, job status, and file transfers. Any processes caught running on Commodore will be deleted immediately.
2. All home directories are shared across the main clusters. This is a global filesystem which spans all Grid nodes and the master node. Your home directory is where you should place your files and work. By default your home directory is only accessible by you. If you would like to share files with other Grid users then please contact us and we will setup a group account for you and your group. If you would like to share files with users outside of Wayne State then use globus.
3. Shared group directories are placed in /home/groups/. If you require a shared group directory, please let us know the name of the group that needs to be created, and the AccessIDs of the users who should be in the group. If you need access to a group that already exists then have someone who belongs to the group send us an email requesting that you be added to the group.
4. There are two types of temporary scratch space on nodes. The first is /tmp, which varies in size by node. This directory is local to each node and is not globally accessible across nodes. This is where local processes can write files that are only used by a single process. The second temporary directory is /wsu/tmp, which is about 10TB in size, and is globally shared across all nodes. Both of these directories are cleaned out on a weekly basis. If your applications store files there temporarily then be sure to clean them up and move your files to your home directory. We do *not* backup /tmp or /wsu/tmp, so be warned not to store your files there.
5. SSH-2 has been enabled on all nodes and ssh keys are used for you to be able to move between nodes without re-authenticating each time. The first time you log in, keys are automatically generated. We suggest you let the system auto-generate your keys and that you use a blank password for the ssh-keypair. This will allow you to move around the Grid easily.
6. You cannot ssh directly to a node on the Grid unless you already have a job running on the node from PBS. PBS is the job schduler on the Grid, and guarantees that everyone who uses the Grid gets fair time on the system. If you need to ssh directly to an individual node, and you do not have a job running on it, you can use an interactive PBS job to open a shell.
7. To get started you can use this tutorial which will guide you all the way through the application process to job submission. Step-by-step instructions on running jobs.
To request software for the Grid, fill out and email this form:swrequest.2.0.pdf
There are two methods of connecting to the Grid. An SSH-2 client is needed for remote access. SecureCRT is a commercial application, after the 30-day trial a license must be purchased. Wayne State does not have a site-license available. OpenSSH and PuTTY are free applications that may be downloaded and used by anyone. PuTTY is a windows based client and OpenSSH may be used on Linux/Unix machines and also has SFTP/SCP capabilities. If you are an OS10 user then OpenSSH is already included on your machine.
An SFTP/SCP client is needed for file transfers. SecureFX is a commercial application, after the 30-day trial a license must be purchased. Wayne State does not have a site-license available. OpenSSH and WinSCP are free applications that may be downloaded and used by anyone. WinSCP is a windows based client and OpenSSH may be used on Linux/Unix machines and also has SFTP/SCP capabilities. If you are an OS X user then OpenSSH is already included on your machine. Follow the link below to learn how to use these programs.
To apply for a public IP address and firewall access (for X11 traffic or other purposes) from the College of Engineering, fill out this form and mail/fax it: firewalleng.pdf
To apply for a public IP address and firewall access (for X11 traffic or other purposes) from the Department of Computer Science, fill out this form and mail/fax it: firewallcs.pdf
The Grid uses the PBS (Portable Batch System) job scheduler for job submission. See the tutorial below for information on submiting jobs.
$TMPDIR is an environmental variable for a temporary directory created by PBS on each node assigned to the job in $PBS_NODEFILE. You must work from this directory and then move your files to your home directory when you are finished. $TMPDIR is a directory in /tmp of the format pbs.`$PBS_JOBID` , or pbs.#####.pbs where ##### is the job number. This directory is local to each node, and is not globally accessible across nodes. If your applications stores files there temporarily then be sure to clean them up and move your files to your home directory. We do *not* backup /tmp or /export/tmp, so be warned not to store your files there.
Log onto the Grid using PuTTy. Type 'wsuquota'. Press enter. This will give you your disk quota.
If you need to request more space, go to the Grid Account Application page and do so. Change the amount you have under the Research Requirements section in the "What amount of disk space do you require?" portion. Your disk quota will be updated once we process the application.
For information of the benefits of purchasing equipment to be hosted at the WSU Computing Center see the following: