Getting started

We allow login to the subMIT pool using ssh keys with authentication done through LDAP. Once, you have uploaded your ssh keys, you will also be given a home and work directory in which you can directly start working. This section will guide you on how to set up your ssh keys and upload them to the submit portal to allow login as well as describe the initial resources available to you.

How to get an account

If you already have a general MIT account then getting access to subMIT is easy. You only need to upload your ssh key to the submit portal.

You might be prompted for not being authorized to access the portal. Please, follow the instructions on the screen.

If you are not familiar with ssh keys, generating keys is a straightforward process.

Open the Terminal application. This can be done by clicking on Lanuchpad or hitting F4, and then typing “terminal” followed by the Return key.

To generate keys on your machine use the following command (chose the defaults when being prompted):

ssh-keygen -t ed25519

This should create both a private and a public key (id_ed25519, the private key, and id_ed25519.pub, the public key) in your .ssh directory.

In the output on your screen you will see a line specifying the path to your public key, such as “Your public key has been saved in /Users/[username]/.ssh/id_ed25519.pub”. To copy your public key, run the following command using the path to your public key

cat /Users/[username]/.ssh/id_ed25519.pub | pbcopy

Or simply open the public key file in your favorite text editor and highlight & copy the text.

Simply paste the contents of the public key (id_ed25519.pub) into the submit portal link above and you are ready. The private key is like your password and should never be exposed to anybody. Please do not paste this into the subMIT website; if you do you should re-create your keys by running the ssh-keygen command again.

We recommend that you use the standard name (as prompted by ssh-keygen) for the keys, as this will make the process easier. Some advanced users may want to create differently named keys within their .ssh directory, as they may wish to keep separate keys for separate machines. If you do this, please remember to either create the appropriate configuration within .ssh/config, or log in with ssh -i /path/to/identity/file.

Login and basic areas

Now that you can login you can simply ssh into the submit machines like below:

ssh <username>@submit.mit.edu

You should now be logged into one of our main submit server machines. By default you will get several areas (directories) automatically created for you which are public, which means other users can see your files. For more information regarding file permissions, please see Tutorial 0.

The starting quotas are 5 GB for the home directory and 50 GB for the work directory. If you need more space than the foreseen, please make your case to the help desk at <submit-help@mit.edu>.

# You will start from home which has a quota of 5 GB of space
/home/submit/<username>

# You will also get a workspace in which has a quota of 50 GB of space
/work/submit/<username>

# For larger files, you will get a data storage space with a quota of 1 TB of space
/ceph/submit/data/user/<first letter>/<username>

Note that storing and editing code located in /ceph is discouraged.

Common issues with keys

A common issue with keys is when you already have a key in your .ssh directory and ssh defaults to a different key than the one uploaded to the submit-portal. For example, GitHub commonly uses id_ed25519 keys which can interfere when you try to login.

There are a few ways to handle this issue. If you prefer, you can simply upload the other key to the submit-portal. If you would like to use the ed25519 key, log in with a command like ssh -i <path>/.ssh/id_ed25519 <username>@submit.mit.edu or create the appropriate configuration within .ssh/config like below.

Host submit
  HostName submit.mit.edu
  User <username>
  IdentitiesOnly=yes
  PreferredAuthentications publickey
  PasswordAuthentication no
  IdentityFile <path>/id_ed25519

One thing to note, is that if you have already tried this ssh command once and it broke, you will need to modify your known_hosts file in the .ssh directory. In order to fix this you will need to remove lines with submit in them from the known_hosts. Please note that it is safe to remove the lines from this file.

Connecting directly to a specific node

For most use cases, a user will not need to ssh into a specific node. Rather one will typically ssh into submit.mit.edu as described above and then offload significant computational work to HTCondor or Slurm as batch jobs or interactive sessions. There are, however, a few use cases where one legitimately desires access to a specific node. For these cases, please use ProxyJump. For instance, to ssh to submit37 from your laptop, type:

ssh -J <username>@submit.mit.edu <username>@submit37.mit.edu

Include any other flags you may ordinarily use such as -i. (Replace submit37 with the desired hostname). You can verify that you are indeed on the desired node by running the command: hostname

If you use a ssh/config file … (click here to show/hide more info)

If you use a .ssh/config file as described above, you can instead use the command

ssh -J submit <username>@submit37.mit.edu

or, if you plan to access this node frequently, you can add it as a separate entry to your .ssh/config file below your submit entry:

Host submit37
   HostName submit37.mit.edu
   User <username>
   IdentitiesOnly=yes
   PreferredAuthentications publickey
   PasswordAuthentication no
   IdentityFile <path>/id_ed25519
   ProxyJump submit

then you can simply access it from your laptop by typing

ssh submit37

Creating a personal webpage

In addition to the areas above, you have the ability to create a personal webpage in order to store and share your files. In order to create this site you will need a directory named public_html in your home directory:

mkdir $HOME/public_html

Once that is created, you can now access your personal webpage after inserting your username in http://submit.mit.edu/~<username>/. You can customize the appearance and content of your webpage for example by adding the index.php file inside the public_html directory or subdirectory to see previews of .png files.

Tips when coming from another cluster

Each cluster is a bit different and subMIT is no exception. Here are a few things about subMIT which may be different from a cluster you previously used:

  • In addition to the SLURM-managed subMIT nodes, subMIT is a login pool which also connects to other resources

  • subMIT does not provide software through module avail. Instead, we prefer users to install the specific toolbox they need for their workflow. We give you examples, for instance on how to setup conda to manage environments and packages for Python and more, or Mathematica. When you install software, make sure they are installed in your /work directory.

  • On subMIT, SLURM is not set to reserve entire nodes by default; SLURM will request the resources (cores & memory) you request for your job. On subMIT, it is best to think in units of cores, not nodes when making SLURM requests. The subMIT SLURM cluster contains several ‘standard’ nodes as well as high-density nodes with a large number of cores and memory on a single node. Given this heterogeneous nature, it is important to think how many cores your jobs need and request number of cores explicitly in your batch scripts. One high-density node can do the work of several standard nodes, and you will likely wait a long time (and end up with more cores than you need) if you request a full high-density node.

The rules for an account

Remember that these machines are shared. As such, do not max out resources on interactive jobs. As a guideline: If your job takes langer than 15 minutes it makes sense to dispatch it to a batch system. The machines in this login pool are connected to large computing clusters which are accessed by batch programs like HTCondor or Slurm.