Workspaces¶

Workspaces are temporary scratch directories on the IBM Storage Scale (GPFS) work file system, designed for data that only needs to persist between jobs for a limited time — such as intermediate results or job inputs and outputs. The work file system has no backup, so workspaces should only be used for data that can be recreated, e.g. by running the same batch jobs again.

Initially workspaces have a maximum lifetime of 60 days. You can extend the lifetime 3 times for another 60 days, but you should do this near the end of the lifetime since the new lifetime starts when you execute the extension command.

Quotas¶

For your account (user ID) there is a quota limit for capacity and number of inodes (number of files and directories). The limits apply for all of your workspaces and for your expired workspaces (as long as they are not yet completely removed). In order to check the current usage and limits see general information on quotas.

Create a workspace¶

To create a workspace you need to state the name of your workspace and lifetime in days (maximum 60):

ws_allocate myspace 30

Example output:

Info: creating workspace.
/hfs2/work/workspace/scratch/USERNAME-myspace
remaining extensions  : 3
remaining time in days: 30

List all your workspaces¶

ws_list

This returns:

• Workspace ID
• Workspace location
• Creation date, remaining time and expiration date
• Available extensions

Find workspace location¶

Workspace location/path can be prompted for any workspace ID using ws_find:

ws_find myspace

Returns:

/hfs2/work/workspace/scratch/USERNAME-myspace

Extend lifetime of your workspace¶

Any workspace's lifetime can only be extended three times. There are two equivalent commands:

ws_extend myspace 40          # extends myspace by 40 days from now
ws_allocate -x myspace 40     # same effect

Delete a workspace¶

ws_release myspace   # Manually erase your workspace myspace

Reminder for workspace deletion¶

By default you will get an email about an expiring workspace 7 days before it expires. You can adapt this time using the -r option of ws_allocate.

You can also send yourself a calendar entry:

ws_send_ical <workspace> <email>

Restoring expired workspaces¶

At expiration time your workspace will be moved to a special, hidden directory. Expired workspaces are currently kept for 30 days. During this time you can still restore your data into a valid workspace. Released workspaces are only kept until the next night.

To restore an expired workspace, as first step create a new workspace (e.g. with name my_restored) and then:

# List expired workspaces
ws_restore -l

# Restore into an existing active workspace
ws_restore <full_name_of_expired_workspace> my_restored

Linking workspaces in Home¶

The command ws_register DIR will create and manage links to all personal workspaces within the directory DIR:

ws_register DIR

• The directory DIR will be created if necessary.
• Links to all personal workspaces will be managed: links to available workspaces are created if not already present, and links to released or expired workspaces are removed.

Data sharing¶

Workspaces are the recommended location for sharing data with other users or project members, since they are located on the globally visible work file system and are not subject to per-user home directory restrictions.

Access control lists (ACLs) allow fine-grained control over who can read or write files in a workspace. ACLs extend the standard Unix permission model and are supported on the IBM Storage Scale (GPFS) file systems used on HoreKa 2.

Grant access to another user¶

To give another user read and execute access to a workspace directory:

setfacl -R -m u:<username>:rX <path-to-workspace>

To additionally allow write access:

setfacl -R -m u:<username>:rwX <path-to-workspace>

Grant access to a group¶

To grant access to all members of a project group:

setfacl -R -m g:<groupname>:rX <path-to-workspace>

Check current ACLs¶

getfacl <path-to-workspace>

Remove an ACL entry¶

setfacl -R -x u:<username> <path-to-workspace>