Frequently Asked Questions
- Can I Access Oak from Sherlock and XStream?
- Can I Archive files to Oak from Sherlock?
- Can I access Oak from my desktop/laptop?
- Can I mount Oak on my own Linux cluster at Stanford?
Can I Access Oak from Sherlock and XStream?
Oak storage is available from all nodes on Sherlock and XStream under /oak. Like Sherlock’s /scratch, Oak is based on the Lustre parallel filesystem and is connected to Sherlock (1.0 and 2.0) and XStream through a low-latency Infiniband network.
You need an account on both Oak and Sherlock (or XStream) to access Oak from Sherlock (or XStream). The environment variable **$OAK** should be defined on Sherlock and XStream and contains the path to your Oak group directory. You may also use the full path starting with /oak as described above.
Can I Access Oak from SCG?
As of January 2020, all SCG labs (in /labs) and projects (in /projects) are hosted on Oak. When you get access to a lab or project in SCG, you also get access to the corresponding space in Oak. SCG is connected to Oak using a pair of 100 Gigabit Ethernet Lustre routers.
Groups who directly rent Oak storage will be able to access that Oak storage under /oak, so long as SCG users are in the corresponding workgroup. The groups used to manage SCG labs and projects space are separate from the workgroups used to manage directly-rented Oak space.
The $OAK environment variable is not set on SCG.
Can I Archive files to Oak from Sherlock?
The mpiFileUtils utilities are designed to copy files in parallel so you can quickly archives terabytes of data from scratch to Oak. The example below shows how to launch screen and launch a job that uses the dcp tool to copy a large directory:
[login_node ~]$ screen [login_node ~]$ module load system mpifileutils [login_node ~]$ srun -p dev -n 2 dcp $SCRATCH/dir $OAK/scratch_archive/
If you’re a Sherlock owner, you may want to replace
-p dev with
-p your_partition and increase the number of MPI tasks (
-n) to copy even faster!
Can I access Oak from my desktop/laptop?
Yes, please see the Oak Gateways page (Stanford only).
Can I mount Oak on my own Linux cluster at Stanford?
Yes, the SRCC team can deploy specific Oak NFSv4 gateway(s) to mount your Oak group/project directory on your Linux-based compute cluster (or desktop for specific applications). It is mandatory to use SUNet IDs on your cluster and Kerberos is required to access Oak. Such a service will incur additional costs to the PI.
What about automatic backups?
While the hardware configuration is quite robust, Oak does not provide local or remote data backup, and should be considered as a single copy. The SRCC is currently evaluating options for adding automatic remote backups (to cloud storage). Such a service will incur additional costs to the PI.