Summer and Early Fall Training
The SSCC’s summer training is well underway, but we still have upcoming classes in Python, web scraping, regression and regression diagnostics taught in both Stata and R, and running big research computing jobs. Visit the training page for details and to register.
Then in late August, just before the fall semester begins, we will teach another round of Stata, R, and Python classes. All are welcome, but these are especially valuable to new graduate students. If you are in touch with incoming graduate students, please make sure they know about these classes before it’s too late for them to arrange their schedules so that they can attend!
Account Renewal Deadline
Accounts that were not renewed during the renewal period will be disabled shortly. However, if you’re still at UW-Madison it is not too late to renew your account. SSCC Members should fill out the account renewal form. Drop-in users should email the Help Desk.
You’ll know you need to renew your account if you suddenly can’t log in.
New Data Storage Hardware
The SSCC recently purchased new data storage hardware, as our old storage was both out of space and reaching the end of its expected life. This was, we believe, the largest single purchase in the SSCC’s history: roughly $140,000 for 700 terabytes of usable storage. With an expected lifespan of five years, disk space now costs the SSCC about $40/TB/year. Given that cost, we ask that you be mindful of your use of disk space–but we’re also mindful of your time. If you aren’t using at least tens of gigabytes of space (i.e. at least 1% of a terabyte), the cost of storing your files is trivial and reducing it isn’t worth your time.
One way you can easily reduce your data storage is to use temporary space for files you don’t need to keep, like intermediate data sets. Files in temporary space are not backed up, saving us space immediately. Then they are deleted automatically after 30 days so you don’t have to remember to do so. The Windows Y: drive is temporary space that is shared with others; Linux /temp30days (which can also be mapped from Windows) is private temporary space. Just make yourself a folder in either location (or both) and put it to work.
A small but increasing number of SSCC researchers are now using multiple terabytes of data, and in some cases many terabytes. We want to be sure that anyone who is using that much space is using it efficiently and intentionally. We also need to ensure individuals don’t fill particular disk volumes and cause problems for others. We have therefore set a default quota of 10TB on project directories. (Note that this is an enormous amount of space.) If you need more than 10TB of space, reach out to the Help Desk and we’ll talk about what you’re doing and how it can be done as efficiently as possible, then increase your quota. Anyone who is currently using more than 10TB of space already has a higher quota.
Use Slurm If You Need More Memory
If you’re running jobs that need very large amounts of memory, Slurm is the place to run them. The Linstat servers have 768GB of memory but it must be shared with all the other users of the server. When you submit a job to Slurm you tell it how much memory the job needs, and it will reserve that amount just for you. We recently moved one of the Linstat servers into the Slurm cluster, so if your job needs 700+GB of memory, submit it to Slurm and you’ll get it.
We anticipate moving more of the Linstat servers into the Slurm cluster over the coming months, making Linstat a development and testing environment while the Slurm cluster is the place to get work done. But we need more people to give Slurm a try so we can be confident all of our research computing work can be done using Slurm. (To the best of our knowledge everything anyone has tried has worked, sometimes after a little tweaking–if you’ve run into problems, talk to us.)
If you haven’t used Slurm yet, Using the SSCC Slurm Cluster will get you started, or sign up for one of our Running Big Research Computing Jobs at the SSCC workshops and we’ll show you how.
Global Protect For Mac Update
Now that multi-factor authentication via Duo is required to connect to VPN using GlobalProtect, some Mac users are receiving “phantom” Duo pushes, particularly in the evening, as their Mac tries to reconnect to VPN automatically. This has been fixed in a new version of GlobalProtect for Mac, which Mac users should download and install.