New Funding for Computing Resources
We’re excited to announce that the SSCC has received a $250,000 award from the Research Core Revitalization Program, and $2 million in capital equipment funding from UW-Madison. We are grateful to the University for this funding, and to College of Letters & Science leadership for their advocacy on behalf of our proposals. It will allow us to replace aging Linstat servers, add more GPU servers to the Slurm clusters, purchase next-generation data storage that will give better performance at lower cost, and meet other needs.
Watch this space for details as the new hardware comes online!
Fall Training
The SSCC’s statistical consultants will be teaching our core workshops just before the start of the fall semester. This includes:
- Introductions to R, Stata, and Python. These workshops will teach you the core concepts of the language and the basics of how to use it. Taking one of these workshops before you take a class that uses one of these languages will let you focus on the class content rather than struggling with the language. Interested in using AI in your research? Python is the primary language for that.
- Data Wrangling in R and Stata. These workshops will teach you the skills needed to turn raw data into data you can analyze. These skills are especially critical if the data you use tend to be messy, not contain all the variables you need, come from multiple sources, start in the wrong structure…in other words, if you’re a social scientist.
- Using the SSCC Linux Servers. These workshops will teach you how to access the full power of SSCC’s computing clusters, from easy-to-use interactive servers like Linstat, to large-scale CPU and GPU computing in Slurm–or the Silo equivalents if you work with HIPAA or other sensitive or restricted data.
Visit the training page for details and to register.
Go Easy on the File System
You may have noticed the SSCC’s Linux servers sometimes being slow over the last couple of weeks. That’s been caused by a fairly small number of users putting an unusually heavy load on the SSCC’s Linux file server.
Reading and writing files is one of the slowest things a modern computer does. Also, while we have many servers, there’s just one file server so overloading it affects everyone. The long-term solution is the next-generation storage mentioned earlier, but in the meantime a few simple precautions will prevent most problems:
- Don’t save files unnecessarily. Saving your data at the end of a block of code can be useful for testing it, but remove that save when you no longer need it.
- If your job creates temporary files like intermediate data sets, save them in /tmp if you can. The /tmp directory is on the local drive of each server, so working with files in /tmp is both faster and puts no load on the file server. But its size is limited (240GB-720GB depending on the server) and files there are temporary. Save anything you want to keep after your job is done elsewhere. Also, remember a file in /tmp on one server will not be available to any other server.
- If your job does a lot of reading and writing of files, do not submit many instances of it to Slurm. This is a very easy way to overload the file server. Think single digits, maybe even one job at a time.
- Some light databases that work directly with files on disk are not designed for network file systems. (DuckDB is one example.) We recently upgraded our ResearchDB database server–email the Help Desk if you would like to use it.
The SSCC’s statistical consultants can help you implement these suggestions.
AI Skill Files for SSCC’s Servers
The Guide to Research Computing at the SSCC has been the place for humans to learn how to run jobs on the SSCC’s servers, but now it’s ready to teach AIs too. A new chapter includes two “skill” files, one for the general SSCC environment and one for Silo, that include all the SSCC-specific information tools like Posit Assistant and Claude Code need to write code that will run on the SSCC’s servers. While these tools can read and understand the SSCC’s documentation, using the skill files will consume far fewer tokens.
Help Improve Data Classification Guidance
The University is working on major changes to how data is classified, which determines the level of security it requires. This includes research data, so most SSCC members will be affected either directly or indirectly. You are invited to read the draft guidance document and then give feedback by clicking on the “Give us your Feedback” link in the document. We encourage you to make your voice heard. More information can be found on the Data Classification Initiative Website.