ricardobarroslourenco / ccprimer

Compute Canada for geoscientists in a rush

• 👋 Hi, I’m @ricardobarroslourenco !
• 👀 I’m interested in researching XAI methods for physical science emulators.
• 🌱 I’m currently learning about model interpretability and explainability.
• 💞️ I want to collaborate with CS and Geosci researchers working in this nexus.
• 📫 You can reach me by social media or e-mail. Look at my URL aggregator, which has all my contacts.
• A summary of my CodersRank profile:

ccprimer's Issues

Add info on how to zip/unzip

Related to long-term storage on /nearline

Portuguese

How to do it, and sync on GitHub pages...

Include detach reattach on the screen setup

Really about the screen -d -r command that gets a screen session that is hanged (supposedly someone is using it, however is not) and force reattachment.

Remove page caching

To maximize refresh. Refer to https://stackoverflow.com/q/24851824/4743714

Queueing strategies

Include partition stats:

[lourenco@gra-login3 ~]$ partition-stats 

Node type |                     Max walltime                            
          |   3 hr   |  12 hr  |  24 hr  |  72 hr  |  168 hr |  672 hr |
----------|-------------------------------------------------------------
       Number of Queued Jobs by partition Type (by node:by core)        
----------|-------------------------------------------------------------
Regular   |    0:245 |   1:342 | 153:667 |  54:659 |  69:372 |   0:0   |
Large Mem |    0:401 |  88:948 |   3:252 |   2:10  |   1:133 |   0:0   |
GPU       |    0:1   |   0:2   |   0:76  |   8:162 |  33:11  |   0:0   |
----------|-------------------------------------------------------------
      Number of Running Jobs by partition Type (by node:by core)        
----------|-------------------------------------------------------------
Regular   |    1:8   |  23:84  |  40:1301|  55:886 |  58:1887|   0:0   |
Large Mem |    0:1   |   1:50  |   1:108 |   8:14  |   1:21  |   0:0   |
GPU       |    0:10  |   0:23  |   2:38  |  29:80  |   5:15  |   0:0   |
----------|-------------------------------------------------------------
        Number of Idle nodes by partition Type (by node:by core)        
----------|-------------------------------------------------------------
Regular   |    0:0   |   0:0   |   0:0   |   0:0   |   0:0   |   0:0   |
Large Mem |    0:0   |   0:0   |   0:0   |   0:0   |   0:0   |   0:0   |
GPU       |   43:38  |  29:24  |  24:24  |  21:15  |  15:13  |   0:0   |
----------|-------------------------------------------------------------
       Total Number of nodes by partition Type (by node:by core)        
----------|-------------------------------------------------------------
Regular   | 1074:632 |1074:632 |1044:612 | 753:381 | 371:170 |   0:0   |
Large Mem |   27:19  |  27:19  |  24:16  |  20:4   |   5:4   |   0:0   |
GPU       |  214:136 | 194:116 | 182:110 | 139:81  |  42:39  |   0:0   |
----------|-------------------------------------------------------------


[lourenco@gra-login3 ~]$ 

And a comment on how to get around scheduling queues, when the CPU queue is huge...

CC account creation

Primer on setting up an account

HPC stack

• General outline of HPC systems vs Cloud based
• Schedulers and Scheduling
• Filesystems
• Data transfer (the large ones) and Data Management
• Reproducible experiments on HPC (usage of containers and other orchestration need to reproduce experiments)

Bash

Primer on bash commands

Data transfer

Following chat with @cherylarogers , write a section on Globus Transfer, and also on THREDDS, and other OGC-compliant possible data transfers. Perhaps covering rsync.

Prior to v1.0

Backlog/Checklist for when getting into v1.0

• Remove backup folder and its content;

Documentation

• Licensing
• How-to contribute
• Github Pages anchor
• Rendered EPUB/PDF versions
• Badges
• Versioning

Git and GitHub

First git primer and GitHub general account setup