Skip to content

Commit 7ec4fd9

Browse files
authored
Merge pull request #208 from dsajdak/good25
Good25
2 parents 33e32be + 772d4e8 commit 7ec4fd9

File tree

1 file changed

+35
-19
lines changed

1 file changed

+35
-19
lines changed

xdmod/README.md

Lines changed: 35 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,8 @@ Submitted batch job 19
5555

5656
---
5757

58-
## Open XDMoD Installation [Documentation](https://open.xdmod.org/install.html)
58+
## Open XDMoD Installation
59+
[Documentation](https://open.xdmod.org/install.html)
5960

6061
**Note: In the PEARC tutorials we demonstrated all the steps for this installation / configuration section to participants. You do not need to run these steps yourself as the containers are already populated with a working XDMoD installation.**
6162

@@ -66,9 +67,11 @@ instructions to install the packages.
6667

6768
Reference: [RPM Installation Guide](https://open.xdmod.org/install-rpm.html)
6869

69-
### Pre-Requisites Installation [Documentation](https://open.xdmod.org/software-requirements.html)
70+
### Pre-Requisites Installation
71+
[Documentation](https://open.xdmod.org/software-requirements.html)
7072

71-
### Open XDMoD RPM Installation [Documentation](https://open.xdmod.org/install-rpm.html)
73+
### Open XDMoD RPM Installation
74+
[Documentation](https://open.xdmod.org/install-rpm.html)
7275

7376
First ssh to the XDMoD container
7477
```shell
@@ -87,7 +90,8 @@ And install the the XDMoD RPM
8790

8891
---
8992

90-
## Open XDMoD Configuration [Documentation](https://open.xdmod.org/configuration.html)
93+
## Open XDMoD Configuration
94+
[Documentation](https://open.xdmod.org/configuration.html)
9195

9296
**Note** This part will be brief in the PEARC2023 tutorial. These processes have been done already as part of the docker.
9397

@@ -183,7 +187,8 @@ The next series of steps will involve completing each numbered setup item in tur
183187
setup after we have successfully shredded, ingested, and aggregated the accounting data generated by the jobs we
184188
submitted at the beginning of this tutorial.
185189

186-
### General Settings [Documentation](https://open.xdmod.org/configuration.html#general-settings)
190+
### General Settings
191+
[Documentation](https://open.xdmod.org/configuration.html#general-settings)
187192

188193
```shell
189194
1) General Settings
@@ -245,15 +250,16 @@ Center Logo Path: [/srv/xdmod/small-logo.png]
245250
```
246251
- Press the `Enter` key.
247252

248-
#### Center Logo Width ( in pixels )
253+
#### Center Logo Width (in pixels)
249254
If you do provide a custom center image, you will also need to provide its width in pixels.
250255

251256
```shell
252257
Center Logo Width: [354]
253258
```
254259
- Press the `Enter` key to accept the default value.
255260

256-
#### XDMoD Dashboard [Documentation](https://open.xdmod.org/dashboard.html)
261+
#### XDMoD Dashboard
262+
[Documentation](https://open.xdmod.org/dashboard.html)
257263
The XDMoD Dashboard provides users with targeted statistics and reports based on the level of access they've been granted in XDMoD. We highly recommend you turn this feature on in your installation.
258264

259265
```
@@ -267,7 +273,8 @@ You will now be prompted to overwrite the config file `/etc/xdmod/portal_setting
267273

268274
If you need to change any of your answers then you can type `no`, press `Enter` and you will be taken back to the main setup menu. At which point you can press `1` and to go through the General Settings setup again.
269275

270-
### Database Settings [Documentation](https://open.xdmod.org/configuration.html#database-settings)
276+
### Database Settings
277+
[Documentation](https://open.xdmod.org/configuration.html#database-settings)
271278

272279
Now that we've successfully completed the `General Settings` setup, we can move on to `Database Settings`.
273280

@@ -411,7 +418,8 @@ Press ENTER to continue.
411418
```
412419
- Press the `Enter` key again, this will return you to the main setup menu.
413420

414-
### Organization Settings [Documentation](https://open.xdmod.org/configuration.html#organization-settings)
421+
### Organization Settings
422+
[Documentation](https://open.xdmod.org/configuration.html#organization-settings)
415423

416424
```shell
417425
3) Organization
@@ -467,7 +475,8 @@ Overwrite config file '/etc/xdmod/organization.json' (yes, no)? [yes]
467475
```
468476
- Press the `Enter` key.
469477

470-
### Resource Setup [Documentation](https://open.xdmod.org/configuration.html#resources)
478+
### Resource Setup
479+
[Documentation](https://open.xdmod.org/configuration.html#resources)
471480

472481
Next we will go through the process of adding a Resource to XDMoD.
473482

@@ -700,7 +709,8 @@ Press ENTER to continue.
700709

701710
- Press the `Enter` key once more to be taken back to the main XDMoD configuration menu.
702711

703-
## Create Admin User [Documentation](https://open.xdmod.org//configuration.html#create-admin-user)
712+
## Create Admin User
713+
[Documentation](https://open.xdmod.org//configuration.html#create-admin-user)
704714

705715
```shell
706716
5) Create Admin User
@@ -777,7 +787,8 @@ Admin user created.
777787
Press ENTER to continue.
778788
```
779789

780-
## Hierarchy Setup [Documentation](https://open.xdmod.org/hierarchy.html)
790+
## Hierarchy Setup
791+
[Documentation](https://open.xdmod.org/hierarchy.html)
781792

782793
```shell
783794
6) Hierarchy
@@ -841,7 +852,8 @@ Press ENTER to continue.
841852

842853
- Press the `Enter` key to continue.
843854

844-
## Data Warehouse Batch Export [Documentation](https://open.xdmod.org/dw-export.html#configuration)
855+
## Data Warehouse Batch Export
856+
[Documentation](https://open.xdmod.org/dw-export.html#configuration)
845857

846858
```shell
847859
7) Data Warehouse Batch Export
@@ -980,15 +992,16 @@ The `xdmod-setup` script is used for the basic setup of Open XDMoD. The script i
980992

981993
Now that we have XDMoD setup, it's time to ingest some data. The jobs that we started at the beginning of the tutorial should be complete so let's walk through the shred, ingest, and aggregate steps for job accounting data.
982994

983-
### Shredding [Documentation](https://open.xdmod.org/shredder.html)
995+
### Shredding
996+
[Documentation](https://open.xdmod.org/shredder.html)
984997
XDMoD provides a special command line tool for working with directly with slurm via `sacct` called `xdmod-slurm-helper`. To see what command line arguments it accepts you can run `xdmod-slurm-helper -h`. For our purposes we will be using it as follows:
985998

986999
**INTERACTIVE**
9871000

9881001
```shell
9891002
[root@xdmod ~] sudo -u xdmod xdmod-slurm-helper -r hpc --start-time $yesterday --end-time $tomorrow
9901003
```
991-
*Note, you will need to replace `$yesterday` and `$tomrrow` with their respective values in YYYY-MM-DD format.*
1004+
*Note, you will need to replace `$yesterday` and `$tomorrow` with their respective values in YYYY-MM-DD format.*
9921005

9931006
If your organization doesn't use Slurm or will be using the log files instead of querying `sacct` directly, then `xdmod-shredder` will be the command for you. You can find our Shredder guide at [Shredder Guide](https://open.xdmod.org/shredder.html)
9941007

@@ -1004,7 +1017,8 @@ Upon running `xdmod-slurm-helper` you should see output similar to:
10041017
[root@xdmod ~]#
10051018
```
10061019

1007-
### Ingesting & Aggregating [Documentation](https://open.xdmod.org/ingestor.html)
1020+
### Ingesting & Aggregating
1021+
[Documentation](https://open.xdmod.org/ingestor.html)
10081022
Now that the accounting logs have been shredded we can now ingest and aggregate the information from them, thus making the information available in XDMoD.
10091023

10101024
**INTERACTIVE**
@@ -1435,7 +1449,7 @@ Once the jobs that we started in the last step complete, it's time to run throug
14351449
[hpcadmin@xdmod ~] sudo su -
14361450
[root@xdmod ~] sudo -u xdmod xdmod-slurm-helper -r hpc --start-time $yesterday --end-time $tomorrow
14371451
```
1438-
**_Note: you will need to replace `$yesterday` and `$tomrrow` with their respective values in YYYY-MM-DD format._**
1452+
**_Note: you will need to replace `$yesterday` and `$tomorrow` with their respective values in YYYY-MM-DD format._**
14391453

14401454
If your organization doesn't use Slurm or will be using the log files instead of querying `sacct` directly, then
14411455
`xdmod-shredder` will be the command for you. You can find our Shredder guide at [Shredder Guide](https://open.xdmod.org/shredder.html)
@@ -1637,9 +1651,11 @@ If you completed the optional step 4 in the previous section, login as the `sfos
16371651
10. Change `hello.py` to `plot_rbm_logistic_classification.py` and save. Close this window.
16381652
11. Go back to Job Composer and submit the job
16391653

1640-
If you'd like to see this recent job show up in the OnDemand dashboard widgets, go back to the XDMoD container and re-run the job ingestion steps. We'll use the helper script discussed above to complete all three tasks:
1654+
If you'd like to see this recent job show up in the OnDemand dashboard widgets, go back to the XDMoD container and re-run the job ingestion steps. You'll likely be logged into the ondemand server so log out of that, confirm you're on the xdmod server and login as root. We'll use the helper script discussed above to complete all three tasks:
16411655

16421656
```shell
1657+
[hpcadmin@ondemand config]$ exit
1658+
[hpcadmin@xdmod ~]$ sudo su -
16431659
[root@xdmod /] sudo -u xdmod /srv/xdmod/scripts/shred-ingest-aggregate-all.sh
16441660
```
16451661

@@ -1655,7 +1671,7 @@ Since the `xdmod-ondemand` module displays usage of Open OnDemand you need to ha
16551671
file generated from Open OnDemand so there is data to show in XDMoD.
16561672

16571673
### Installation & Configuration
1658-
In preparation for this tutorial, these steps were done in advance. For details on what these steps entail, [see here for installation](#installation) and [here for configuration](#configuration).
1674+
In preparation for this tutorial, the OnDemand module was installed and the OnDemand resource was setup in XDMoD. For details on what these steps entail, refer to the full day tutorial "Resource Setup" steps.
16591675

16601676
### Obtaining Open OnDemand logs
16611677
The `xdmod-ondemand` module parses the webserver log files from Open OnDemand. The `xdmod` container in this tutorial includes an example log file that

0 commit comments

Comments
 (0)