Skip to content

Commit

Permalink
renaming to slurm-exporter and moving to reference MW repository
Browse files Browse the repository at this point in the history
  • Loading branch information
Giacomo Tirabassi committed Sep 23, 2022
1 parent 056c81a commit 7f1810a
Show file tree
Hide file tree
Showing 19 changed files with 85 additions and 56 deletions.
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## Changelog

Full commit history per tag: https://github.com/vpenso/prometheus-slurm-exporter/commits/{tag number}
Full commit history per tag: https://github.com/MarshallWace/slurm-exporter/commits/{tag number}

* **0.19**
- Merge PR#50
Expand Down
8 changes: 4 additions & 4 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ Use Git to clone the source code of the exporter, run all the tests and build th

```bash
# clone the source code
git clone https://github.com/vpenso/prometheus-slurm-exporter.git
cd prometheus-slurm-exporter
git clone https://github.com/MarshallWace/slurm-exporter.git
cd slurm-exporter
make
```

Expand All @@ -35,13 +35,13 @@ make test
Start the exporter (foreground), and query all metrics:

```bash
./bin/prometheus-slurm-exporter
./bin/slurm-exporter
```

If you wish to run the exporter on a different port, or the default port (8080) is already in use, run with the following argument:

```bash
./bin/prometheus-slurm-exporter --listen-address="0.0.0.0:<port>"
./bin/slurm-exporter --listen-address="0.0.0.0:<port>"
...

# query all metrics (default port)
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
PROJECT_NAME = prometheus-slurm-exporter
PROJECT_NAME = slurm-exporter
SHELL := $(shell which bash) -eu -o pipefail

GOPATH := $(shell pwd)/go/modules
Expand Down
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Prometheus collector and exporter for metrics extracted from the [Slurm](https://slurm.schedmd.com/overview.html) resource scheduling system.

> This work use https://github.com/vpenso/prometheus-slurm-exporter as starting point
> This work uses https://github.com/vpenso/prometheus-slurm-exporter as starting point
## Exported Metrics

Expand Down Expand Up @@ -58,7 +58,7 @@ Since version **0.18**, the following information are also extracted and exporte
* Memory: _allocated_ and in _total_.
* Labels: hostname and its Slurm status (e.g. _idle_, _mix_, _allocated_, _draining_, etc.).

See the related [test data](https://github.com/vpenso/prometheus-slurm-exporter/blob/master/test_data/sinfo_mem.txt) to check the format of the information extracted from Slurm.
See the related [test data](https://github.com/MarshallWace/slurm-exporter/blob/master/test_data/sinfo_mem.txt) to check the format of the information extracted from Slurm.

### Status of the Jobs

Expand Down Expand Up @@ -119,9 +119,9 @@ Collect _share_ statistics for every Slurm account. Refer to the [manpage of the
## Installation

* Read [DEVELOPMENT.md](DEVELOPMENT.md) in order to build the Prometheus Slurm Exporter. After a successful build copy the executable
`bin/prometheus-slurm-exporter` to a node with access to the Slurm command-line interface.
`bin/slurm-exporter` to a node with access to the Slurm command-line interface.

* A [Systemd Unit][sdu] file to run the executable as service is available in [lib/systemd/prometheus-slurm-exporter.service](lib/systemd/prometheus-slurm-exporter.service).
* A [Systemd Unit][sdu] file to run the executable as service is available in [contrib/lib/systemd/slurm-exporter.service](contrib/lib/systemd/slurm-exporter.service).

* (**optional**) Distribute the exporter as a Snap package: consult the [following document](packages/snap/README.md). **NOTE**: this method requires the use of [Snap](https://snapcraft.io), which is built by [Canonical](https://canonical.com).

Expand Down Expand Up @@ -167,11 +167,11 @@ Checking prometheus.yml
A [dashboard](https://grafana.com/dashboards/4323) is available in order to
visualize the exported metrics through [Grafana](https://grafana.com):

![Status of the Nodes](images/Node_Status.png)
![Status of the Nodes](contrib/images/Node_Status.png)

![Status of the Jobs](images/Job_Status.png)
![Status of the Jobs](contrib/images/Job_Status.png)

![SLURM Scheduler Information](images/Scheduler_Info.png)
![SLURM Scheduler Information](contrib/images/Scheduler_Info.png)


## License
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Description=Prometheus SLURM Exporter

[Service]
ExecStart=/usr/bin/prometheus-slurm-exporter
ExecStart=/usr/bin/slurm-exporter
Restart=always
RestartSec=15

Expand Down
2 changes: 1 addition & 1 deletion contrib/packages/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Packages

* Build RPM packages from
[rpm/prometheus-slurm-exporter.spec](rpm/prometheus-slurm-exporter.spec)
[rpm/slurm-exporter.spec](rpm/slurm-exporter.spec)
following documentation in [rpm/README.md](rpm/README.md]).
* Build a [Snap](https://snapcraft.io) package from
[../snap/snapcraft.yaml](../snap/snapcraft.yaml) following documentation in
Expand Down
10 changes: 5 additions & 5 deletions contrib/packages/rpm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,18 +19,18 @@ echo "%_topdir $HOME/rpmbuild" > ~/.rpmmacros

4. Clone this repo:
```bash
git clone https://github.com/vpenso/prometheus-slurm-exporter.git
git clone https://github.com/MarshallWace/slurm-exporter.git
```
5. Get into the source directory and copy the following files under ``~/rpmbuild/SOURCES``:
```bash
cd prometheus-slurm-exporter
cd slurm-exporter
cp README.md ~/rpmbuild/SOURCES
cp LICENSE ~/rpmbuild/SOURCES
cp lib/systemd/prometheus-slurm-exporter.service ~/rpmbuild/SOURCES
cp lib/systemd/slurm-exporter.service ~/rpmbuild/SOURCES
```
6. Copy the SPEC file in the proper directory:
```bash
cd prometheus-slurm-exporter
cd slurm-exporter
cp packages/rpm/*.spec ~/rpmbuild/SPECS
```

Expand All @@ -39,6 +39,6 @@ cp packages/rpm/*.spec ~/rpmbuild/SPECS
8. Build the RPM based on your SPEC file:
```bash
cd $HOME/rpmbuild/SPECS
rpmbuild -ba prometheus-slurm-exporter.spec
rpmbuild -ba slurm-exporter.spec
```
9. The RPM package will be placed under $HOME/rpmbuild/RPMS/x86_64
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,16 @@
%define debug_package %{nil}
%define __os_install_post %{_dbpath}/brp-compress

Name: prometheus-slurm-exporter
Name: slurm-exporter
Version: 0.3
Release: 1%{?dist}
Summary: Prometheus exporter for SLURM metrics
Group: Monitoring

License: GPL 3.0
URL: https://github.com/vpenso/prometheus-slurm-exporter

Source0: https://github.com/vpenso/prometheus-slurm-exporter/releases/download/%{version}/slurm_exporter-%{version}.linux-amd64.tar.gz
Source1: prometheus-slurm-exporter.service
URL: https://github.com/MarshallWace/slurm-exporter
Source0: https://github.com/MarshallWace/slurm-exporter/releases/download/%{version}/slurm_exporter-%{version}.linux-amd64.tar.gz
Source1: slurm-exporter.service
Source2: LICENSE
Source3: README.md

Expand Down Expand Up @@ -40,12 +39,12 @@ rm -rf %{buildroot}
mkdir -vp %{buildroot}
mkdir -vp %{buildroot}%{_unitdir}/
mkdir -vp %{buildroot}/usr/bin
mkdir -vp %{buildroot}/usr/share/doc/prometheus-slurm-exporter-%{version}
mkdir -vp %{buildroot}/usr/share/doc/slurm-exporter-%{version}
mkdir -vp %{buildroot}/var/lib/prometheus
install -m 755 prometheus-slurm-exporter %{buildroot}/usr/bin/prometheus-slurm-exporter
install -m 644 %{SOURCE1} %{buildroot}/usr/lib/systemd/system/prometheus-slurm-exporter.service
install -m 644 %{SOURCE2} %{buildroot}/usr/share/doc/prometheus-slurm-exporter-%{version}/LICENSE
install -m 644 %{SOURCE3} %{buildroot}/usr/share/doc/prometheus-slurm-exporter-%{version}/README.md
install -m 755 slurm-exporter %{buildroot}/usr/bin/slurm-exporter
install -m 644 %{SOURCE1} %{buildroot}/usr/lib/systemd/system/slurm-exporter.service
install -m 644 %{SOURCE2} %{buildroot}/usr/share/doc/slurm-exporter-%{version}/LICENSE
install -m 644 %{SOURCE3} %{buildroot}/usr/share/doc/slurm-exporter-%{version}/README.md

%clean
rm -rf %{buildroot}
Expand All @@ -71,7 +70,7 @@ systemctl start %{name}.service
%defattr(-,root,root,-)
%doc LICENSE
%doc README.md
%{_bindir}/prometheus-slurm-exporter
%{_bindir}/slurm-exporter
%{_unitdir}/%{name}.service
%attr(755, prometheus, prometheus)/var/lib/prometheus

Expand All @@ -81,4 +80,4 @@ systemctl start %{name}.service
* Wed Jan 31 2018 Matteo <[email protected]> - 0.2
- Fix issue #3
* Mon Dec 04 2017 Matteo <[email protected]> - 0.1
- Full commit history: https://github.com/vpenso/prometheus-slurm-exporter/commits/master
- Full commit history: https://github.com/MarshallWace/slurm-exporter/commits/master
16 changes: 8 additions & 8 deletions contrib/packages/snap/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Building the prometheus-slurm-exporter snap
Packaging and delivering the prometheus-slurm-exporter as a snap provides users of prometheus-slurm-exporter
Packaging and delivering the slurm-exporter as a snap provides users of slurm-exporter
a hardened, streamlined, and idempotent experience when consuming this software. See [snapcraft](https://snapcraft.io/) for more information on snaps.


Expand All @@ -21,12 +21,12 @@ snapcraft --use-lxd
Once the snap build has completed, list the current working directory to see the resultant snap artifact.
```bash
$ ls -la *.snap
-rw-r--r-- 1 bdx bdx 5562368 Aug 16 18:19 prometheus-slurm-exporter_0.11-1-g01dd959_amd64.snap
-rw-r--r-- 1 bdx bdx 5562368 Aug 16 18:19 slurm-exporter_0.11-1-g01dd959_amd64.snap
```

### Install locally built snap
```bash
sudo snap install prometheus-slurm-exporter_`git describe --tags`_amd64.snap --classic --dangerous
sudo snap install slurm-exporter_`git describe --tags`_amd64.snap --classic --dangerous
```
* `--classic` - this snap uses classic confinement to allow it to find the slurm commands in the system.
* `--dangerous` - because we are installing this snap from a local resource and sha can't be verified by the snapstore.
Expand All @@ -35,13 +35,13 @@ sudo snap install prometheus-slurm-exporter_`git describe --tags`_amd64.snap --c
Use `ps` to verify the process is running.
```bash
$ ps aux | grep prometheus | head -1
root 2271391 0.0 0.0 1453596 14012 ? SLsl 18:32 0:00 /snap/prometheus-slurm-exporter/x1/bin/prometheus-slurm-exporter
root 2271391 0.0 0.0 1453596 14012 ? SLsl 18:32 0:00 /snap/slurm-exporter/x1/bin/slurm-exporter
```
Use `netstat` to verify that the installed `prometheus-slurm-exporter` snap process is listening on port 8080.
Use `netstat` to verify that the installed `slurm-exporter` snap process is listening on port 8080.
```bash
$ sudo netstat -peanut | grep prometheus
tcp6 0 0 :::8080 :::* LISTEN 0 15042010 2271391/prometheus-slurm-exporter
tcp6 0 0 :::8080 :::* LISTEN 0 15042010 2271391/slurm-exporter
```
Lastly, curl the metrics endpoint.
Expand Down Expand Up @@ -90,7 +90,7 @@ slurm_scheduler_mean_cycle 481
...
```
To uninstall the prometheus-slurm-exporter snap:
To uninstall the slurm-exporter snap:
```bash
sudo snap remove prometheus-slurm-exporter
sudo snap remove slurm-exporter
```
12 changes: 6 additions & 6 deletions contrib/snap/snapcraft.yaml
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@
name: prometheus-slurm-exporter
name: slurm-exporter
summary: Prometheus Slurm Exporter
description: |
Prometheus collector and exporter for metrics extracted from the Slurm resource scheduling system.
adopt-info: prometheus-slurm-exporter
adopt-info: slurm-exporter

grade: stable
confinement: classic

base: core20

apps:
prometheus-slurm-exporter:
slurm-exporter:
daemon: simple
environment:
PATH: $PATH:/snap/bin
command: bin/prometheus-slurm-exporter
command: bin/slurm-exporter

parts:
prometheus-slurm-exporter:
source: https://github.com/vpenso/prometheus-slurm-exporter.git
slurm-exporter:
source: https://github.com/MarshallWace/slurm-exporter.git
plugin: go
go-channel: 1.14/stable
override-build: |
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
module github.com/vpenso/prometheus-slurm-exporter
module github.com/MarshallWace/slurm-exporter

go 1.17

Expand Down
4 changes: 2 additions & 2 deletions main.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
/* Copyright 2017-2020 Victor Penso, Matteo Dessalvi, Joeri Hermans
/* Copyright 2017-2022 Victor Penso, Matteo Dessalvi, Joeri Hermans, Marshall Wace Asset Management
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
Expand All @@ -21,10 +21,10 @@ import (

"log"

"github.com/MarshallWace/slurm-exporter/pkg/slurm"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/collectors"
"github.com/prometheus/client_golang/prometheus/promhttp"
"github.com/vpenso/prometheus-slurm-exporter/pkg/slurm"
)

var listenAddress = flag.String(
Expand Down
15 changes: 15 additions & 0 deletions pkg/ldapsearch/ldapsearch.go
Original file line number Diff line number Diff line change
@@ -1,3 +1,18 @@
/* Copyright 2022 Marshall Wace Asset Management
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>. */

package ldapsearch

import (
Expand Down
15 changes: 15 additions & 0 deletions pkg/ldapsearch/ldapsearch_test.go
Original file line number Diff line number Diff line change
@@ -1,3 +1,18 @@
/* Copyright 2022 Marshall Wace Asset Management
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>. */

package ldapsearch

// import (
Expand Down
4 changes: 2 additions & 2 deletions pkg/slurm/jobs.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
/* Copyright 2017 Victor Penso, Matteo Dessalvi
/* Copyright 2022 Marshall Wace Asset Management
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
Expand All @@ -20,8 +20,8 @@ import (
"fmt"
"strconv"

"github.com/MarshallWace/slurm-exporter/pkg/ldapsearch"
"github.com/prometheus/client_golang/prometheus"
"github.com/vpenso/prometheus-slurm-exporter/pkg/ldapsearch"
)

const (
Expand Down
2 changes: 1 addition & 1 deletion pkg/slurm/jobs_test.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
/* Copyright 2017 Victor Penso, Matteo Dessalvi
/* Copyright 2022 Marshall Wace Asset Management
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
Expand Down
2 changes: 1 addition & 1 deletion pkg/slurm/nodes.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
/* Copyright 2017 Victor Penso, Matteo Dessalvi
/* Copyright 2022 Marshall Wace Asset Management
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
Expand Down
2 changes: 1 addition & 1 deletion pkg/slurm/nodes_test.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
/* Copyright 2017 Victor Penso, Matteo Dessalvi
/* Copyright 2022 Marshall Wace Asset Management
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
Expand Down
4 changes: 2 additions & 2 deletions pkg/slurm/slurm.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
/* Copyright 2017 Victor Penso, Matteo Dessalvi
/* Copyright 2022 Marshall Wace Asset Management
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
Expand All @@ -23,8 +23,8 @@ import (
"strings"
"time"

"github.com/MarshallWace/slurm-exporter/pkg/ldapsearch"
"github.com/prometheus/client_golang/prometheus"
"github.com/vpenso/prometheus-slurm-exporter/pkg/ldapsearch"
)

var (
Expand Down

0 comments on commit 7f1810a

Please sign in to comment.