Build system changes, doc fixes, few attribute renames (#359)

* pyslurm.Job: add attribute run_time_remaining
* pyslurm.JobStep: add run_time_remaining and elapsed_cpu_time attributes
* rename __version__.py to just version.py
* update doc_requirements to exclude cython 3.1.* releases
* mkdocs.yml updates
* move flake8 config to pyproject.toml
* move away from invoking setup.py directly
* build.sh: use pip install
* rename pyslurm.pyx to deprecated.pyx

anything in there is effectively the old api, which is considered
deprecated. The name change doesn't affect anything, because all the
stuff is wildcard-imported anyway.

But it prevents the documentation from being built due to circular alias
errors.

* add first working pyslurm.spec file for building rpms

* ability to provide core-count for builddocs.sh

* remove configure.sh script

obsolete, should use a slurm-container solution for testing and dev

* remove run_tests* scripts
* add cython-lint config, update flake8 config
* remove cython.cfg, flake8 config is in pyproject.toml
* update tox.ini
* pyslurm.JobStep: rename step.alloc_cpus to step.cpus
* update copyright for docs
* doc: add JobStepStatistics
* core job+step: fix docs

doc_requirements.txt

@@ -1,4 +1,4 @@
-cython>=3.0.0b1
+cython>=3.0.11,<3.1
 wheel
 setuptools
 mkdocstrings[python]

docs/reference/config.md

@@ -6,4 +6,4 @@ title: Config
     This API is currently being completely reworked, and is subject to be
     removed in the future when a replacement is introduced
 
-::: pyslurm.config
+::: pyslurm.deprecated.config

docs/reference/db/cluster.md

@@ -6,4 +6,4 @@ title: Cluster
     This API is currently being completely reworked, and is subject to be
     removed in the future when a replacement is introduced
 
-::: pyslurm.slurmdb_clusters
+::: pyslurm.deprecated.slurmdb_clusters

docs/reference/db/event.md

@@ -6,4 +6,4 @@ title: Event
     This API is currently being completely reworked, and is subject to be
     removed in the future when a replacement is introduced
 
-::: pyslurm.slurmdb_events
+::: pyslurm.deprecated.slurmdb_events

docs/reference/db/jobstats.md

@@ -3,3 +3,4 @@ title: JobStatistics
 ---
 
 ::: pyslurm.db.JobStatistics
+::: pyslurm.db.JobStepStatistics

docs/reference/db/reservation.md

@@ -6,4 +6,4 @@ title: Reservation
     This API is currently being completely reworked, and is subject to be
     removed in the future when a replacement is introduced
 
-::: pyslurm.slurmdb_reservations
+::: pyslurm.deprecated.slurmdb_reservations

docs/reference/frontend.md

@@ -6,4 +6,4 @@ title: Frontend
     This API is currently being completely reworked, and is subject to be
     removed in the future when a replacement is introduced
 
-::: pyslurm.front_end
+::: pyslurm.deprecated.front_end

docs/reference/hostlist.md

@@ -6,4 +6,4 @@ title: Hostlist
     This API is currently being completely reworked, and is subject to be
     removed in the future when a replacement is introduced
 
-::: pyslurm.hostlist
+::: pyslurm.deprecated.hostlist

docs/reference/old/db/job.md

@@ -6,5 +6,5 @@ title: Job
     This is superseded by [pyslurm.db.Job](../../db/job.md) class and will
     be removed in a future release
 
-::: pyslurm.slurmdb_jobs
+::: pyslurm.deprecated.slurmdb_jobs
     handler: python

docs/reference/old/job.md

@@ -6,5 +6,5 @@ title: Job
     This class is superseded by [pyslurm.Job](../job.md) and will be removed
     in a future release.
 
-::: pyslurm.job
+::: pyslurm.deprecated.job
     handler: python

docs/reference/old/jobstep.md

@@ -6,5 +6,5 @@ title: JobStep
     This class is superseded by [pyslurm.JobStep](../jobstep.md) and will be
     removed in a future release.
 
-::: pyslurm.jobstep
+::: pyslurm.deprecated.jobstep
     handler: python

docs/reference/old/node.md

@@ -6,5 +6,5 @@ title: Node
     This class is superseded by [pyslurm.Node](../node.md) and will be
     removed in a future release.
 
-::: pyslurm.node
+::: pyslurm.deprecated.node
     handler: python

docs/reference/old/partition.md

@@ -6,5 +6,5 @@ title: Partition
     This class is superseded by [pyslurm.Partition](../partition.md) and will
     be removed in a future release.
 
-::: pyslurm.partition
+::: pyslurm.deprecated.partition
     handler: python

docs/reference/reservation.md

@@ -6,4 +6,4 @@ title: Reservation
     This API is currently being completely reworked, and is subject to be
     removed in the future when a replacement is introduced
 
-::: pyslurm.reservation
+::: pyslurm.deprecated.reservation

docs/reference/statistics.md

@@ -6,4 +6,4 @@ title: Statistics
     This API is currently being completely reworked, and is subject to be
     removed in the future when a replacement is introduced
 
-::: pyslurm.statistics
+::: pyslurm.deprecated.statistics

docs/reference/topology.md

@@ -6,4 +6,4 @@ title: Topology
     This API is currently being completely reworked, and is subject to be
     removed in the future when a replacement is introduced
 
-::: pyslurm.topology
+::: pyslurm.deprecated.topology

docs/reference/trigger.md

@@ -6,4 +6,4 @@ title: Trigger
     This API is currently being completely reworked, and is subject to be
     removed in the future when a replacement is introduced
 
-::: pyslurm.trigger
+::: pyslurm.deprecated.trigger

mkdocs.yml

@@ -3,7 +3,7 @@ site_name: "pyslurm"
 site_url: "https://pyslurm.github.io"
 repo_url: "https://github.com/PySlurm/pyslurm"
 repo_name: "PySlurm/pyslurm"
-copyright: Copyright © 2023 PySlurm Developers
+copyright: Copyright © 2024 PySlurm Developers
 
 nav:
   - Home:
@@ -22,13 +22,13 @@ theme:
     - navigation.top
     - content.code.copy
     - toc.follow
-  palette: 
+  palette:
     - media: "(prefers-color-scheme: light)"
       scheme: default
       accent: purple
       primary: indigo
       toggle:
-        icon: material/brightness-7 
+        icon: material/brightness-7
         name: Switch to dark mode
     - media: "(prefers-color-scheme: dark)"
       scheme: slate
@@ -54,6 +54,8 @@ plugins:
           options:
             filters: ["!^_"]
             docstring_style: google
+            allow_inspection: true
+            inherited_members: false
             show_signature: true
             show_root_heading: true
             show_symbol_type_toc: true

pyproject.toml

@@ -1,16 +1,33 @@
 [build-system]
-# Minimum requirements
+build-backend = "setuptools.build_meta"
 requires = [
     "setuptools>=59.2.0",
     "wheel>=0.37.0",
     "Cython>=0.29.37",
     "packaging>=21.3"
 ]
 
+[tool.flake8]
+filename = ["*.py", "*.pyx", "*.pxd"]
+max-line-length = 80
+exclude = [
+    ".git",
+    "__pycache__",
+    ".eggs",
+    "*.egg",
+    "build",
+    "tests",
+    "examples",
+    "scripts",
+]
+extend-ignore = ["E203", "E901", "E225", "E226", "E227", "E999"]
+
+[tool.cython-lint]
+max-line-length = 80
+ignore = ['E503', 'E504']
+exclude = 'deprecated.pyx'
 
 [tool.codespell]
 skip = '.git,*.pdf,*.svg,*.lock,*.css'
 check-hidden = true
-# ignore-regex = ''
-# ignore some variables etc
 ignore-words-list = 'mmaped,splitted'

pyslurm.spec

@@ -0,0 +1,46 @@
+%define python3_pkgversion 3.11
+
+Name:            python-pyslurm
+Version:         24.5.0
+%define rel      1
+Release:         %{rel}%{?dist}
+Summary:         Python interface to Slurm
+License:         GPLv2+
+URL:             https://github.com/PySlurm/pyslurm
+Source:          pyslurm-%{version}.tar.gz
+
+BuildRequires:   python%{python3_pkgversion}-devel
+BuildRequires:   python%{python3_pkgversion}-setuptools
+BuildRequires:   python%{python3_pkgversion}-wheel
+BuildRequires:   python%{python3_pkgversion}-Cython
+BuildRequires:   python%{python3_pkgversion}-packaging
+BuildRequires:   python-rpm-macros
+BuildRequires:   slurm-devel >= 24.05.0
+BuildRequires:   slurm >= 24.05.0
+Requires:        python%{python3_pkgversion}
+
+%description
+pyslurm is a Python interface to Slurm
+
+%package -n python%{python3_pkgversion}-pyslurm
+Summary:        %{summary}
+
+%description -n python%{python3_pkgversion}-pyslurm
+pyslurm is a Python interface to Slurm
+
+%prep
+%autosetup -p1 -n pyslurm-%{version}
+
+%generate_buildrequires
+%pyproject_buildrequires -R
+
+%build
+%pyproject_wheel
+
+%install
+%pyproject_install
+%pyproject_save_files pyslurm
+
+%files -f %{pyproject_files}
+%license COPYING.txt
+%doc README.md

pyslurm/__init__.py

@@ -2,19 +2,16 @@
 
 pyslurm is a wrapper around the Slurm C-API.
 """
-from __future__ import absolute_import
-
 import os
 import sys
 
-sys.setdlopenflags(sys.getdlopenflags() | os.RTLD_GLOBAL | os.RTLD_DEEPBIND )
+sys.setdlopenflags(sys.getdlopenflags() | os.RTLD_GLOBAL | os.RTLD_DEEPBIND)
 
 # Initialize slurm api
 from pyslurm.api import slurm_init, slurm_fini
 slurm_init()
 
-from .pyslurm import *
-from .__version__ import __version__
+from .version import __version__
 
 from pyslurm import db
 from pyslurm import utils
@@ -36,6 +33,5 @@
 )
 from pyslurm.core import slurmctld
 
-
-def version():
-    return __version__
+# The old API in deprecated.pyx
+from pyslurm.deprecated import *

pyslurm/core/job/job.pxd

@@ -96,7 +96,7 @@ cdef class Jobs(MultiClusterMap):
             Slurm controllers memory will not be removed either.
             The default is False, so old jobs will be removed, and new Jobs
             will be added - basically the same behaviour as doing Jobs.load().
-        stats (JobStatistics):
+        stats (pyslurm.db.JobStatistics):
             Real-time statistics of all Jobs in this collection.
             Before you can access the stats data for this, you have to call
             the `load_stats` method on this collection.
@@ -120,12 +120,12 @@ cdef class Job:
             An Integer representing a Job-ID.
 
     Attributes:
-        steps (JobSteps):
+        steps (pyslurm.JobSteps):
             Steps this Job has.
             Before you can access the Steps data for a Job, you have to call
             the `reload()` method of a Job instance or the `load_steps()`
             method of a Jobs collection.
-        stats (JobStatistics):
+        stats (pyslurm.db.JobStatistics):
             Real-time statistics of a Job.
             Before you can access the stats data for a Job, you have to call
             the `load_stats` method of a Job instance or the Jobs collection.
@@ -380,6 +380,9 @@ cdef class Job:
             Amount of CPU-Time used by the Job so far.
             This is the result of multiplying the run_time with the amount of
             cpus requested.
+        run_time_remaining (int):
+            The amount of seconds the job has still left until hitting the
+            `time_limit`.
     """
     cdef:
         slurm_job_info_t *ptr