Skip to content

Home

Jump to bottom
Valvanuz Fernandez edited this page Nov 23, 2021 · 11 revisions

DRM4G

DRM4G is an open platform, based on GridWay, to define, submit, and manage computational jobs. DRM4G is a Python (3.5+) implementation that provides a single point of control for computing resources without installing any intermediate middlewares. As a result, a user is able to run the same job on laptops, desktops, workstations, clusters, supercomputers, and any grid.

Why DRM4G?

  • Ready-to-run: The DRM4G bundle can be unpacked and then ran on any on any Linux OS.

  • Huge amount of jobs: DRM4G is able to manage around 1,000,000 jobs at the same time.

  • Different identities on different resources: DRM4G handles different user accounts on various resources through a single interface.

  • One command: In order to make easier to use DRM4G, there is only one command available, which is drm4g.

What is DRM4G's architecture?

Infrastructure layer consists of the following components :

  • DRM4G CLI offers users commands to submit, cancel, and monitor jobs and configure resources.

  • !GridWay core is in charge of job execution and resource brokering. DRM4G takes the most of !GridWay by using its core.

  • DRM4G Middleware Access Driver (MAD) gives access to computing resources. In order to do that, there are two kind of components:

    • Resource Manager is responsible for collecting resource information.
    • Communicator provides the authentication type to access them.

  • DRM4G Sched is responsible for scheduling jobs.

New releases

Releases

{{{#!comment TOC(DRM4G/change_log/*, depth=2, inline, noheading) }}} {{{ #!html

· DRM4G Release 2.6.4 ·&nbsp&nbsp 2017-03-03 &nbsp&nbsp·
· DRM4G Release 2.6.3 ·&nbsp&nbsp 2017-02-09 &nbsp&nbsp·
· DRM4G Release 2.6.2 ·&nbsp&nbsp 2017-01-12 &nbsp&nbsp·
}}}

[./ChangeLog List of all releases]

\ {{{#!comment -- https://meteo.unican.es/trac/wiki/DRM4G1/Getting_started unsused }}}

Getting started

{{{#!comment TOC(DRM4G/Installation, noheading, inline, depth=2) }}}

  • [./Installation Installation]

  • [./Installation#Withpip With pip]

  • [./Installation#Withoutpip Without pip]

  • [./Installation#Verifyingtheinstallation Verifying the installation]

  • [./Installation#OptionalEnvironmentVariables Optional Environment Variables]

  • [./Installation#InstallingExtras Installing Extras]

  • [./ResourceConfiguration Resource Configuration]

  • [./ResourceConfiguration#Configurationformat Configuration format]

  • [./ResourceConfiguration#DEFAULTsection DEFAULT section]

  • [./ResourceConfiguration#Resourcesection Resource section]

  • [./ResourceConfiguration#Examples Examples]

{{{#!comment

  • [https://meteo.unican.es/trac/wiki/DRM4G1/example_usage Example of Usage]

  • We can have it like this or ,as I prefer, a separate section as shown bellow (which makes it's sections more accessible) }}}

  • [./JobDescription Job Description]

  • [./JobDescription#Syntax Syntax]

  • [./JobDescription#Templateoptions Template options]

  • [./JobDescription#RequirementExpressions Requirement Expressions]

  • [./JobDescription#EnvironmentExpressions Environment Expressions]

Examples of usage

This section contains examples on how to use the DRM4G CLI.

  • [./ResourceConfiguration/TorquePBSResource How to configure a TORQUE/PBS resource by using the private/public key without password]

  • [./ResourceConfiguration/EGIESR How to configure an EGI ESR VM]

  • [./ResourceConfiguration/IBERGRID How to configure an IBERGRID VM]

  • [./ResourceConfiguration/EGIFedCloud How to configure an EGI FedCloud/rOCCI VM]

  • [./ManageJob How to manage a job from start to end]

DRM4G CLI

drm4g is DRM4G's Command Line Interface (CLI) for administrating both computing resources and jobs.

  • [./AvailableCommands Available Commands]

Python API

In order to use the DRM4G API, you only have to install DRM4G on your machine. This python API allows you to create, submit, cancel and monitor DRM4G jobs.

  • [./API DRM4G API]
  • [./API#Howtocreateajob How to create a job]
  • [./API#Howtosubmitajob How to submit a job]
  • [./API#Howtomonitorajob How to monitor a job]

Tutorial

  • [./Tutorial DRM4G tutorial]
  • [./Tutorial#StarttorunDRM4G Start to run DRM4G]
  • [./Tutorial#Myfirstjob My first job]
  • [./Tutorial#HowtoconfigureaTORQUEPBSresource How to configure a TORQUE/PBS resource using the ssh-agent]
  • [./Tutorial#UserScenarios User Scenarios]
  • [https://www.youtube.com/watch?v=sgc7oq9jo1o Video from presentation given by C. Blanco at HPCKP'15 ]

Contributing

  • [./Development]

FAQ

  • [./FAQ Frequently Asked Questions]
  • [./FAQ#CommonErrors Common Errors]
Clone this wiki locally