Jobbergate

https://img.shields.io/pypi/v/jobbergate.svg https://travis-ci.com/HeMan/jobbergate.svg?branch=master Documentation Status

Jobbergate is a questionnaire application that populates Jinja2 templates with given answers.

In its simplest form you only need a views.py that defines mainflow and a template file (called templates/job_template.j2) which gets populated with your answers. To support advanced workflows you could define multiple levels of questions, change to other templates, run functions before and after subworkflows, have follow up questions to boolean questions and so on.

To install, just do:

pip install jobbergate

Configure jobbergate.yaml to point to your directory where you have all applications. Set JOBBERGATE_PATH environment to point to where your jobbergate.yaml resides.

Jobbergate is a Flask application but could be run both as a web application and as a cli application.

To run as web application, just do:

flask run

To run as cli application, you can find out which applications it has in its configuration directory with:

flask --help

If you have an application called simple you run it with:

flask simple outputfile.sh

This will populate the simple application template with the answers you give in the following interactive session, and create outputfile.sh.

If you want the output file to be run in bash automatically, you may explicitly give the command in your implemented application. For example, if you define a function in your application’s controller.py such as:

@workflow.logic
def post_generic(data):
    retval = {"cmd_command":f"cat {data['filename']}"}
    return retval

the application will run:

cat outputfile.sh

which shows the content of the output file. This feature can be suppressed by using the ‘–no-cmd’ flag:

flask simple outputfile.sh --no-cmd

Workflow

Simple workflow

A simple workflow is implemented with the function mainflow defined in views.py and a template defined in templates/job_template.j2:

+-- views.py
+-+ templates/
  + job_template.j2

views.py:

from jobbergate import appform

def mainflow(data):
    return [appform.Text("jobname", "What is the jobname?", default="simulation")]

job_template.j2:

#!/bin/bash
#SBATCH -j {{ data.jobname }}
sleep 30

Workflow with implicit workflows

A workflow with implicit workflows is built by defining mainflow and functions decorated with appform.workflow:

+-- views.py
+-+ templates/
  + job_template.j2

views.py:

from jobbergate import appform

def mainflow(data):
    return [appform.Text("jobname", "What is the jobname?", default="simulation")]

@appform.workflow
def debug(data):
    return [appform.Confirm("debug", "Add debug info?")]

@appform.workflow
def gpu(data):
    return [appform.Integer("gpus", "Number of gpus?", default=1, maxval=10)]

job_template.j2:

#!/bin/bash
#SBATCH -j {{ data.jobname }}


{% if data.gpus %}
NUMBER_OF_GPUS={{ data.gpus }}
{% else %}
NUMBER_OF_GPUS=0
{% endif %}

{% if data.debug %}
/application/debug_prepare
{% endif %}

/application/run_application -gpus $NUMBER_OF_GPUS

Indices and tables