Console Scripts¶
This software is primarily intended to be a library, however some management tasks are just easier from the console. For that reason, fyrd has a frontend console script that makes tasks such as managing the local config and profiles trivial, it also has modes to inspect the queue easily, and to wait for jobs from the console, as well as to clean the working directory.
fyrd¶
This software has uses a subcommand system to separate modes, and has six modes:
- config — show and edit the contents of the config file
- profile - inspect and manage cluster profiles
- keywords - print a list of current keyword arguments with descriptions for each
- queue - show running jobs, makes filtering jobs very easy
- wait - wait for a list of jobs
- clean - clean all script and output files in the given directory
Several of the commands have aliases (conf and prof being the two main ones)
Examples¶
fyrd prof list
fyrd prof add large cores:92 mem:200GB partition:high_mem time:00:06:00
fyrd queue # Shows all of your current jobs
fyrd queue -a # Shows all users jobs
fyrd queue -p long -u bob dylan # Show all jobs owned by bob and dylan in the long queue
fyrd wait 19872 19876
fyrd wait -u john
fyrd clean
All Options¶
Manage fyrd config, profiles, and queue.
| Author | Michael D Dacre <mike.dacre@gmail.com> |
| Organization | Stanford University |
| License | MIT License, use as you wish |
| Version | 0.6.2-beta.6 |
usage: fyrd [-h] [-v] {conf,prof,keywords,queue,wait,clean} ...
- Options:
-v=False, --verbose=False Show debug outputs - Sub-commands:
- conf (config)
View and manage the config
This script allows display and management of the fyrd config file found here: /home/docs/.fyrd/config.txt.
usage: fyrd conf [-h] {show,list,help,update,alter,init} ...
- Sub-commands:
- show (list)
Show current config
By default shows the current config, which is based on the config file, but with all parsing modifications applied, meaning that extra options or sections are ignored. To print the file as is, pass –file, or to limit to a single section, pass –section <section>
usage: fyrd conf show [-h] [-s [...]] [-f]
- Options:
-s, --sections Limit results to a list of sections
Possible choices: jobs, queue, jobqueue
-f=False, --file=False Print file contents only
- help
Show info on every config option
usage: fyrd conf help [-h] [-s [...]]
- Options:
-s, --sections Limit results to a list of sections
Possible choices: jobs, queue, jobqueue
- update (alter)
Update the config
Update one option at a time. Sections: {‘jobs’, ‘queue’, ‘jobqueue’}
usage: fyrd conf update [-h] section option value
- Positional arguments:
section Section to update
Possible choices: jobs, queue, jobqueue
option Option to update
Possible choices: max_jobs, filepath, profile_file, queue_update, file_block_time, queue_type, suffix, jobno, clean_files, sleep_len, clean_outputs, generic_python, auto_submit, res_time
value New value for option
- init
Interactively initialize the config
Overwrites the exiting config and starts from scratch. Asks for input from user for default options.
usage: fyrd conf init [-h] [--defaults] [--yes]
- Options:
--defaults=False Non-interactive, just use builtin defaults --yes=True Do not as for confirmation
- prof (profile)
Manage profiles
Fyrd jobs use keyword arguments to run (for a complete list run this script with the keywords command). These keywords can be bundled into profiles, which are kept in /home/docs/.fyrd/profiles.txt. This file can be edited directly or manipulated here.
usage: fyrd prof [-h] {show,list,add,new,update,alter,edit,remove-option,del-option,delete,del} ...
- Sub-commands:
- show (list)
Print current profiles
usage: fyrd prof show [-h] [-n NAME [NAME ...]]
- Options:
-n, --name Limit to only these profiles
- add (new)
Add a new profile
usage: fyrd prof add [-h] name options [options ...]
- Positional arguments:
name Profile name options Options to update
- update (alter, edit)
Update an existing profile
usage: fyrd prof update [-h] name options [options ...]
- Positional arguments:
name Profile name options Options to update
- remove-option (del-option)
Remove a profile option
usage: fyrd prof remove-option [-h] name options [options ...]
- Positional arguments:
name Profile name options Options to remove
- delete (del)
Delete an existing profile
usage: fyrd prof delete [-h] name
- Positional arguments:
name Profile name
- keywords (keys, options)
Print available keyword arguments.
usage: fyrd keywords [-h] [-t | -s | -l]
- Options:
-t=False, --table=False Print keywords as a table -s=False, --split-tables=False Print keywords as multiple tables -l=False, --list=False Print a list of keywords only
- queue (q)
Search the queue
Check the local queue, similar to squeue or qstat but simpler, good for quickly checking the queue. By default it searches only your own jobs, pass ‘–all-users’ or ‘–users <user> [<user2>...]’ to change that behavior. To just list jobs with some basic info, run with no arguments.
usage: fyrd queue [-h] [-u [...] | -a] [-p [...]] [-r | -q | -d | -b] [-l | -c]
- Options:
-u, --users Limit to these users -a=False, --all-users=False Display jobs for all users -p, --partitions Limit to these partitions (queues) -r=False, --running=False Show only running jobs -q=False, --queued=False Show only queued jobs -d=False, --done=False Show only completed jobs -b=False, --bad=False Show only completed jobs -l=False, --list=False Print job numbers only, works well with xargs -c=False, --count=False Print job count only
- wait
Wait for jobs
Wait on a list of jobs, block until they complete.
usage: fyrd wait [-h] [-u USERS] [jobs [jobs ...]]
- Positional arguments:
jobs Job list to wait for - Options:
-u, --users A comma-separated list of users to wait for
- clean
Clean up a job directory
Clean all intermediate files created by the cluster module from this dir. Uses the fyrd.job.clean_dir() function Caution: The clean() function will delete **EVERY** file with extensions matching those these:: .<suffix>.err .<suffix>.out .<suffix>.sbatch & .fyrd.script for slurm mode .<suffix>.qsub for torque mode .<suffix> for local mode _func.<suffix>.py _func.<suffix>.py.pickle.in _func.<suffix>.py.pickle.out
usage: fyrd clean [-h] [-o] [-d DIR] [-s SUFFIX] [-q {torque,slurm,local}] [-n]
- Options:
-o=False, --outputs=False Clean output files too -d, --dir Directory to clean -s=cluster, --suffix=cluster Suffix to use for cleaning -q, --qtype Limit deletions to this qtype
Possible choices: torque, slurm, local
-n=True, --no-confirm=True Do not confirm before deleting (for scripts)
aliases¶
Several shell scripts are provided in bin/ to provide shortcuts to the fyrd subcommands:
- my-queue (or myq): fyrd queue
- clean-job-files: fyrd clean
- monitor-jobs: fyrd wait
- cluster-keywords: fyrd keywords