Tutorial

Acknowledgement: this tutorial is inspired by and largely copied from the Python Argparse Tutorial by Tshepang Lekhonkhobe, made available under the Zero Clause BSD License.

This tutorial is intended to be an introduction to the lower-level interface of argparse. This interface deals with building command line parsers, and is generated by the higher-level annotation-based interface. As such, it is helpful to understand these concepts, even if you only use the higher-level interface.

Concepts

Let's show the sort of functionality that we are going to explore in this introductory tutorial by making use of the ls command:

$ ls
argparse  build.sc  ci  examples  ini  LICENSE.md  mill  out  README.md
$ ls argparse
src  src-2  src-3  test
$ ls -l
total 48
drwxr-xr-x 9 jodersky jodersky  4096 Jul  3 10:12 argparse
-rw-r--r-- 1 jodersky jodersky  4836 Jul  3 15:30 build.sc
drwxr-xr-x 2 jodersky jodersky  4096 Jan 31 18:05 ci
drwxr-xr-x 5 jodersky jodersky  4096 Jul  3 15:07 examples
drwxr-xr-x 4 jodersky jodersky  4096 May 20 19:58 ini
-rw-r--r-- 1 jodersky jodersky  1473 Apr 30  2020 LICENSE.md
-rwxr-xr-x 1 jodersky jodersky  1646 Mar  7  2021 mill
drwxr-xr-x 5 jodersky jodersky  4096 Jul  3 15:54 out
-rw-r--r-- 1 jodersky jodersky 10090 Jul  3 17:10 README.md
$ ls --help
Usage: ls [OPTION]... [FILE]...
List information about the FILEs (the current directory by default).
Sort entries alphabetically if none of -cftuvSUX nor --sort is specified.
...

A few concepts we can learn from the four commands:

  • The ls command is useful when run without any arguments at all. It defaults to displaying the contents of the current directory.

  • If we want beyond what it provides by default, we tell it a bit more. In this case, we want it to display a different directory, argparse. What we did is specify what is known as a positional argument. It's called so because the program should know what to do with the value, solely based on where it appears on the command line. This concept is more relevant to a command like cp, whose most basic usage is cp SRC DEST. The first position is what you want copied, and the second position is where you want it copied to.

  • Now, say we want to change behaviour of the program. In our example, we display more info for each file instead of just showing the file names. The -l in that case is known as a named argument.

  • That's a snippet of the help text. It's very useful in that you can come across a program you have never used before, and can figure out how it works simply by reading its help text.

These concepts are core to argparse:

parameter : a named variable, a placeholder, in a command line definition

argument : the value assigned to a parameter

named argument : an argument that starts with -. The characters following determine the name of the parameter that the argument is assigned to. The actual value assigned to the parameter is given after an '=' or a space. For instance --foo=bar assigns bar to foo. Named arguments may appear in any order on a command line.

positional argument : an argument that is not named. Positional arguments are assigned to positional parameters according to their respective order of occurence.

Basics

Let's start with an example that does almost nothing:

def main(args: Array[String]): Unit =
  val parser = argparse.default.ArgumentParser()
  parser.parseOrExit(args)

The following is a result of running the code:

$ app


$ app --help
Usage: [OPTIONS]
Options:
      --bash-completion string  generate bash completion for this command
      --help                    show this message and exit

$ app --verbose
unknown argument: --verbose
run with '--help' for more information

$ app foo
unknown argument: foo
run with '--help' for more information

Here is what is happening:

  • Running the program without any arguments results in nothing displayed to stdout. Not so useful.

  • The second one starts to display the usefulness of the argparse library. We have done almost nothing, but already we get a help message.

  • The --help option is the only option we get for free (i.e. no need to specify it). Specifying anything else results in an error. But even then, we do get a useful usage message, also for free.

Required Parameters

An example:

def main(args: Array[String]): Unit =
  val parser = argparse.default.ArgumentParser()
  val echo = parser.requiredParam[String]("echo")
  parser.parseOrExit(args)
  println(echo.value)

And running the code:

$ app
missing argument: echo
run with '--help' for more information

$ app --help
Usage: [OPTIONS] ECHO
Options:
      --bash-completion string  generate bash completion for this command
      --help                    show this message and exit

$ app foo
foo

Here is what's happening:

  • We've added the requiredParam() method, which is what we use to specify which command-line arguments the program needs. In this case, I've named it echo so that it's in line with its function.

    The result of this method is a holder to some future value of an argument, which can be accessed by calling the .value method.

  • Calling our program now requires us to specify an argument.

  • We can specify how an argument should be read from the command line by specifying the type of the parameter, String in this case.

  • The parser.parseOrExit() method is what actually goes through the command line arguments and sets the argument holders' values.

    After calling this method, the arguments can be accessed via the .value method of the argument holders.

Note however that, although the help display looks nice and all, it currently is not as helpful as it can be. For example we see that we got echo as a positional argument, but we don't know what it does, other than by guessing or by reading the source code. So, let's make it a bit more useful:

def main(args: Array[String]): Unit =
  val parser = argparse.default.ArgumentParser()
  val echo: argparse.Argument[String] = parser.requiredParam[String](
    "echo",
    help = "echo the string you use here"
  )
  parser.parseOrExit(args)
  print(echo.value)

And we get:

$ app --help
Usage: [OPTIONS] ECHO
Options:
      --bash-completion string  generate bash completion for this command
      --help                    show this message and exit

Now, how about doing something even more useful:

def main(args: Array[String]): Unit =
  val parser = argparse.default.ArgumentParser()
  val square = parser.requiredParam[Int](
    "square",
    help = "display a square of a given number"
  )
  parser.parseOrExit(args)
  print(square.value * square.value)

Following is a result of running the code:

$ app 4
16

$ app four
error processing argument square: 'four' is not an integral number
run with '--help' for more information

That went well. The program now even helpfully quits on bad input.

Optional Parameters

So far the parameters that we have specified were required. Let's look at how we can make an argument optional.

An example:

def main(args: Array[String]): Unit =
  val parser = argparse.default.ArgumentParser()
  val answer = parser.param[Int](
    "answer",
    default = 42,
    help = "display the answer to the universe"
  )
  parser.parseOrExit(args)
  println(answer.value)

Following is a result of running the code:

$ app
42

$ app 1000
1000

Here is what's happening:

  • The parameter is made optional by declaring it with the param() method instead of the requiredParam() method.

  • This method requires a default value.

  • The default value will be used if the argument is not encountered on the command-line.

Repeated Parameters

An example:

def main(args: Array[String]): Unit =
  val parser = argparse.default.ArgumentParser()
  val files = parser.repeatedParam[os.FilePath](
    "files",
    help = "remove these files"
  )
  parser.parseOrExit(args)
  for (file <- files.value) {
    println(s"if this were a real program, we would delete $file")
  }

Following is a result of running the code:

$ app


$ app file1
if this were a real program, we would delete file1

$ app file1 file2 file3
if this were a real program, we would delete file1
if this were a real program, we would delete file2
if this were a real program, we would delete file3

Here is what's happening:

  • The parameter is made repeated by declaring it with the repeatedParam().

  • Repeated parameters accumulate all ocurences in the argument holder.

    Thus, here, value will give us back a Seq[os.FilePath] rather than a single os.FilePath.

Named Parameters and Positional Parameters

So far you may have noticed that all our examples have used positional parameters. Recall from the initial ls example, that a positional parameter is one which is set solely based on the position of its argument. When parameter lists get very long or change over time, it can become very difficult to keep matching argument lists coherent. Therefore, most command line tools will mostly use named parameters.

An example:

def main(args: Array[String]): Unit =
  val parser = argparse.default.ArgumentParser()
  val verbosity = parser.param[Int](
    "--verbosity",
    default = 0,
    help = "level of verbosity"
  )
  val files = parser.repeatedParam[java.nio.file.Path](
    "files",
    help = "remove these files"
  )
  parser.parseOrExit(args)
  println(s"verbosity: ${verbosity.value}")
  for (file <- files.value) {
    println(s"if this were a real program, we would delete $file")
  }

Following is a result of running the code:

$ app --verbosity
error processing argument --verbosity: argument expected
run with '--help' for more information

$ app --verbosity 1
verbosity: 1

$ app --verbosity=1
verbosity: 1

$ app --verbosity 1 file1 file2
verbosity: 1
if this were a real program, we would delete file1
if this were a real program, we would delete file2

$ app file1 --verbosity 1 file2
verbosity: 1
if this were a real program, we would delete file1
if this were a real program, we would delete file2

$ app file1 -- --verbosity 1 file2
verbosity: 0
if this were a real program, we would delete file1
if this were a real program, we would delete --verbosity
if this were a real program, we would delete 1
if this were a real program, we would delete file2

Here is what is happening:

  • A named parameter is a parameter that is identified by a name on the command line instead of a position. Named arguments are syntactically distinguished from positional arguments by a leading - (or -- as is common).

  • Arguments to named parameters can be either separated by a space or an equals sign.

  • Named arguments may appear in any order on the command line. They can be intermingled with positional arguments.

  • A standalone -- serves as a delimiter, and allows arguments that start with - to be treated as positionals. This is very handy for accepting untrusted input in scripts, or deleting files that start with a hyphen.

All parameter declaration methods, param(), requiredParam() and repeatedParam() allow defining parameters as positional and named. The only hint you can see is in the name: if it starts with '-', then it is a named parameter, otherwise it is positional. Note however, that it is most common and good practice to only use positional parameters for required parameters, or conversely, always make named parameters optional.

Flags

Flags are a special kind of named parameter. They don't take an argument, and we are only ever interested if they are present on the command-line or not.

An example:

def main(args: Array[String]): Unit =
  val parser = argparse.default.ArgumentParser()
  val verbose = parser.param[Boolean](
    "--verbose",
    default = false,
    help = "use verbose output",
    flag = true
  )
  parser.parseOrExit(args)
  println(verbose.value)

Running it:

$ app
false

$ app --verbose
true

$ app --verbose=false
false

How it works:

  • A parameter is declared as a flag by setting the flag parameter,

    This instructs the argument parser that the parameter does not take an argument.

  • If the flag is encountered on the command line, then it is assigned the string value "true".

    Thus, it only makes sense to declare Boolean parameters as flags.

  • You can still override the argument by explicitly passing it after an equals sign.

Short Named Parameters and Aliases

If you are familiar with command line usage, you will notice that I haven't yet touched on the topic of short versions of named parameters. It's quite simple:

def main(args: Array[String]): Unit =
  val parser = argparse.default.ArgumentParser()
  val verbose = parser.param[Boolean](
    "--verbose",
    default = false,
    aliases = Seq("-v", "--talkative"),
    flag = true,
    help = "use verbose output"
  )
  parser.parseOrExit(args)
  println(verbose.value)

And here goes:

$ app -v
true

$ app --help
Usage: [OPTIONS]
Options:
      --bash-completion string  generate bash completion for this command
      --help                    show this message and exit
  -v, --verbose                 use verbose output

Note that the new aliases are also reflected in the help text.

Reading from the Environment

In some cases it can be useful to fall back to reading a command line from an environment variable.

Example:

def main(args: Array[String]): Unit =
  val parser = argparse.default.ArgumentParser()
  val creds = parser.param[os.Path](
    "--credentials-file",
    default = os.root / "etc" / "creds",
    env = "APP_CREDENTIALS_FILE",
    help = "the file containing service credentials"
  )
  parser.parseOrExit(args)
  println(creds.value)

Run it:

$ APP_CREDENTIALS_FILE=/etc/foo app
/etc/foo

$ APP_CREDENTIALS_FILE=/etc/foo2 app --credentials-file=/etc/foo1
/etc/foo1

$ app
/etc/creds

How it works:

  • Required and optional parameter declarations can specify an env, which will name an environment variable to use if the argument cannot be found on the command line.

  • The order of precedence is:

    1. the argument on the command line
    2. the environment variable
    3. the default value