Test commands

When running commands, there are various ways to test if the command succeeded and ran as expected.

You can also use tests to update the context for subsequent commands based on the output of a command.

Note

You can specify multiple tests as well (e.g. test the output and check if a port was opened).

Test the status code

structured-tutorials will abort a tutorial if a specified command does not exit with a status code of 0. To check for a different status code, simply specify status_code:

parts:
  - commands:
      # Do not fail the tutorial if this command returns exit code 1.
      - command: "false"
        run:
          status_code: 1

Test output of the command

You can test either the standard output or standard error stream of a command with a regular expression. You can also test line and character count. Regular expressions with named patterns can be used to update the context for later commands:

Configuration

docs/tutorials/test-output/tutorial.yaml

configuration:
  doc:
    context:
      timestamp: "<timestamp>"
parts:
  - commands:
      - command: date +%s
        doc:
          output: "{{ timestamp }}"
        run:
          show_output: false
          test:
            # Test that stdout matches the given regular expression.
            - regex: ^(?P<timestamp>\d*)$
              # Optional: test stderr instead
              #stream: stderr

              # Both line and character count can be tested with an integer (= exact match)
              # or a tuple (from/to, with either optionally null):
              line_count: 1
              character_count: [ 8, 11 ]  # or [null, 11] == up to 11 characters
              # Optional: do not strip output before testing
              #strip: false


        # If named patterns are tested, you can use matches in subsequent commands
      - command: "echo \"It is now: {{ timestamp }}\""
        doc:
          output: "It is now: {{ timestamp }}"

Documentation

user@host:~$ date +%s
<timestamp>
user@host:~$ echo "It is now: <timestamp>"
It is now: <timestamp>

Run a test command

You can run a test command to verify that the command actually ran successfully:

docs/tutorials/test-command/tutorial.yaml

parts:
  - commands:
      - command: touch test.txt
        run:
          test:
            # Verify that file was indeed created
            - command: test -e test.txt
          cleanup:
            - command: rm test.txt

Note that you can also retry commands, as shown for ports below.

Test port

You can test that a port is opened by the command:

docs/tutorials/test-port/tutorial.yaml

parts:
  - commands:
      # TODO: retrieve pid from stdout
      # Start an ncat echo server, but have a delay of three seconds
      - command: "sleep 3s && ncat -e /bin/cat -k -l 1234 &"
        run:
          test:
            # test if the command succeeded by connecting to that port.
            - host: localhost
              port: 1234
              delay: 2  # wait for three seconds before even trying
              retry: 3  # retry up to three times (command will be run up to four times)
              backoff_factor: 1  # wait for 1, 2 and 4 seconds before retries
          cleanup:
            - command: pkill sleep
            - command: pkill ncat

For ports and commands, you can also specify a retry to run the test command multiple times before the main command is considered to have failed. A delay will delay the first run of the command and a backoff_factor will introduce an increasing delay between retries. A common use case is a daemon that will eventually open a port, but subsequent commands want to connect to that daemon.