clitest-0.0+git20150312.339b2d/0000755000000000000000000000000012500322604014012 5ustar rootrootclitest-0.0+git20150312.339b2d/.gitignore0000644000000000000000000000002312500322604015775 0ustar rootroot/dev/other/ /docs/ clitest-0.0+git20150312.339b2d/README.md0000644000000000000000000003065712500322604015304 0ustar rootroot# clitest – Command Line Tester clitest is a [portable][1] POSIX shell script that performs automatic testing in Unix command lines. It's the same concept as in Python's [doctest][2] module: you document both the commands and their expected output, using the familiar interactive prompt format, and a specialized tool tests them. In fact, the doctest [official][3] description can also be used for clitest: * The **doctest** module searches for pieces of text that look like interactive **Python sessions**, and then executes those **sessions** to verify that they work exactly as shown. * The **clitest** command searches for pieces of text that look like interactive **Unix command lines**, and then executes those **command lines** to verify that they work exactly as shown. ## Download The full program is just [a single shell script file][4]. Save it and make it executable: `chmod +x clitest` ## Quick Intro Save the commands and their expected output in a text file: ♦ [examples/intro.txt][5] ``` $ echo "Hello World" Hello World $ cd /tmp $ pwd /tmp $ cd "$OLDPWD" $ ``` Use clitest to run these commands and check their output: ``` $ clitest examples/intro.txt #1 echo "Hello World" #2 cd /tmp #3 pwd #4 cd "$OLDPWD" OK: 4 of 4 tests passed $ ``` ## CLI Syntax There's no syntax to learn. The test files are identical to the good old command line interface (CLI) you're so familiar: ♦ [examples/cut.txt][6] ``` $ echo "one:two:three:four:five:six" | cut -d : -f 1 one $ echo "one:two:three:four:five:six" | cut -d : -f 4 four $ echo "one:two:three:four:five:six" | cut -d : -f 1,4 one:four $ echo "one:two:three:four:five:six" | cut -d : -f 4,1 one:four $ echo "one:two:three:four:five:six" | cut -d : -f 1-4 one:two:three:four $ echo "one:two:three:four:five:six" | cut -d : -f 4- four:five:six $ ``` That's it. Just paste your shell session inside a text file and you have a ready-to-use test suite. ``` $ clitest examples/cut.txt #1 echo "one:two:three:four:five:six" | cut -d : -f 1 #2 echo "one:two:three:four:five:six" | cut -d : -f 4 #3 echo "one:two:three:four:five:six" | cut -d : -f 1,4 #4 echo "one:two:three:four:five:six" | cut -d : -f 4,1 #5 echo "one:two:three:four:five:six" | cut -d : -f 1-4 #6 echo "one:two:three:four:five:six" | cut -d : -f 4- OK: 6 of 6 tests passed $ ``` ## Test Documents Ever wanted to test the command line instructions you give in the `INSTALL.txt` or `README.md` files for your projects? Now you can! clitest can also extract and run command lines from technical documents. Given the following Markdown sample document, which uses tabs to mark code blocks: ♦ [examples/cut.md][7] ``` The numeric ranges of the Unix command "cut" ============================================ Use single numbers to extract one specific field: $ echo "one:two:three:four:five:six" | cut -d : -f 1 one $ echo "one:two:three:four:five:six" | cut -d : -f 4 four Use commas to inform more than one field: $ echo "one:two:three:four:five:six" | cut -d : -f 1,4 one:four Note that inverting the order will *not* invert the output: $ echo "one:two:three:four:five:six" | cut -d : -f 4,1 one:four Use an hyphen to inform a range of fields, from one to four: $ echo "one:two:three:four:five:six" | cut -d : -f 1-4 one:two:three:four If you omit the second range number, it matches until the last: $ echo "one:two:three:four:five:six" | cut -d : -f 4- four:five:six cut is cool, isn't it? ``` It's easy to convert it to a readable HTML document with your favorite Markdown program. It's also easy to test this file directly with clitest, just inform that the command lines are prefixed by a tab: ``` $ clitest --prefix tab examples/cut.md #1 echo "one:two:three:four:five:six" | cut -d : -f 1 #2 echo "one:two:three:four:five:six" | cut -d : -f 4 #3 echo "one:two:three:four:five:six" | cut -d : -f 1,4 #4 echo "one:two:three:four:five:six" | cut -d : -f 4,1 #5 echo "one:two:three:four:five:six" | cut -d : -f 1-4 #6 echo "one:two:three:four:five:six" | cut -d : -f 4- OK: 6 of 6 tests passed $ ``` For Markdown files with 4-spaces indented code blocks, use `--prefix 4`. Of course, this [README.md][8] file you are now reading is also testable. Since it uses non-indented fenced code blocks (\`\`\`), no prefix option is needed: `clitest README.md`. ## Alternative Syntax: Inline Output Now a nice extension to the original idea. Using the special marker `#→` you can embed the expected command output at the end of the command line. ``` $ echo "foo" #→ foo $ echo $((10 + 2)) #→ 12 ``` This is the same as doing: ``` $ echo "foo" foo $ echo $((10 + 2)) 12 $ ``` Inline outputs are very readable when testing series of commands that result in short texts. ``` $ echo "abcdef" | cut -c 1 #→ a $ echo "abcdef" | cut -c 4 #→ d $ echo "abcdef" | cut -c 1,4 #→ ad $ echo "abcdef" | cut -c 1-4 #→ abcd ``` > Note: The Unicode character `→` (U+2192) was chosen because it's > meaningful and less likely to appear on a real command. If needed, > you can change this marker (i.e., to `#->`) at the top of the script > or using the `--inline-prefix` option. ## Advanced Tests When using the `#→` marker, you can take advantage of special options to change the default output matching method. ``` $ head /etc/passwd #→ --lines 10 $ tac /etc/passwd | tac #→ --file /etc/passwd $ cat /etc/passwd #→ --egrep ^root: $ echo $((2 + 10)) #→ --regex ^\d+$ $ pwd #→ --eval echo $PWD ``` * Using `#→ --lines` the test will pass if the command output has exactly `N` lines. Handy when the output text is variable (unpredictable), but the number of resulting lines is constant. * Using `#→ --file` the test will pass if the command output matches the contents of an external file. Useful to organize long/complex outputs into files. * Using `#→ --egrep` the test will pass if `egrep` matches at least one line of the command output. * Using `#→ --regex` the test will pass if the command output is matched by a [Perl regular expression][9]. A multiline output is matched as a single string, with inner `\n`'s. Use the `(?ims)` modifiers when needed. * Using `#→ --eval` the test will pass if both commands result in the same output. Useful to expand variables which store the full or partial output. ## Options ``` $ clitest --help Usage: clitest [options] Options: -1, --first Stop execution upon first failed test -l, --list List all the tests (no execution) -L, --list-run List all the tests with OK/FAIL status -t, --test RANGE Run specific tests, by number (1,2,4-7) -s, --skip RANGE Skip specific tests, by number (1,2,4-7) --pre-flight COMMAND Execute command before running the first test --post-flight COMMAND Execute command after running the last test -q, --quiet Quiet operation, no output shown -V, --version Show program version and exit Customization options: -P, --progress TYPE Set progress indicator: test, number, dot, none --color WHEN Set when to use colors: auto, always, never --diff-options OPTIONS Set diff command options (default: '-u') --inline-prefix PREFIX Set inline output prefix (default: '#→ ') --prefix PREFIX Set command line prefix (default: '') --prompt STRING Set prompt string (default: '$ ') $ ``` When running sequential tests, where the next test depends on the correct result of the previous test, use the `--first` option to abort the execution if any test fails. To rerun a specific problematic test, or to limit the execution to a set of tests, use `--test`. To ignore one or more tests, use `--skip`. If needed, you can combine both options to inform a very specific test range. Examples: clitest --test 1-10 tests.txt # Run the first 10 tests clitest --test 1,2,6-8 tests.txt # Run tests #1, #2, #6, #7 and #8 clitest --skip 11,15 tests.txt # Run all tests, except #11 and #15 clitest -t 1-10 -s 5 tests.txt # Run first 10 tests, but skip #5 You can run a preparing script or command before the first test with `--pre-flight`, for setting env variables and create auxiliary files. At the end of all tests, run a final cleanup script/command with `--post-flight` to remove temporary files or other transient data. Example: clitest --pre-flight ./test-init.sh --post-flight 'rm *.tmp' tests.txt Use the customization options to extract and test command lines from documents or wiki pages. For example, to test all the command line examples listed inside a Markdown file using the 4-spaces syntax for code blocks: clitest --prefix 4 README.md Or maybe you use a different prompt (`$PS1`) in your documentation? clitest --prefix 4 --prompt '[john@localhost ~]$ ' README.md When automating the tests execution, use `--quiet` to show no output and just check the exit code to make sure all tests have passed. Example: if clitest --quiet tests.txt then # all tests passed else # one or more tests failed :( fi ## Nerdiness * Use any file format for the tests, it doesn't matter. The command lines just need to be grepable and have a fixed prefix (or none). Even Windows text files (CR+LF) will work fine. * The cmdline power is available in your test files: use variables, pipes, redirection, create files, folders, move around… * All the commands are tested in the same shell. Defined variables, aliases and functions will persist between tests. * Both STDIN and STDOUT are catch, you can also test error messages. * To test the exit code, just add a `;echo $?` after the command. * Use an empty `$` prompt to close the last command output. * In the output, every single char (blank or not) counts. Any difference will cause a test to fail. To ignore the difference in blanks, use `--diff-options '-u -w'`. * Unlike doctest's ``, in clitest blank lines in the command output aren't a problem. Just insert them normally. * To test outputs with no final `\n`, such as `printf foo`, use `#→ --regex ^foo$`. * In multifile mode, the current folder (`$PWD`) is reset when starting to test a new file. This avoids that a `cd` command in a previous file will affect the next. * Multiline prompts (`$PS2`) are not yet supported. * Ellipsis (as in doctest) are not supported. Use `#→ --regex` instead. * Simple examples in [examples/][10]. Hardcore examples in [dev/test.md][11] and [dev/test/][12], the clitest own test-suite. ## Portability This script was carefully coded to be portable between [POSIX][13] shells. It was tested in: * Bash 3.2 * dash 0.5.5.1 * ksh 93u 2011-02-08 Portability issues are considered serious bugs, please [report them][14]! Developers: Learn more about portability in POSIX shells: * [How to make bash scripts work in dash][15] * [Ubuntu — Dash as /bin/sh][16] * [Rich’s sh (POSIX shell) tricks][17] * [lintsh][18] * [Official POSIX specification: Shell & Utilities][19] ## [KISS][20] A shell script to test shell commands. No other language or environment involved. ## Meta * Author: [Aurelio Jargas][21] * Created: 2013-07-24 * Language: Shell Script * License: [MIT][22] [1]: #portability [2]: http://en.wikipedia.org/wiki/Doctest [3]: http://docs.python.org/3/library/doctest.html [4]: https://raw.github.com/aureliojargas/clitest/master/clitest [5]: https://github.com/aureliojargas/clitest/blob/master/examples/intro.txt [6]: https://github.com/aureliojargas/clitest/blob/master/examples/cut.txt [7]: https://github.com/aureliojargas/clitest/blob/master/examples/cut.md [8]: https://github.com/aureliojargas/clitest/blob/master/README.md [9]: http://perldoc.perl.org/perlre.html [10]: https://github.com/aureliojargas/clitest/blob/master/examples/ [11]: https://github.com/aureliojargas/clitest/blob/master/dev/test.md [12]: https://github.com/aureliojargas/clitest/blob/master/dev/test/ [13]: http://en.wikipedia.org/wiki/POSIX [14]: https://github.com/aureliojargas/clitest/issues [15]: http://mywiki.wooledge.org/Bashism [16]: https://wiki.ubuntu.com/DashAsBinSh [17]: http://www.etalabs.net/sh_tricks.html [18]: http://code.dogmap.org/lintsh/ [19]: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/contents.html [20]: http://en.wikipedia.org/wiki/KISS_principle [21]: http://aurelio.net/about.html [22]: https://github.com/aureliojargas/clitest/blob/master/LICENSE.txt clitest-0.0+git20150312.339b2d/examples/0000755000000000000000000000000012500322604015630 5ustar rootrootclitest-0.0+git20150312.339b2d/examples/hello-world.txt0000644000000000000000000000004312500322604020616 0ustar rootroot$ echo "Hello World" Hello World $ clitest-0.0+git20150312.339b2d/examples/README.md0000644000000000000000000000341012500322604017105 0ustar rootroot# clitest Examples Here's some simple examples to show you how a test file looks like. ## Pure CLI Tests Take a look at the `.txt` files. They're just like a shell session snapshot. You have the `$ ` prompt, the command to be executed, and the results. ``` $ echo "Hello World" Hello World $ ``` To test these files, just call `clitest` with no options. ``` $ clitest intro.txt #1 echo "Hello World" #2 cd /tmp #3 pwd #4 cd "$OLDPWD" OK: 4 of 4 tests passed $ ``` ### Easily create your own test files: 1. Go to your terminal 2. Set your prompt accordingly: `PS1='$ '` 3. Type and run the desired commands 4. Copy & paste it all into a text file 5. Done ## Documentation Tests Now take a look at the `.md` files. They're normal Markdown documents (with titles, paragraphs, code blocks), created to be read by humans (after HTML conversion). Inside the code blocks there are examples of command lines and their results. `clitest` can extract and run these commands for you! Now you can guarantee that all your examples are correct. ``` $ clitest --prefix tab cut.md #1 echo "one:two:three:four:five:six" | cut -d : -f 1 #2 echo "one:two:three:four:five:six" | cut -d : -f 4 #3 echo "one:two:three:four:five:six" | cut -d : -f 1,4 #4 echo "one:two:three:four:five:six" | cut -d : -f 4,1 #5 echo "one:two:three:four:five:six" | cut -d : -f 1-4 #6 echo "one:two:three:four:five:six" | cut -d : -f 4- OK: 6 of 6 tests passed $ ``` Note that since the code blocks in these Markdown documents are prefixed by a tab, you must use the `--prefix` option. Even this `README.md` file you're reading is testable. No options needed, since the code blocks here do not use prefixes. ## Play Around Run the tests, change the expected output to force a test fail, use the `--list-run` option, ... clitest-0.0+git20150312.339b2d/examples/will-fail.txt0000644000000000000000000000004212500322604020245 0ustar rootroot$ echo "Hello World" Hola Mundo $ clitest-0.0+git20150312.339b2d/examples/install-software.txt0000644000000000000000000000145712500322604021676 0ustar rootroot$ cd /tmp $ url="http://txt2tags.googlecode.com/files/txt2tags-2.6.tgz" $ curl -O -s -S "$url" # download $ du -h txt2tags-2.6.tgz # verify size 532K txt2tags-2.6.tgz $ tar tzf txt2tags-2.6.tgz | sed -n '$=' # verify number of files 545 $ tar xzf txt2tags-2.6.tgz # extract $ cd txt2tags-2.6 $ ls -1F # list contents COPYING ChangeLog README doc/ extras/ po/ samples/ test/ txt2tags* $ ./txt2tags -V # test execution txt2tags version 2.6 $ test -d ~/bin || mkdir ~/bin # create folder if needed $ cp txt2tags ~/bin/ # install to user folder $ ~/bin/txt2tags -V # final test txt2tags version 2.6 $ clitest-0.0+git20150312.339b2d/examples/intro.txt0000644000000000000000000000010712500322604017522 0ustar rootroot$ echo "Hello World" Hello World $ cd /tmp $ pwd /tmp $ cd "$OLDPWD" $ clitest-0.0+git20150312.339b2d/examples/cut.md0000644000000000000000000000161712500322604016752 0ustar rootrootThe numeric ranges of the Unix command "cut" ============================================ Use single numbers to extract one specific field: $ echo "one:two:three:four:five:six" | cut -d : -f 1 one $ echo "one:two:three:four:five:six" | cut -d : -f 4 four Use commas to inform more than one field: $ echo "one:two:three:four:five:six" | cut -d : -f 1,4 one:four Note that inverting the order will *not* invert the output: $ echo "one:two:three:four:five:six" | cut -d : -f 4,1 one:four Use an hyphen to inform a range of fields, from one to four: $ echo "one:two:three:four:five:six" | cut -d : -f 1-4 one:two:three:four If you omit the second range number, it matches until the last: $ echo "one:two:three:four:five:six" | cut -d : -f 4- four:five:six cut is cool, isn't it? > Note: To automatically test all the shell commands in this article, > just run: `clitest --prefix tab cut.md` clitest-0.0+git20150312.339b2d/examples/install-software.md0000644000000000000000000000412212500322604021447 0ustar rootroot# How to install txt2tags v2.6 > This file is an example of a technical “how to” document > that can also be automatically tested: > `clitest --prefix tab install-software.md` ## 1. Prepare First, move to the temporary directory, where we will download, extract and test the txt2tags package before installing it to the system. $ cd /tmp ## 2. Download Download the .tgz file for the version 2.6, directly from Google Code. $ url="http://txt2tags.googlecode.com/files/txt2tags-2.6.tgz" $ curl -O -s -S "$url" ## 3. Verify Let's verify if the downloaded package is not corrupted, by checking the file size and the total number of files inside the tgz. $ du -h txt2tags-2.6.tgz 532K txt2tags-2.6.tgz $ tar tzf txt2tags-2.6.tgz | sed -n '$=' 545 > Note: Using `sed` to count lines because the output format of `wc -l` > differs between implementations, regarding leading blank spaces. ## 4. Extract Since the download is ok, now we can extract the package's files. If `tar` shows no message, it's because everything went fine and all the files were extracted. $ tar xzf txt2tags-2.6.tgz A new `txt2tags-2.6` directory was created. Let's enter into it and list the main files, just to be sure. $ cd txt2tags-2.6 $ ls -1F COPYING ChangeLog README doc/ extras/ po/ samples/ test/ txt2tags* ## 5. Test The main `txt2tags` file is executable? Python is installed? Python version is compatible with the program? So many questions... But a simple command answers them all. $ ./txt2tags -V txt2tags version 2.6 If the version was shown, it's a proof that the program was run successfully: Python is installed and it's compatible. ## 6. Install By default, the program is installed in the `~/bin` user directory. Usually this directory is already there, but let's play safe and create it if necessary. $ test -d ~/bin || mkdir ~/bin The install process itself is just a simple file copy. $ cp txt2tags ~/bin/ Now just a final test, executing the program directly from `~/bin`. $ ~/bin/txt2tags -V txt2tags version 2.6 Ok, we're done. clitest-0.0+git20150312.339b2d/examples/cut.txt0000644000000000000000000000060312500322604017163 0ustar rootroot$ echo "one:two:three:four:five:six" | cut -d : -f 1 one $ echo "one:two:three:four:five:six" | cut -d : -f 4 four $ echo "one:two:three:four:five:six" | cut -d : -f 1,4 one:four $ echo "one:two:three:four:five:six" | cut -d : -f 4,1 one:four $ echo "one:two:three:four:five:six" | cut -d : -f 1-4 one:two:three:four $ echo "one:two:three:four:five:six" | cut -d : -f 4- four:five:six $ clitest-0.0+git20150312.339b2d/dev/0000755000000000000000000000000012500322604014570 5ustar rootrootclitest-0.0+git20150312.339b2d/dev/test.md0000644000000000000000000015371212500322604016102 0ustar rootroot# Test suite for clitest This is the test file for the `clitest` program. Yes, the program can test itself! This file runs all the files inside the `dev/test` folder and checks the results. The command line options are also tested. Usage: ./clitest dev/test.md ## Preparing Make sure we're on the same folder as `clitest`, since all the file paths here are relative, not absolute. ``` $ test -f ./clitest; echo $? 0 $ test -d ./dev/test/; echo $? 0 $ ``` Set a default terminal width of 80 columns. It's used by separator lines. ``` $ COLUMNS=80 $ export COLUMNS $ ``` Ok. Now the real tests begins. ## Variables are persistent between tests? ``` $ echo $COLUMNS 80 $ not_exported=1 $ echo $not_exported 1 $ echo $not_exported #→ 1 $ echo $not_exported #→ --regex ^1$ ``` ## Check the temporary dir creation ``` $ TMPDIR___SAVE="$TMPDIR" $ TMPDIR=/XXnotfoundXX $ export TMPDIR $ ./clitest dev/test/ok-1.sh 2>&1 | sed 's/clitest\.[0-9]*$/clitest.NNN/' mkdir: /XXnotfoundXX: No such file or directory clitest: Error: cannot create temporary dir: /XXnotfoundXX/clitest.NNN $ TMPDIR="$TMPDIR___SAVE" $ ``` ## I/O, file reading (message and exit code) Missing input file ``` $ ./clitest; echo $? clitest: Error: no test file informed (try --help) 2 $ ./clitest -- clitest: Error: no test file informed (try --help) $ ./clitest --list clitest: Error: no test file informed (try --help) $ ``` File not found ``` $ ./clitest XXnotfoundXX.sh; echo $? clitest: Error: cannot read input file: XXnotfoundXX.sh 2 $ ./clitest . clitest: Error: cannot read input file: . $ ./clitest ./ clitest: Error: cannot read input file: ./ $ ./clitest /etc clitest: Error: cannot read input file: /etc $ ``` ## No test found (message and exit code) ``` $ ./clitest dev/test/no-test-found.sh; echo $? clitest: Error: no test found in input file: dev/test/no-test-found.sh 2 $ ./clitest dev/test/empty-file.sh clitest: Error: no test found in input file: dev/test/empty-file.sh $ ./clitest dev/test/empty-prompt-file.sh clitest: Error: no test found in input file: dev/test/empty-prompt-file.sh $ ./clitest dev/test/empty-prompts-file.sh clitest: Error: no test found in input file: dev/test/empty-prompts-file.sh $ ``` ## Option --version The exit code must always be zero. ``` $ ./clitest --version > /dev/null; echo $? 0 $ ``` Test the output text and the short option `-V`. ``` $ ./clitest --version clitest HEAD https://github.com/aureliojargas/clitest/tree/HEAD $ ./clitest -V clitest HEAD https://github.com/aureliojargas/clitest/tree/HEAD $ ``` ## Option --help Test the full help text contents and the exit code (zero). ``` $ ./clitest --help; echo $? Usage: clitest [options] Options: -1, --first Stop execution upon first failed test -l, --list List all the tests (no execution) -L, --list-run List all the tests with OK/FAIL status -t, --test RANGE Run specific tests, by number (1,2,4-7) -s, --skip RANGE Skip specific tests, by number (1,2,4-7) --pre-flight COMMAND Execute command before running the first test --post-flight COMMAND Execute command after running the last test -q, --quiet Quiet operation, no output shown -V, --version Show program version and exit Customization options: -P, --progress TYPE Set progress indicator: test, number, dot, none --color WHEN Set when to use colors: auto, always, never --diff-options OPTIONS Set diff command options (default: '-u') --inline-prefix PREFIX Set inline output prefix (default: '#→ ') --prefix PREFIX Set command line prefix (default: '') --prompt STRING Set prompt string (default: '$ ') 0 $ ``` The short option `-h` is working? Testing just the first and last lines for brevity. ``` $ ./clitest -h | sed -n '1p; $p' Usage: clitest [options] --prompt STRING Set prompt string (default: '$ ') $ ``` ## Option --quiet and exit code ``` $ ./clitest -q dev/test/ok-2.sh; echo $? 0 $ ./clitest --quiet dev/test/ok-2.sh; echo $? 0 $ ./clitest --quiet dev/test/ok-2.sh dev/test/ok-2.sh; echo $? 0 $ ./clitest --quiet dev/test/fail-2.sh; echo $? 1 $ ./clitest --quiet dev/test/fail-2.sh dev/test/fail-2.sh; echo $? 1 $ ./clitest --quiet dev/test/ok-2.sh dev/test/fail-2.sh; echo $? 1 $ ``` ## Option --quiet has no effect in error messages ``` $ ./clitest --quiet /etc clitest: Error: cannot read input file: /etc $ ``` ## Option --quiet has no effect in --debug (disabled) ``` # $ ./clitest --quiet --debug dev/test/ok-2.sh # [INPUT_LINE: $ echo ok] # [ LINE_CMD: $ echo ok] # [ NEW_CMD: echo ok] # [INPUT_LINE: ok] # [ LINE_*: ok] # [ OK_TEXT: ok] # [INPUT_LINE: $ echo ok #→ ok] # [ LINE_CMD: $ echo ok #→ ok] # [ EVAL: echo ok] # [ OUTPUT: ok] # [ NEW_CMD: echo ok ] # [ OK_INLINE: ok] # [ OK_TEXT: ok] # [ EVAL: echo ok ] # [ OUTPUT: ok] # [ LOOP_OUT: $test_command=] # $ ``` ## Option --color Invalid value ``` $ ./clitest --color foo dev/test/ok-1.sh clitest: Error: invalid value 'foo' for --color. Use: auto, always or never. $ ``` Color ON ``` $ ./clitest --color always dev/test/ok-1.sh #1 echo ok OK: 1 of 1 test passed $ ./clitest --color yes dev/test/ok-1.sh #1 echo ok OK: 1 of 1 test passed $ ``` Color OFF ``` $ ./clitest --color never dev/test/ok-1.sh #1 echo ok OK: 1 of 1 test passed $ ./clitest --color no dev/test/ok-1.sh #1 echo ok OK: 1 of 1 test passed $ ``` Color AUTO Inside this file, the output is not a terminal, so the default is no colored output. ``` $ ./clitest dev/test/ok-1.sh #1 echo ok OK: 1 of 1 test passed $ ``` The real default `--color auto` cannot be tested here. Test it by hand at the command line. ``` ## $ ./clitest dev/test/ok-1.sh ## OK! The single test has passed. ## $ ./clitest --color auto dev/test/ok-1.sh ## OK! The single test has passed. ## $ ``` ## Option --list Listing a file with no tests ``` $ ./clitest --list dev/test/empty-file.sh clitest: Error: no test found in input file: dev/test/empty-file.sh $ ``` Normal results and exit code ``` $ ./clitest --list dev/test/no-nl-command.sh; echo $? #1 printf 'ok\n' #2 printf 'fail' #3 printf 'ok\nok\nfail' #4 printf 'ok\n' #5 printf 'fail' #6 printf 'ok'; echo #7 printf 'ok' 0 $ ``` Short option `-l` ``` $ ./clitest -l dev/test/no-nl-command.sh #1 printf 'ok\n' #2 printf 'fail' #3 printf 'ok\nok\nfail' #4 printf 'ok\n' #5 printf 'fail' #6 printf 'ok'; echo #7 printf 'ok' $ ``` Multifile and exit code ``` $ ./clitest --list dev/test/no-nl-command.sh dev/test/ok-1.sh; echo $? ---------------------------------------- dev/test/no-nl-command.sh #1 printf 'ok\n' #2 printf 'fail' #3 printf 'ok\nok\nfail' #4 printf 'ok\n' #5 printf 'fail' #6 printf 'ok'; echo #7 printf 'ok' ---------------------------------------- dev/test/ok-1.sh #8 echo ok 0 $ ``` ## Option --list-run Listing a file with no tests ``` $ ./clitest --list-run dev/test/empty-file.sh clitest: Error: no test found in input file: dev/test/empty-file.sh $ ``` Normal results (using colors) and exit code ``` $ ./clitest --list-run --color yes dev/test/no-nl-command.sh; echo $? #1 printf 'ok\n' #2 printf 'fail' #3 printf 'ok\nok\nfail' #4 printf 'ok\n'  #5 printf 'fail'  #6 printf 'ok'; echo  #7 printf 'ok'  1 $ ``` Normal results (no colors, use OK/FAIL column) and exit code ``` $ ./clitest --list-run dev/test/no-nl-command.sh; echo $? #1 OK printf 'ok\n' #2 FAIL printf 'fail' #3 FAIL printf 'ok\nok\nfail' #4 OK printf 'ok\n' #5 FAIL printf 'fail' #6 OK printf 'ok'; echo #7 OK printf 'ok' 1 $ ``` Short option `-L` ``` $ ./clitest -L dev/test/no-nl-command.sh #1 OK printf 'ok\n' #2 FAIL printf 'fail' #3 FAIL printf 'ok\nok\nfail' #4 OK printf 'ok\n' #5 FAIL printf 'fail' #6 OK printf 'ok'; echo #7 OK printf 'ok' $ ``` Multifile and exit code ``` $ ./clitest -L dev/test/no-nl-command.sh dev/test/ok-1.sh; echo $? ---------------------------------------- dev/test/no-nl-command.sh #1 OK printf 'ok\n' #2 FAIL printf 'fail' #3 FAIL printf 'ok\nok\nfail' #4 OK printf 'ok\n' #5 FAIL printf 'fail' #6 OK printf 'ok'; echo #7 OK printf 'ok' ---------------------------------------- dev/test/ok-1.sh #8 OK echo ok 1 $ ./clitest -L dev/test/ok-1.sh; echo $? #1 OK echo ok 0 $ ``` ## Option --progress First, some invalid values: ``` $ ./clitest --progress dev/test/ok-1.sh clitest: Error: no test file informed (try --help) $ ./clitest --progress '' dev/test/ok-1.sh clitest: Error: invalid value '' for --progress. Use: test, number, dot or none. $ ./clitest --progress foo dev/test/ok-1.sh clitest: Error: invalid value 'foo' for --progress. Use: test, number, dot or none. $ ./clitest --progress DOT dev/test/ok-1.sh clitest: Error: invalid value 'DOT' for --progress. Use: test, number, dot or none. $ ./clitest --progress @@ dev/test/ok-1.sh clitest: Error: invalid value '@@' for --progress. Use: test, number, dot or none. $ ./clitest --progress -1 dev/test/ok-1.sh clitest: Error: invalid value '-1' for --progress. Use: test, number, dot or none. $ ``` If no `--progress` option, defaults to `--progress test`: ``` $ ./clitest dev/test/ok-1.sh #1 echo ok OK: 1 of 1 test passed $ ./clitest --progress test dev/test/ok-1.sh #1 echo ok OK: 1 of 1 test passed $ ``` Numbers: ``` $ ./clitest --progress number dev/test/ok-10.sh 1 2 3 4 5 6 7 8 9 10 OK: 10 of 10 tests passed $ ./clitest --progress n dev/test/ok-10.sh 1 2 3 4 5 6 7 8 9 10 OK: 10 of 10 tests passed $ ./clitest --progress 0 dev/test/ok-10.sh 1 2 3 4 5 6 7 8 9 10 OK: 10 of 10 tests passed $ ./clitest --progress 5 dev/test/ok-10.sh 1 2 3 4 5 6 7 8 9 10 OK: 10 of 10 tests passed $ ./clitest --progress 9 dev/test/ok-10.sh 1 2 3 4 5 6 7 8 9 10 OK: 10 of 10 tests passed $ ``` Chars: ``` $ ./clitest --progress dot dev/test/ok-10.sh .......... OK: 10 of 10 tests passed $ ./clitest --progress . dev/test/ok-10.sh .......... OK: 10 of 10 tests passed $ ./clitest --progress @ dev/test/ok-10.sh @@@@@@@@@@ OK: 10 of 10 tests passed $ ./clitest --progress x dev/test/ok-10.sh xxxxxxxxxx OK: 10 of 10 tests passed $ ``` No progress: ``` $ ./clitest --progress none dev/test/ok-1.sh OK: 1 of 1 test passed $ ./clitest --progress no dev/test/ok-1.sh OK: 1 of 1 test passed $ ``` Short option `-P`: ``` $ ./clitest -P dot dev/test/ok-1.sh . OK: 1 of 1 test passed $ ./clitest -P no dev/test/ok-1.sh OK: 1 of 1 test passed $ ``` Ok & fail functionality with dot: ``` $ ./clitest --progress . dev/test/ok-1.sh . OK: 1 of 1 test passed $ ./clitest --progress . dev/test/ok-2.sh .. OK: 2 of 2 tests passed $ ./clitest --progress . dev/test/ok-50.sh .................................................. OK: 50 of 50 tests passed $ ./clitest --progress . dev/test/fail-1.sh . -------------------------------------------------------------------------------- [FAILED #1, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- FAIL: 1 of 1 test failed $ ``` Multifile with dot: ``` $ ./clitest --progress . dev/test/ok-1.sh dev/test/ok-2.sh dev/test/ok-10.sh Testing file dev/test/ok-1.sh . Testing file dev/test/ok-2.sh .. Testing file dev/test/ok-10.sh .......... ok fail skip 1 - - dev/test/ok-1.sh 2 - - dev/test/ok-2.sh 10 - - dev/test/ok-10.sh OK: 13 of 13 tests passed $ ./clitest --progress . dev/test/ok-1.sh dev/test/fail-1.sh Testing file dev/test/ok-1.sh . Testing file dev/test/fail-1.sh . -------------------------------------------------------------------------------- [FAILED #2, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- ok fail skip 1 - - dev/test/ok-1.sh - 1 - dev/test/fail-1.sh FAIL: 1 of 2 tests failed $ ./clitest --progress . dev/test/fail-1.sh dev/test/ok-1.sh Testing file dev/test/fail-1.sh . -------------------------------------------------------------------------------- [FAILED #1, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- Testing file dev/test/ok-1.sh . ok fail skip - 1 - dev/test/fail-1.sh 1 - - dev/test/ok-1.sh FAIL: 1 of 2 tests failed $ ``` Multifile with no progress: ``` $ ./clitest --progress none dev/test/ok-1.sh dev/test/ok-2.sh dev/test/ok-10.sh Testing file dev/test/ok-1.sh Testing file dev/test/ok-2.sh Testing file dev/test/ok-10.sh ok fail skip 1 - - dev/test/ok-1.sh 2 - - dev/test/ok-2.sh 10 - - dev/test/ok-10.sh OK: 13 of 13 tests passed $ ./clitest --progress none dev/test/ok-1.sh dev/test/fail-1.sh Testing file dev/test/ok-1.sh Testing file dev/test/fail-1.sh -------------------------------------------------------------------------------- [FAILED #2, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- ok fail skip 1 - - dev/test/ok-1.sh - 1 - dev/test/fail-1.sh FAIL: 1 of 2 tests failed $ ./clitest --progress none dev/test/fail-1.sh dev/test/ok-1.sh Testing file dev/test/fail-1.sh -------------------------------------------------------------------------------- [FAILED #1, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- Testing file dev/test/ok-1.sh ok fail skip - 1 - dev/test/fail-1.sh 1 - - dev/test/ok-1.sh FAIL: 1 of 2 tests failed $ ``` ### Option --progress and skipped tests Since skipped tests affect the output (show nothing), it's worth testing if the line break issues won't appear. ``` $ ./clitest --progress . --skip 1 dev/test/ok-2.sh . OK: 1 of 2 tests passed (1 skipped) $ ./clitest --progress . --skip 2 dev/test/ok-2.sh . OK: 1 of 2 tests passed (1 skipped) $ ./clitest --progress . --skip 1 dev/test/fail-2.sh . -------------------------------------------------------------------------------- [FAILED #2, line 3] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- FAIL: 1 of 2 tests failed (1 skipped) $ ./clitest --progress . --skip 2 dev/test/fail-2.sh . -------------------------------------------------------------------------------- [FAILED #1, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- FAIL: 1 of 2 tests failed (1 skipped) $ ``` Error messages appear with no leading blank line? ``` $ ./clitest --progress . --skip 1,2 dev/test/ok-2.sh clitest: Error: no test found. Maybe '--skip 1,2' was too much? $ ``` ## Options --quiet, --progress, --list and --list-run are mutually exclusive * Only one can be active, the others must be off. * The last informed will be the one used. ``` $ ./clitest --list --quiet dev/test/ok-1.sh $ ./clitest --list-run --quiet dev/test/ok-1.sh $ ./clitest --progress . --quiet dev/test/ok-1.sh $ ./clitest --list --list-run --progress . --quiet dev/test/ok-1.sh $ ./clitest --quiet --progress . --list-run --list dev/test/ok-1.sh #1 echo ok $ ./clitest --quiet --progress . --list --list-run dev/test/ok-1.sh #1 OK echo ok $ ./clitest --quiet --list --list-run --progress . dev/test/ok-1.sh . OK: 1 of 1 test passed $ ``` ## Option --test and --skip combined with --list and --list-run Error: Out of range ``` $ ./clitest --list -t 99 dev/test/ok-10.sh clitest: Error: no test found for the specified number or range '99' $ ./clitest --list-run -t 99 dev/test/ok-10.sh clitest: Error: no test found for the specified number or range '99' $ ``` Error: Skipped all tests ``` $ ./clitest --list -s 1-10 dev/test/ok-10.sh clitest: Error: no test found. Maybe '--skip 1-10' was too much? $ ./clitest --list-run -s 1-10 dev/test/ok-10.sh clitest: Error: no test found. Maybe '--skip 1-10' was too much? $ ``` Error: The combination of `-t` and `-s` resulted in no tests ``` $ ./clitest --list -t 9 -s 9 dev/test/ok-10.sh clitest: Error: no test found. The combination of -t and -s resulted in no tests. $ ./clitest --list-run -t 9 -s 9 dev/test/ok-10.sh clitest: Error: no test found. The combination of -t and -s resulted in no tests. $ ``` Using `-t` alone ``` $ ./clitest --list -t 3,5-7 dev/test/ok-10.sh #3 echo 3 #5 echo 5 #6 echo 6 #7 echo 7 $ ./clitest --list-run -t 3,5-7 dev/test/ok-10.sh #3 OK echo 3 #5 OK echo 5 #6 OK echo 6 #7 OK echo 7 $ ``` Using `-t` to limit to a range and the `-s` exclude some more ``` $ ./clitest --list -t 3,5-7 -s 6 dev/test/ok-10.sh #3 echo 3 #5 echo 5 #7 echo 7 $ ./clitest --list-run -t 3,5-7 -s 6 dev/test/ok-10.sh #3 OK echo 3 #5 OK echo 5 #7 OK echo 7 $ ``` Multifile, using `-t` alone ``` $ ./clitest --list -t 1,3,5-7 dev/test/ok-1.sh dev/test/fail-2.sh dev/test/ok-10.sh ---------------------------------------- dev/test/ok-1.sh #1 echo ok ---------------------------------------- dev/test/fail-2.sh #3 echo ok ---------------------------------------- dev/test/ok-10.sh #5 echo 2 #6 echo 3 #7 echo 4 $ ./clitest --list-run -t 1,3,5-7 dev/test/ok-1.sh dev/test/fail-2.sh dev/test/ok-10.sh ---------------------------------------- dev/test/ok-1.sh #1 OK echo ok ---------------------------------------- dev/test/fail-2.sh #3 FAIL echo ok ---------------------------------------- dev/test/ok-10.sh #5 OK echo 2 #6 OK echo 3 #7 OK echo 4 $ ``` Multifile, using `-t` and `-s` ``` $ ./clitest --list -t 1,3,5-7 -s 3,6 dev/test/ok-1.sh dev/test/fail-2.sh dev/test/ok-10.sh ---------------------------------------- dev/test/ok-1.sh #1 echo ok ---------------------------------------- dev/test/fail-2.sh ---------------------------------------- dev/test/ok-10.sh #5 echo 2 #7 echo 4 $ ./clitest --list-run -t 1,3,5-7 -s 3,6 dev/test/ok-1.sh dev/test/fail-2.sh dev/test/ok-10.sh ---------------------------------------- dev/test/ok-1.sh #1 OK echo ok ---------------------------------------- dev/test/fail-2.sh ---------------------------------------- dev/test/ok-10.sh #5 OK echo 2 #7 OK echo 4 $ ``` ## Single file, OK ``` $ ./clitest dev/test/ok-1.sh #1 echo ok OK: 1 of 1 test passed $ ./clitest dev/test/ok-2.sh #1 echo ok #2 echo ok OK: 2 of 2 tests passed $ ./clitest dev/test/ok-50.sh | tail -1 OK: 50 of 50 tests passed $ ./clitest dev/test/ok-100.sh | tail -1 OK: 100 of 100 tests passed $ ./clitest dev/test/ok-2.sh #1 echo ok #2 echo ok OK: 2 of 2 tests passed $ ``` ## Multifile, all OK ``` $ ./clitest dev/test/ok-2.sh dev/test/ok-2.sh Testing file dev/test/ok-2.sh #1 echo ok #2 echo ok Testing file dev/test/ok-2.sh #3 echo ok #4 echo ok ok fail skip 2 - - dev/test/ok-2.sh 2 - - dev/test/ok-2.sh OK: 4 of 4 tests passed $ ./clitest dev/test/ok-[0-9]*.sh | grep -v ^# Testing file dev/test/ok-1.sh Testing file dev/test/ok-10.sh Testing file dev/test/ok-100.sh Testing file dev/test/ok-2.sh Testing file dev/test/ok-50.sh ok fail skip 1 - - dev/test/ok-1.sh 10 - - dev/test/ok-10.sh 100 - - dev/test/ok-100.sh 2 - - dev/test/ok-2.sh 50 - - dev/test/ok-50.sh OK: 163 of 163 tests passed $ ./clitest dev/test/ok-?.sh dev/test/ok-10.sh Testing file dev/test/ok-1.sh #1 echo ok Testing file dev/test/ok-2.sh #2 echo ok #3 echo ok Testing file dev/test/ok-10.sh #4 echo 1 #5 echo 2 #6 echo 3 #7 echo 4 #8 echo 5 #9 echo 6 #10 echo 7 #11 echo 8 #12 echo 9 #13 echo 10 ok fail skip 1 - - dev/test/ok-1.sh 2 - - dev/test/ok-2.sh 10 - - dev/test/ok-10.sh OK: 13 of 13 tests passed $ ``` ## Multifile, OK and fail ``` $ ./clitest dev/test/ok-1.sh dev/test/fail-1.sh dev/test/ok-2.sh dev/test/fail-2.sh Testing file dev/test/ok-1.sh #1 echo ok Testing file dev/test/fail-1.sh #2 echo ok -------------------------------------------------------------------------------- [FAILED #2, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- Testing file dev/test/ok-2.sh #3 echo ok #4 echo ok Testing file dev/test/fail-2.sh #5 echo ok -------------------------------------------------------------------------------- [FAILED #5, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- #6 echo ok -------------------------------------------------------------------------------- [FAILED #6, line 3] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- ok fail skip 1 - - dev/test/ok-1.sh - 1 - dev/test/fail-1.sh 2 - - dev/test/ok-2.sh - 2 - dev/test/fail-2.sh FAIL: 3 of 6 tests failed $ ./clitest dev/test/ok-1.sh dev/test/fail-1.sh dev/test/ok-2.sh dev/test/fail-2.sh Testing file dev/test/ok-1.sh #1 echo ok Testing file dev/test/fail-1.sh #2 echo ok -------------------------------------------------------------------------------- [FAILED #2, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- Testing file dev/test/ok-2.sh #3 echo ok #4 echo ok Testing file dev/test/fail-2.sh #5 echo ok -------------------------------------------------------------------------------- [FAILED #5, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- #6 echo ok -------------------------------------------------------------------------------- [FAILED #6, line 3] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- ok fail skip 1 - - dev/test/ok-1.sh - 1 - dev/test/fail-1.sh 2 - - dev/test/ok-2.sh - 2 - dev/test/fail-2.sh FAIL: 3 of 6 tests failed $ ``` ## Fail messages ``` $ ./clitest --prefix tab -P none dev/test/fail-messages.md #→ --file test/fail-messages.out.txt $ ``` ## Fails ``` $ ./clitest dev/test/fail-1.sh #1 echo ok -------------------------------------------------------------------------------- [FAILED #1, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- FAIL: 1 of 1 test failed $ ./clitest dev/test/fail-2.sh #1 echo ok -------------------------------------------------------------------------------- [FAILED #1, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- #2 echo ok -------------------------------------------------------------------------------- [FAILED #2, line 3] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- FAIL: 2 of 2 tests failed $ ./clitest dev/test/fail-50.sh | tail -1 FAIL: 50 of 50 tests failed $ ./clitest -1 dev/test/fail-2.sh #1 echo ok -------------------------------------------------------------------------------- [FAILED #1, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- $ ./clitest --first dev/test/fail-2.sh #1 echo ok -------------------------------------------------------------------------------- [FAILED #1, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- $ ./clitest --first dev/test/fail-2.sh #1 echo ok -------------------------------------------------------------------------------- [FAILED #1, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- $ ./clitest dev/test/fail-2.sh #1 echo ok -------------------------------------------------------------------------------- [FAILED #1, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- #2 echo ok -------------------------------------------------------------------------------- [FAILED #2, line 3] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- FAIL: 2 of 2 tests failed $ ``` ## Inline output with #→ ``` $ ./clitest dev/test/inline.sh #1 echo 'one space' #2 echo 'one tab' #3 echo 'multi spaces' #4 echo 'multi tabs' #5 echo 'mixed' #6 echo ' leading space' #7 echo ' leading spaces' #8 printf '\tleading tab\n' #9 printf '\t\tleading tabs\n' #10 echo 'trailing space ' #11 echo 'trailing spaces ' #12 printf 'trailing tab\t\n' #13 printf 'trailing tabs\t\t\n' #14 echo ' ' #15 echo ' ' #16 printf '\t\n' #17 printf '\t\t\t\n' #18 printf ' \t \t\t \n' #19 echo "both inline and normal output" OK: 19 of 19 tests passed $ ``` ## Inline match modes Mode #→ --text * This is the default mode. * The --text part can be omitted. ``` $ ./clitest --list-run dev/test/inline-match-text.sh #1 OK echo 'abc' #2 OK echo 'abc' #3 OK printf '%s\n' '\t' #4 OK printf '%s\n' '\n' #5 OK echo '$PWD' #6 OK echo '$(date)' #7 OK echo '$' #8 OK echo '>' #9 OK echo '?' #10 OK echo '!' #11 OK echo '*' #12 OK echo '[' #13 OK echo '(' #14 OK echo #15 OK echo "not inline output" #→ #16 OK echo '123456789' #17 OK echo '1 3 7 9' #18 OK echo ' 5 ' #19 OK echo ' leading space' #20 OK echo ' leading spaces' #21 OK printf '\tleading tab\n' #22 OK printf '\t\tleading tabs\n' #23 OK echo 'trailing space ' #24 OK echo 'trailing spaces ' #25 OK printf 'trailing tab\t\n' #26 OK printf 'trailing tabs\t\t\n' #27 OK echo ' ' #28 FAIL echo ' ' #29 OK printf '\t\n' #30 OK printf '\t\t\t\n' #31 OK printf ' \t \t\t \n' #32 OK printf 'ok\n' #33 FAIL printf 'fail' #34 OK printf 'ok'; echo #35 FAIL echo 'fail' #36 FAIL echo 'fail' #37 OK echo ' ok' #38 OK echo '--text' #39 OK echo '--textual' #40 OK echo '--text is cool' $ ``` Mode #→ --eval ``` $ ./clitest --list-run dev/test/inline-match-eval.sh #1 OK folder=$(pwd) #2 OK echo $folder #3 OK var='abc' #4 OK echo abc #5 OK echo 4 #6 OK today=$(date +%D) #7 OK echo "Today is $today" #8 OK printf 'ok' #9 OK echo ' leading space' #10 OK echo ' leading spaces' #11 OK printf '\tleading tab\n' #12 OK printf '\t\tleading tabs\n' #13 OK echo 'trailing space ' #14 OK echo 'trailing spaces ' #15 OK printf 'trailing tab\t\n' #16 OK printf 'trailing tabs\t\t\n' #17 OK echo ' ' #18 OK echo ' ' #19 OK printf '\t\n' #20 OK printf '\t\t\t\n' #21 OK printf ' \t \t\t \n' #22 FAIL echo 'fail' #23 FAIL echo 'fail' #24 OK echo '--eval' #25 OK echo '--evaluate' #26 OK echo '--eval is evil' $ ``` Mode #→ --egrep ``` $ ./clitest --list-run dev/test/inline-match-egrep.sh #1 OK echo 'abc123' #2 OK echo 'abc123' #3 OK echo 'abc123' #4 OK echo 'abc123' #5 OK echo 'abc123' #6 OK echo 'abc123' #7 OK echo 'abc123' #8 OK echo 'abc123' #9 OK echo 'abc 123' #10 OK echo ' ' #11 OK echo ' ' #12 OK printf '\t\n' #13 OK printf '\t\t\t\n' #14 OK printf ' \t \t\t \n' #15 OK printf 'may\tfail' #16 FAIL printf 'may\tfail' #17 OK printf 'will\tmatch' #18 FAIL printf 'will\nfail' #19 FAIL printf 'will\nfail' #20 OK printf '1\n2\n3\n4\nok\n' #21 OK printf 'ok' #22 OK printf 'ok\n' #23 FAIL echo 'fail' #24 FAIL echo 'fail' #25 OK echo ' ok' #26 OK echo '--egrep' #27 OK echo '--egreppal' #28 OK echo '--egrep is cool' $ ``` Mode #→ --perl * --regex is an alias to --perl ``` $ ./clitest --list-run dev/test/inline-match-perl.sh #1 OK echo 'abc123' #2 OK echo 'abc123' #3 OK echo 'abc123' #4 OK echo 'abc123' #5 OK echo 'abc123' #6 OK echo 'abc123' #7 OK echo 'abc123' #8 OK echo 'abc123' #9 OK echo 'abc 123' #10 OK echo ' ' #11 OK echo ' ' #12 OK printf '\t\n' #13 OK printf '\t\t\t\n' #14 OK printf ' \t \t\t \n' #15 OK echo '01/01/2013' #16 OK echo "won't fail" #17 OK printf 'will\tmatch' #18 OK printf 'will\tmatch' #19 OK printf 'will\tmatch' #20 FAIL printf 'will\nfail' #21 OK printf 'will\nmatch' #22 FAIL printf 'will\nfail' #23 OK printf 'will\nmatch' #24 FAIL printf 'will\nfail' #25 OK printf 'will\nmatch' #26 OK printf 'ok' #27 OK printf 'ok\n' #28 OK printf '1\n2\n3\n' #29 OK printf '1\n2\n3\n' #30 FAIL echo 'fail' #31 FAIL echo 'fail' #32 OK echo ' ok' #33 OK echo '--perl' #34 OK echo '--perlism' #35 OK echo '--perl is cool' $ ``` Mode #→ --file ``` $ ./clitest --list-run dev/test/inline-match-file.sh #1 OK printf '$ echo ok\nok\n' #2 OK echo 'ok' > /tmp/foo.txt #3 OK echo 'ok' #4 FAIL echo 'fail' #5 FAIL echo 'fail' #6 OK echo '--file' #7 OK echo '--filer' #8 OK echo '--file is cool' $ ``` Mode #→ --lines ``` $ ./clitest --list-run dev/test/inline-match-lines.sh #1 OK a=1 #2 OK echo 'ok' #3 OK printf '1\n2\n3\n' #4 OK printf 'no-nl' #5 OK printf '1\n2\nno-nl' #6 FAIL echo 'fail' #7 FAIL echo 'fail' #8 FAIL echo 'fail' #9 FAIL echo 'fail' #10 OK echo '--lines' #11 OK echo '--linesout' #12 OK echo '--lines is cool' $ ./clitest --first dev/test/inline-match-lines.sh #1 a=1 #2 echo 'ok' #3 printf '1\n2\n3\n' #4 printf 'no-nl' #5 printf '1\n2\nno-nl' #6 echo 'fail' -------------------------------------------------------------------------------- [FAILED #6, line 16] echo 'fail' Expected 99 lines, got 1. -------------------------------------------------------------------------------- $ ``` Errors for #→ --egrep ``` $ ./clitest dev/test/inline-match-egrep-error-1.sh clitest: Error: empty --egrep at line 1 of dev/test/inline-match-egrep-error-1.sh $ ./clitest dev/test/inline-match-egrep-error-2.sh 2>&1 | sed 's/^egrep: .*/egrep: ERROR_MSG/' #1 echo "error: malformed regex" egrep: ERROR_MSG clitest: Error: check your inline egrep regex at line 1 of dev/test/inline-match-egrep-error-2.sh $ ``` Errors for #→ --perl (and --regex) ``` $ ./clitest dev/test/inline-match-perl-error-1.sh clitest: Error: empty --perl at line 1 of dev/test/inline-match-perl-error-1.sh $ ./clitest dev/test/inline-match-perl-error-2.sh #1 echo "error: malformed regex" Unmatched ( in regex; marked by <-- HERE in m/( <-- HERE / at -e line 1. clitest: Error: check your inline Perl regex at line 1 of dev/test/inline-match-perl-error-2.sh $ ``` Errors for #→ --file ``` $ ./clitest dev/test/inline-match-file-error-1.sh clitest: Error: empty --file at line 1 of dev/test/inline-match-file-error-1.sh $ ./clitest dev/test/inline-match-file-error-2.sh #1 echo "error: file not found" clitest: Error: cannot read inline output file 'dev/test/XXnotfoundXX', from line 1 of dev/test/inline-match-file-error-2.sh $ ./clitest dev/test/inline-match-file-error-3.sh #1 echo "error: directory" clitest: Error: cannot read inline output file '/etc/', from line 1 of dev/test/inline-match-file-error-3.sh $ ``` Errors for #→ --lines ``` $ ./clitest dev/test/inline-match-lines-error-1.sh clitest: Error: --lines requires a number. See line 1 of dev/test/inline-match-lines-error-1.sh $ ./clitest dev/test/inline-match-lines-error-2.sh clitest: Error: --lines requires a number. See line 1 of dev/test/inline-match-lines-error-2.sh $ ./clitest dev/test/inline-match-lines-error-3.sh clitest: Error: --lines requires a number. See line 1 of dev/test/inline-match-lines-error-3.sh $ ./clitest dev/test/inline-match-lines-error-4.sh clitest: Error: --lines requires a number. See line 1 of dev/test/inline-match-lines-error-4.sh $ ``` Errors for #→ --eval ``` $ ./clitest dev/test/inline-match-eval-error-1.sh clitest: Error: empty --eval at line 1 of dev/test/inline-match-eval-error-1.sh $ ./clitest dev/test/inline-match-eval-error-2.sh 2>&1 | sed 's/line [0-9][0-9]*/line N/' #1 echo 'error: syntax error' ./clitest: eval: line N: unexpected EOF while looking for matching `)' ./clitest: eval: line N: syntax error: unexpected end of file -------------------------------------------------------------------------------- [FAILED #1, line N] echo 'error: syntax error' @@ -0,0 +1 @@ +error: syntax error -------------------------------------------------------------------------------- FAIL: 1 of 1 test failed $ ``` ## Option -t, --test Error: Invalid argument ``` $ ./clitest -t - dev/test/ok-2.sh clitest: Error: invalid argument for -t or --test: - $ ./clitest -t -1 dev/test/ok-2.sh clitest: Error: invalid argument for -t or --test: -1 $ ./clitest -t 1- dev/test/ok-2.sh clitest: Error: invalid argument for -t or --test: 1- $ ./clitest -t 1--2 dev/test/ok-2.sh clitest: Error: invalid argument for -t or --test: 1--2 $ ./clitest -t 1-2-3 dev/test/ok-2.sh clitest: Error: invalid argument for -t or --test: 1-2-3 $ ``` Error: Out of range ``` $ ./clitest -t 99 dev/test/ok-2.sh clitest: Error: no test found for the specified number or range '99' $ ``` If range = zero or empty, run all tests ``` $ ./clitest -t '' dev/test/ok-2.sh #1 echo ok #2 echo ok OK: 2 of 2 tests passed $ ./clitest -t 0 dev/test/ok-2.sh #1 echo ok #2 echo ok OK: 2 of 2 tests passed $ ``` * Empty values inside range are ignored * The bogus `0-0` range is ignored * The resulting range is zero ``` $ ./clitest -t ,,,0,0-0,,, dev/test/ok-2.sh #1 echo ok #2 echo ok OK: 2 of 2 tests passed $ ``` Normal operation, using `--test` and `-t` ``` $ ./clitest -t 1 dev/test/ok-10.sh #1 echo 1 OK: 1 of 10 tests passed (9 skipped) $ ./clitest --test 1 dev/test/ok-10.sh #1 echo 1 OK: 1 of 10 tests passed (9 skipped) $ ``` Ranges `0-1` and `1-0` expand to `1` ``` $ ./clitest -t 0-1,1-0 dev/test/ok-10.sh #1 echo 1 OK: 1 of 10 tests passed (9 skipped) $ ``` Range `1-1` expand to `1` ``` $ ./clitest -t 1-1 dev/test/ok-10.sh #1 echo 1 OK: 1 of 10 tests passed (9 skipped) $ ``` Repeated values are OK ``` $ ./clitest -t 1,1,1,0,1 dev/test/ok-10.sh #1 echo 1 OK: 1 of 10 tests passed (9 skipped) $ ``` Range terminator is out of bounds ``` $ ./clitest -t 10-20 dev/test/ok-10.sh #10 echo 10 OK: 1 of 10 tests passed (9 skipped) $ ``` Inverted ranges ``` $ ./clitest -t 3,2,1 dev/test/ok-10.sh #1 echo 1 #2 echo 2 #3 echo 3 OK: 3 of 10 tests passed (7 skipped) $ ./clitest -t 3-1 dev/test/ok-10.sh #1 echo 1 #2 echo 2 #3 echo 3 OK: 3 of 10 tests passed (7 skipped) $ ``` Multifile. The test numbers always increase sequentially, regardless of the file changes. ``` $ ./clitest -t 1,5,13 dev/test/ok-?.sh dev/test/ok-10.sh Testing file dev/test/ok-1.sh #1 echo ok Testing file dev/test/ok-2.sh Testing file dev/test/ok-10.sh #5 echo 2 #13 echo 10 ok fail skip 1 - - dev/test/ok-1.sh - - 2 dev/test/ok-2.sh 2 - 8 dev/test/ok-10.sh OK: 3 of 13 tests passed (10 skipped) $ ./clitest -t 1,5 dev/test/ok-[12].sh dev/test/fail-2.sh Testing file dev/test/ok-1.sh #1 echo ok Testing file dev/test/ok-2.sh Testing file dev/test/fail-2.sh #5 echo ok -------------------------------------------------------------------------------- [FAILED #5, line 3] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- ok fail skip 1 - - dev/test/ok-1.sh - - 2 dev/test/ok-2.sh - 1 1 dev/test/fail-2.sh FAIL: 1 of 5 tests failed (3 skipped) $ ./clitest -t 1 dev/test/ok-[12].sh dev/test/fail-2.sh Testing file dev/test/ok-1.sh #1 echo ok Testing file dev/test/ok-2.sh Testing file dev/test/fail-2.sh ok fail skip 1 - - dev/test/ok-1.sh - - 2 dev/test/ok-2.sh - - 2 dev/test/fail-2.sh OK: 1 of 5 tests passed (4 skipped) $ ``` ## Option -s, --skip Error: Invalid argument ``` $ ./clitest -s - dev/test/ok-2.sh clitest: Error: invalid argument for -s or --skip: - $ ./clitest -s -1 dev/test/ok-2.sh clitest: Error: invalid argument for -s or --skip: -1 $ ./clitest -s 1- dev/test/ok-2.sh clitest: Error: invalid argument for -s or --skip: 1- $ ./clitest -s 1--2 dev/test/ok-2.sh clitest: Error: invalid argument for -s or --skip: 1--2 $ ./clitest -s 1-2-3 dev/test/ok-2.sh clitest: Error: invalid argument for -s or --skip: 1-2-3 $ ``` Error: Skipped all tests ``` $ ./clitest -s 1 dev/test/ok-1.sh clitest: Error: no test found. Maybe '--skip 1' was too much? $ ``` Out of range: no problem, you just skipped a non-existent test. All tests will be run. ``` $ ./clitest -s 99 dev/test/ok-2.sh #1 echo ok #2 echo ok OK: 2 of 2 tests passed $ ``` If range = zero or empty, run all tests ``` $ ./clitest -s '' dev/test/ok-2.sh #1 echo ok #2 echo ok OK: 2 of 2 tests passed $ ./clitest -s 0 dev/test/ok-2.sh #1 echo ok #2 echo ok OK: 2 of 2 tests passed $ ``` * Empty values inside range are ignored * The bogus `0-0` range is ignored * The resulting range is zero ``` $ ./clitest -s ,,,0,0-0,,, dev/test/ok-2.sh #1 echo ok #2 echo ok OK: 2 of 2 tests passed $ ``` Normal operation, using `--skip` and `-s` ``` $ ./clitest -s 1 dev/test/ok-2.sh #2 echo ok OK: 1 of 2 tests passed (1 skipped) $ ./clitest --skip 1 dev/test/ok-2.sh #2 echo ok OK: 1 of 2 tests passed (1 skipped) $ ``` Ranges `0-1` and `1-0` expand to `1` ``` $ ./clitest -s 0-1,1-0 dev/test/ok-2.sh #2 echo ok OK: 1 of 2 tests passed (1 skipped) $ ``` Range `1-1` expand to `1` ``` $ ./clitest -s 1-1 dev/test/ok-2.sh #2 echo ok OK: 1 of 2 tests passed (1 skipped) $ ``` Repeated values are OK ``` $ ./clitest -s 1,1,1,0,1 dev/test/ok-2.sh #2 echo ok OK: 1 of 2 tests passed (1 skipped) $ ``` Range terminator is out of bounds ``` $ ./clitest -s 2-10 dev/test/ok-2.sh #1 echo ok OK: 1 of 2 tests passed (1 skipped) $ ``` Inverted ranges ``` $ ./clitest -s 10,9,8,7,6,5,4 dev/test/ok-10.sh #1 echo 1 #2 echo 2 #3 echo 3 OK: 3 of 10 tests passed (7 skipped) $ ./clitest -s 10-4 dev/test/ok-10.sh #1 echo 1 #2 echo 2 #3 echo 3 OK: 3 of 10 tests passed (7 skipped) $ ``` Multifile. The test numbers always increase sequentially, regardless of the file changes. ``` $ ./clitest -s 2,3,13 dev/test/ok-?.sh dev/test/ok-10.sh Testing file dev/test/ok-1.sh #1 echo ok Testing file dev/test/ok-2.sh Testing file dev/test/ok-10.sh #4 echo 1 #5 echo 2 #6 echo 3 #7 echo 4 #8 echo 5 #9 echo 6 #10 echo 7 #11 echo 8 #12 echo 9 ok fail skip 1 - - dev/test/ok-1.sh - - 2 dev/test/ok-2.sh 9 - 1 dev/test/ok-10.sh OK: 10 of 13 tests passed (3 skipped) $ ./clitest -s 2,3,4 dev/test/ok-[12].sh dev/test/fail-2.sh Testing file dev/test/ok-1.sh #1 echo ok Testing file dev/test/ok-2.sh Testing file dev/test/fail-2.sh #5 echo ok -------------------------------------------------------------------------------- [FAILED #5, line 3] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- ok fail skip 1 - - dev/test/ok-1.sh - - 2 dev/test/ok-2.sh - 1 1 dev/test/fail-2.sh FAIL: 1 of 5 tests failed (3 skipped) $ ./clitest -s 2-10 dev/test/ok-[12].sh dev/test/fail-2.sh Testing file dev/test/ok-1.sh #1 echo ok Testing file dev/test/ok-2.sh Testing file dev/test/fail-2.sh ok fail skip 1 - - dev/test/ok-1.sh - - 2 dev/test/ok-2.sh - - 2 dev/test/fail-2.sh OK: 1 of 5 tests passed (4 skipped) $ ``` ## Option --test combined with --skip Error: The combination of `-t` and `-s` resulted in no tests ``` $ ./clitest -t 9 -s 9 dev/test/ok-10.sh clitest: Error: no test found. The combination of -t and -s resulted in no tests. $ ``` The order does not matter, `-s` always wins ``` $ ./clitest -s 9 -t 9 dev/test/ok-10.sh clitest: Error: no test found. The combination of -t and -s resulted in no tests. $ ``` Using `-t` to limit to a range and the `-s` exclude some more ``` $ ./clitest -t 3,5-7 -s 6 dev/test/ok-10.sh #3 echo 3 #5 echo 5 #7 echo 7 OK: 3 of 10 tests passed (7 skipped) $ ``` Same as previous, but now multifile ``` $ ./clitest -t 1,3,5-7 -s 3,6 dev/test/ok-1.sh dev/test/fail-2.sh dev/test/ok-10.sh Testing file dev/test/ok-1.sh #1 echo ok Testing file dev/test/fail-2.sh Testing file dev/test/ok-10.sh #5 echo 2 #7 echo 4 ok fail skip 1 - - dev/test/ok-1.sh - - 2 dev/test/fail-2.sh 2 - 8 dev/test/ok-10.sh OK: 3 of 13 tests passed (10 skipped) $ ``` ## Option --diff-options ``` $ ./clitest dev/test/option-diff-options.sh #1 echo " diff -w to ignore spaces " -------------------------------------------------------------------------------- [FAILED #1, line 3] echo " diff -w to ignore spaces " @@ -1 +1 @@ -diff -w to ignore spaces + diff -w to ignore spaces -------------------------------------------------------------------------------- #2 echo " diff -w now inline " -------------------------------------------------------------------------------- [FAILED #2, line 5] echo " diff -w now inline " @@ -1 +1 @@ -diff -w now inline + diff -w now inline -------------------------------------------------------------------------------- FAIL: 2 of 2 tests failed $ ./clitest --diff-options '-u -w' dev/test/option-diff-options.sh #1 echo " diff -w to ignore spaces " #2 echo " diff -w now inline " OK: 2 of 2 tests passed $ ``` ## Option --prompt ``` $ ./clitest dev/test/option-prompt.sh clitest: Error: no test found in input file: dev/test/option-prompt.sh $ ./clitest --prompt 'prompt$ ' dev/test/option-prompt.sh #1 echo "1" #2 echo "2" #3 echo "3" OK: 3 of 3 tests passed $ ./clitest --prompt '♥ ' dev/test/option-prompt-unicode.sh #1 echo "1" #2 echo "2" #3 echo "3" OK: 3 of 3 tests passed $ ``` ## Option --inline-prefix ``` $ ./clitest dev/test/option-inline-prefix.sh #1 echo "1 space" #==> 1 space -------------------------------------------------------------------------------- [FAILED #1, line 3] echo "1 space" #==> 1 space @@ -0,0 +1 @@ +1 space -------------------------------------------------------------------------------- #2 echo "8 spaces" #==> 8 spaces -------------------------------------------------------------------------------- [FAILED #2, line 4] echo "8 spaces" #==> 8 spaces @@ -0,0 +1 @@ +8 spaces -------------------------------------------------------------------------------- #3 echo "2 tabs" #==> 2 tabs -------------------------------------------------------------------------------- [FAILED #3, line 5] echo "2 tabs" #==> 2 tabs @@ -0,0 +1 @@ +2 tabs -------------------------------------------------------------------------------- FAIL: 3 of 3 tests failed $ ./clitest --inline-prefix '#==>' dev/test/option-inline-prefix.sh #1 echo "1 space" -------------------------------------------------------------------------------- [FAILED #1, line 3] echo "1 space" @@ -1 +1 @@ - 1 space +1 space -------------------------------------------------------------------------------- #2 echo "8 spaces" -------------------------------------------------------------------------------- [FAILED #2, line 4] echo "8 spaces" @@ -1 +1 @@ - 8 spaces +8 spaces -------------------------------------------------------------------------------- #3 echo "2 tabs" -------------------------------------------------------------------------------- [FAILED #3, line 5] echo "2 tabs" @@ -1 +1 @@ - 2 tabs +2 tabs -------------------------------------------------------------------------------- FAIL: 3 of 3 tests failed $ ./clitest --inline-prefix '#==> ' dev/test/option-inline-prefix.sh #1 echo "1 space" #2 echo "8 spaces" #3 echo "2 tabs" OK: 3 of 3 tests passed $ ``` ## Option --prefix ``` $ ./clitest --prefix ' ' dev/test/option-prefix.sh #1 echo "1" #2 echo "2" #3 echo "3" #4 echo "4" #5 echo "5" #6 echo; echo "6"; echo; echo "7" OK: 6 of 6 tests passed $ ./clitest --prefix 4 dev/test/option-prefix.sh #1 echo "1" #2 echo "2" #3 echo "3" #4 echo "4" #5 echo "5" #6 echo; echo "6"; echo; echo "7" OK: 6 of 6 tests passed $ ./clitest --prefix '\t' dev/test/option-prefix-tab.sh #1 echo "1" #2 echo "2" #3 echo "3" #4 echo "4" #5 echo "5" #6 echo; echo "6"; echo; echo "7" OK: 6 of 6 tests passed $ ./clitest --prefix tab dev/test/option-prefix-tab.sh #1 echo "1" #2 echo "2" #3 echo "3" #4 echo "4" #5 echo "5" #6 echo; echo "6"; echo; echo "7" OK: 6 of 6 tests passed $ ``` ## Option --prefix: glob gotchas ``` $ ./clitest --prefix '?' dev/test/option-prefix-glob.sh #1 echo 'prefix ?' #2 echo 'prefix ?' OK: 2 of 2 tests passed $ ./clitest --prefix '*' dev/test/option-prefix-glob.sh #1 echo 'prefix *' #2 echo 'prefix *' OK: 2 of 2 tests passed $ ./clitest --prefix '#' dev/test/option-prefix-glob.sh #1 echo 'prefix #' #2 echo 'prefix #' OK: 2 of 2 tests passed $ ./clitest --prefix '%' dev/test/option-prefix-glob.sh #1 echo 'prefix %' #2 echo 'prefix %' OK: 2 of 2 tests passed $ ./clitest --prefix '##' dev/test/option-prefix-glob.sh #1 echo 'prefix ##' #2 echo 'prefix ##' OK: 2 of 2 tests passed $ ./clitest --prefix '%%' dev/test/option-prefix-glob.sh #1 echo 'prefix %%' #2 echo 'prefix %%' OK: 2 of 2 tests passed $ ./clitest --prefix '#*' dev/test/option-prefix-glob.sh #1 echo 'prefix #*' #2 echo 'prefix #*' OK: 2 of 2 tests passed $ ./clitest --prefix '*#' dev/test/option-prefix-glob.sh #1 echo 'prefix *#' #2 echo 'prefix *#' OK: 2 of 2 tests passed $ ./clitest --prefix '%*' dev/test/option-prefix-glob.sh #1 echo 'prefix %*' #2 echo 'prefix %*' OK: 2 of 2 tests passed $ ./clitest --prefix '*%' dev/test/option-prefix-glob.sh #1 echo 'prefix *%' #2 echo 'prefix *%' OK: 2 of 2 tests passed $ ``` ## Option --prompt: glob gotchas (char + space) ``` $ ./clitest --prompt '? ' dev/test/option-prompt-glob-space.sh #1 echo 'prompt ? ' #2 echo 'prompt ? ' OK: 2 of 2 tests passed $ ./clitest --prompt '* ' dev/test/option-prompt-glob-space.sh #1 echo 'prompt * ' #2 echo 'prompt * ' OK: 2 of 2 tests passed $ ./clitest --prompt '# ' dev/test/option-prompt-glob-space.sh #1 echo 'prompt # ' #2 echo 'prompt # ' OK: 2 of 2 tests passed $ ./clitest --prompt '% ' dev/test/option-prompt-glob-space.sh #1 echo 'prompt % ' #2 echo 'prompt % ' OK: 2 of 2 tests passed $ ./clitest --prompt '## ' dev/test/option-prompt-glob-space.sh #1 echo 'prompt ## ' #2 echo 'prompt ## ' OK: 2 of 2 tests passed $ ./clitest --prompt '%% ' dev/test/option-prompt-glob-space.sh #1 echo 'prompt %% ' #2 echo 'prompt %% ' OK: 2 of 2 tests passed $ ./clitest --prompt '#* ' dev/test/option-prompt-glob-space.sh #1 echo 'prompt #* ' #2 echo 'prompt #* ' OK: 2 of 2 tests passed $ ./clitest --prompt '*# ' dev/test/option-prompt-glob-space.sh #1 echo 'prompt *# ' #2 echo 'prompt *# ' OK: 2 of 2 tests passed $ ./clitest --prompt '%* ' dev/test/option-prompt-glob-space.sh #1 echo 'prompt %* ' #2 echo 'prompt %* ' OK: 2 of 2 tests passed $ ./clitest --prompt '*% ' dev/test/option-prompt-glob-space.sh #1 echo 'prompt *% ' #2 echo 'prompt *% ' OK: 2 of 2 tests passed $ ``` ## Option --prompt: glob gotchas (chars only) ``` $ ./clitest --prompt '?' dev/test/option-prompt-glob-1.sh #1 echo 'prompt ?' #2 echo 'prompt ?' OK: 2 of 2 tests passed $ ./clitest --prompt '*' dev/test/option-prompt-glob-1.sh #1 echo 'prompt *' #2 echo 'prompt *' OK: 2 of 2 tests passed $ ./clitest --prompt '#' dev/test/option-prompt-glob-1.sh #1 echo 'prompt #' #2 echo 'prompt #' OK: 2 of 2 tests passed $ ./clitest --prompt '%' dev/test/option-prompt-glob-1.sh #1 echo 'prompt %' #2 echo 'prompt %' OK: 2 of 2 tests passed $ ./clitest --prompt '##' dev/test/option-prompt-glob-2.sh #1 echo 'prompt ##' #2 echo 'prompt ##' OK: 2 of 2 tests passed $ ./clitest --prompt '%%' dev/test/option-prompt-glob-2.sh #1 echo 'prompt %%' #2 echo 'prompt %%' OK: 2 of 2 tests passed $ ./clitest --prompt '#*' dev/test/option-prompt-glob-2.sh #1 echo 'prompt #*' #2 echo 'prompt #*' OK: 2 of 2 tests passed $ ./clitest --prompt '*#' dev/test/option-prompt-glob-2.sh #1 echo 'prompt *#' #2 echo 'prompt *#' OK: 2 of 2 tests passed $ ./clitest --prompt '%*' dev/test/option-prompt-glob-2.sh #1 echo 'prompt %*' #2 echo 'prompt %*' OK: 2 of 2 tests passed $ ./clitest --prompt '*%' dev/test/option-prompt-glob-2.sh #1 echo 'prompt *%' #2 echo 'prompt *%' OK: 2 of 2 tests passed $ ``` ## Options --pre-flight and --post-flight ``` $ ./clitest --pre-flight 'tt_test_number=99; tt_nr_total_tests=99' dev/test/ok-1.sh #100 echo ok OK: 100 of 100 tests passed $ ./clitest --post-flight 'tt_nr_total_fails=50' dev/test/ok-50.sh | tail -1 FAIL: 50 of 50 tests failed $ ./clitest --pre-flight 'false' dev/test/ok-1.sh clitest: Error: pre-flight command failed with status=1: false $ ``` ## Options terminator -- ``` $ ./clitest -t 99 -- --quiet clitest: Error: cannot read input file: --quiet $ ``` ## File - meaning STDIN (no support for now) ``` $ cat dev/test/ok-1.sh | ./clitest - clitest: Error: cannot read input file: - $ ``` ## Gotchas Exit code: normal and error ``` $ ./clitest dev/test/exit-code.sh #1 echo "ok" > /dev/null; echo $? #2 cp XXnotfoundXX foo 2> /dev/null; echo $? OK: 2 of 2 tests passed $ ``` STDIN and STDOUT ``` $ ./clitest dev/test/stdout-stderr.sh #1 echo "stdout" #2 echo "stdout" 2> /dev/null #3 echo "stderr" 1>&2 #4 echo "stdout" > /dev/null #5 echo "stdout" 2> /dev/null 1>&2 #6 cp XXnotfoundXX foo #7 cp XXnotfoundXX foo > /dev/null #8 cp XXnotfoundXX foo 2>&1 #9 cp XXnotfoundXX foo 2> /dev/null #10 cp XXnotfoundXX foo > /dev/null 2>&1 OK: 10 of 10 tests passed $ ``` Multiple commands in one line ``` $ ./clitest dev/test/multi-commands.sh #1 echo 1; echo 2; echo 3; echo 4; echo 5 #2 (echo 1; echo 2; echo 3; echo 4; echo 5) | sed -n 3p #3 (echo 1; echo 2; echo 3; echo 4; echo 5) | sed -n 3p OK: 3 of 3 tests passed $ ``` A `cd` command in one test should not affect the next ``` $ ./clitest dev/test/cd.sh dev/test/ok-2.sh Testing file dev/test/cd.sh #1 cd Testing file dev/test/ok-2.sh #2 echo ok #3 echo ok ok fail skip 1 - - dev/test/cd.sh 2 - - dev/test/ok-2.sh OK: 3 of 3 tests passed $ ``` Syntax: End-of-file or empty prompt closes the previous command ``` $ ./clitest dev/test/close-command.sh #1 echo 1 #2 echo 2 #3 echo 3 OK: 3 of 3 tests passed $ ``` Windows files (CR+LF) ``` $ ./clitest dev/test/windows.sh #1 echo "a file with CRLF line ending" #2 echo "inline output" #3 echo "inline regex" OK: 3 of 3 tests passed $ ``` Unicode chars ``` $ ./clitest dev/test/special-chars.sh | tail -1 OK: 206 of 206 tests passed $ ``` Blanks (space, tab, newline) in the output ``` $ ./clitest dev/test/blank-output.sh #1 echo ' ' #2 echo ' ' #3 printf '\t\n' #4 printf '\t\t\t\n' #5 printf ' \t \t\t \n' #6 printf '\n \n \n \n \n\n' #7 printf '\n\t\n\t\t\n\t\t\t\n\t\t\t\t\n\n' #8 printf '\n' #9 printf '\n\n' #10 printf '\n\n\n\n' OK: 10 of 10 tests passed $ ``` Files with no newline (`\n`) at the end 1. No empty prompt at the last line 2. Empty prompt at the last line 3. Inline output ``` $ ./clitest dev/test/no-nl-file-1.sh #1 printf '%s\n' 'a file with no \n at the last line' OK: 1 of 1 test passed $ ./clitest dev/test/no-nl-file-2.sh #1 printf '%s\n' 'another file with no \n at the last line' OK: 1 of 1 test passed $ ./clitest dev/test/no-nl-file-3.sh #1 printf '%s\n' 'oneliner, no \n' OK: 1 of 1 test passed $ ``` Commands whose output has no `\n` ``` $ ./clitest dev/test/no-nl-command.sh #1 printf 'ok\n' #2 printf 'fail' -------------------------------------------------------------------------------- [FAILED #2, line 6] printf 'fail' @@ -1 +1 @@ -fail +fail \ No newline at end of file -------------------------------------------------------------------------------- #3 printf 'ok\nok\nfail' -------------------------------------------------------------------------------- [FAILED #3, line 8] printf 'ok\nok\nfail' @@ -1,3 +1,3 @@ ok ok -fail +fail \ No newline at end of file -------------------------------------------------------------------------------- #4 printf 'ok\n' #5 printf 'fail' -------------------------------------------------------------------------------- [FAILED #5, line 17] printf 'fail' @@ -1 +1 @@ -fail +fail \ No newline at end of file -------------------------------------------------------------------------------- #6 printf 'ok'; echo #7 printf 'ok' FAIL: 3 of 7 tests failed $ ``` ## And now, the colored output tests ``` $ ./clitest --color yes --first dev/test/fail-2.sh #1 echo ok -------------------------------------------------------------------------------- [FAILED #1, line 1] echo ok @@ -1 +1 @@ -fail +ok -------------------------------------------------------------------------------- $ ``` clitest-0.0+git20150312.339b2d/dev/test/0000755000000000000000000000000012500322604015547 5ustar rootrootclitest-0.0+git20150312.339b2d/dev/test/fail-messages.out.txt0000644000000000000000000000307612500322604021644 0ustar rootroot-------------------------------------------------------------------------------- [FAILED #1, line 5] echo fail @@ -1 +1 @@ -ok +fail -------------------------------------------------------------------------------- [FAILED #2, line 6] echo fail @@ -1 +1 @@ -ok +fail -------------------------------------------------------------------------------- [FAILED #3, line 10] echo fail @@ -1 +1 @@ -ok +fail -------------------------------------------------------------------------------- [FAILED #4, line 12] echo fail @@ -1,3 +1 @@ -ok 1 -ok 2 -ok 3 +fail -------------------------------------------------------------------------------- [FAILED #5, line 20] echo fail @@ -1,5 +1 @@ -Lorem ipsum dolor sit amet, consectetur adipiscing elit. -Proin euismod blandit pharetra. -Vestibulum eu neque eget lorem gravida commodo a cursus massa. -Fusce sit amet lorem sem. -Donec eu quam leo. +fail -------------------------------------------------------------------------------- [FAILED #6, line 24] echo fail Expected 9 lines, got 1. -------------------------------------------------------------------------------- [FAILED #7, line 28] echo fail egrep '^[0-9]+$' failed in: fail -------------------------------------------------------------------------------- [FAILED #8, line 32] echo fail Perl regex '^[0-9]+$' not matched in: fail -------------------------------------------------------------------------------- [FAILED #9, line 36] echo fail Perl regex '^[0-9]+$' not matched in: fail -------------------------------------------------------------------------------- FAIL: 9 of 9 tests failed clitest-0.0+git20150312.339b2d/dev/test/cd.sh0000644000000000000000000000034512500322604016473 0ustar rootroot# Some tests may change the current dir. # This could affect the next tests. # This could affect the relative path for the next input file. # To avoid problems, the current dir is always reset when a new test file is read. $ cd clitest-0.0+git20150312.339b2d/dev/test/option-prompt.sh0000644000000000000000000000032512500322604020732 0ustar rootroot# Configurable prompt with --prompt prompt$ echo "1" #→ 1 prompt$ echo "2" 2 prompt$ # Series closed with empty prompt prompt$ echo "3" 3 prompt$ # Series closed with empty prompt (with no trailing space) clitest-0.0+git20150312.339b2d/dev/test/inline-match-egrep.sh0000644000000000000000000000632312500322604021557 0ustar rootroot# Inline matching method: --egrep # Matches a egrep-style regular expression in the command output # # In fact, it's a real egrep match: eval $command | egrep 'regex' # If egrep matched, we have a successful test. That means that in # a multiline result, even if just a single line matches the regex, # the test is considered OK. # # Test your regexes with egrep at the command line before adding # tests using them. # See man re_format in your system # http://www.freebsd.org/cgi/man.cgi?query=re_format&sektion=7 # Use anchors ^ and $ to match the full output text $ echo 'abc123' #→ --egrep ^abc123$ $ echo 'abc123' #→ --egrep ^abc.*3$ $ echo 'abc123' #→ --egrep ^abc[0-9]+$ # Omit one or both anchors to make a parcial match $ echo 'abc123' #→ --egrep ^abc $ echo 'abc123' #→ --egrep 123$ $ echo 'abc123' #→ --egrep [0-9]+$ $ echo 'abc123' #→ --egrep bc $ echo 'abc123' #→ --egrep . # Blanks are preserved, no escaping or quoting needed $ echo 'abc 123' #→ --egrep ^abc [0-9]+$ # Blank output can also be matched $ echo ' ' #→ --egrep ^ $ $ echo ' ' #→ --egrep ^ $ $ printf '\t\n' #→ --egrep ^ $ $ printf '\t\t\t\n' #→ --egrep ^ $ $ printf ' \t \t\t \n' #→ --egrep ^ $ # In some systems, the special sequence \t is expanded to a tab in # egrep regexes. You'll need to test in your system if that's the # case. I recommend using a literal tab to avoid problems. $ printf 'may\tfail' #→ --egrep ^may\tfail$ $ printf 'may\tfail' #→ --egrep ^may[\t]fail$ $ printf 'will\tmatch' #→ --egrep ^will match$ # Since it's an egrep test, regexes are not multiline. # You can only match a single line. # These tests will fail: $ printf 'will\nfail' #→ --egrep will.*fail $ printf 'will\nfail' #→ --egrep will\nfail # If one line of a multiline results matches, the test is OK $ printf '1\n2\n3\n4\nok\n' #→ --egrep ok # As egrep is used for the test and it ignores the line break, # you can match both full (with \n) and partial (without \n). $ printf 'ok' #→ --egrep ok $ printf 'ok\n' #→ --egrep ok # Syntax: Must be exactly one space before and after --egrep $ echo 'fail' #→ --egrep fail with 2 spaces $ echo 'fail' #→ --egrep fail with tab # Syntax: The extra space after '--egrep ' is already part of the regex $ echo ' ok' #→ --egrep ok # Syntax: The space after --egrep is required. # When missing, the '--egrep' is considered a normal text. $ echo '--egrep' #→ --egrep # Syntax: Make sure we won't catch partial matches. $ echo '--egreppal' #→ --egreppal # Syntax: To insert a literal text that begins with '--egrep ' # just prefix it with --text. $ echo '--egrep is cool' #→ --text --egrep is cool # Syntax: Empty inline output contents are considered an error # Note: Tested in a separate file: inline-match-egrep-error-1.sh # # $ echo 'no contents' #→ --egrep clitest-0.0+git20150312.339b2d/dev/test/inline-match-perl.sh0000644000000000000000000000673112500322604021422 0ustar rootroot# Inline matching method: --perl # Matches a Perl-style regular expression in the command output # You can also use the friendlier alias: --regex # # In fact, it's a real Perl match: perl -0777 -ne "exit(!m'regex')" # If Perl matched, we have a successful test. # All the test output lines are matched as a single string. # No modifiers are used by default, inform yours if needed: (?ims) # You don't need to escape the ' delimiter, the script will do it for you. # Just write your regex not worrying about delimiters. # Use anchors ^ and $ to match the full output text $ echo 'abc123' #→ --perl ^abc123$ $ echo 'abc123' #→ --perl ^abc.*3$ $ echo 'abc123' #→ --perl ^abc[0-9]+$ # Omit one or both anchors to make a parcial match $ echo 'abc123' #→ --perl ^abc $ echo 'abc123' #→ --perl 123$ $ echo 'abc123' #→ --perl [0-9]+$ $ echo 'abc123' #→ --perl bc $ echo 'abc123' #→ --perl . # Blanks are preserved, no escaping or quoting needed $ echo 'abc 123' #→ --perl ^abc [0-9]+$ # Blank output can also be matched $ echo ' ' #→ --perl ^ $ $ echo ' ' #→ --perl ^ $ $ printf '\t\n' #→ --perl ^ $ $ printf '\t\t\t\n' #→ --perl ^ $ $ printf ' \t \t\t \n' #→ --perl ^ $ # You don't need to escape any delimiters, escapes are handled by the script $ echo '01/01/2013' #→ --perl ^../../....$ $ echo "won't fail" #→ --perl ^won't \w+$ # To match a tab, you can use \t or a literal tab $ printf 'will\tmatch' #→ --perl ^will\tmatch$ $ printf 'will\tmatch' #→ --perl ^will[\t]match$ $ printf 'will\tmatch' #→ --perl ^will match$ # You need to inform the (?i) modifier to match ignoring case $ printf 'will\nfail' #→ --perl ^WILL $ printf 'will\nmatch' #→ --perl (?i)^WILL # You need to inform the (?s) modifier for the dot to match \n $ printf 'will\nfail' #→ --perl ^will.fail$ $ printf 'will\nmatch' #→ --perl (?s)^will.match$ # You need to inform the (?m) modifier for ^ and $ to match inner lines $ printf 'will\nfail' #→ --perl ^fail $ printf 'will\nmatch' #→ --perl (?m)^match # Perl ignores the last \n, in both the text and the regex $ printf 'ok' #→ --perl ^ok$ $ printf 'ok\n' #→ --perl ^ok$ $ printf '1\n2\n3\n' #→ --perl ^1\n2\n3\n$ $ printf '1\n2\n3\n' #→ --perl ^1\n2\n3$ # Syntax: Must be exactly one space before and after --perl $ echo 'fail' #→ --perl fail with 2 spaces $ echo 'fail' #→ --perl fail with tab # Syntax: The extra space after '--perl ' is already part of the regex $ echo ' ok' #→ --perl ok # Syntax: The space after --perl is required. # When missing, the '--perl' is considered a normal text. $ echo '--perl' #→ --perl # Syntax: Make sure we won't catch partial matches. $ echo '--perlism' #→ --perlism # Syntax: To insert a literal text that begins with '--perl ' # just prefix it with --text. $ echo '--perl is cool' #→ --text --perl is cool # Syntax: Empty inline output contents are considered an error # Note: Tested in a separate file: inline-match-perl-error-1.sh # # $ echo 'no contents' #→ --perl clitest-0.0+git20150312.339b2d/dev/test/option-prefix.sh0000644000000000000000000000171212500322604020707 0ustar rootroot# Test file for the --prefix option # Command blocks here are prefixed by 4 spaces (Markdown-style) # Run with --prefix ' ' or --prefix 4 $ echo "1" #→ 1 $ echo "2" 2 # Any non-prefixed line closes the previous command block. # The empty $ line is not needed. # All other non-indented text is just ignored: $ echo "ignored" # not indented $ echo "ignored" #→ not indented # Lines with the wrong indentation are also ignored $ echo "ignored" # 3 spaces $ echo "ignored" # 5 spaces # Multiple blocks supported in a single file $ echo "3" 3 # What about prefixed blocks with no commands? Prefixed line with no prompt: ignored. But wait, here comes a command: $ echo "4" 4 $ Last command closed by the empty prompt. $ echo "5" #→ 5 Last command is auto-closed (inline output). # Blank lines in the output are supported $ echo; echo "6"; echo; echo "7" 6 7 # Nice. clitest-0.0+git20150312.339b2d/dev/test/option-prompt-glob-2.sh0000644000000000000000000000135012500322604022011 0ustar rootroot/// Gotcha: glob chars as --prompt: ? * # % ## %% #* *# %* *% /// Inline output ?echo 'prompt ?' #→ prompt ? *echo 'prompt *' #→ prompt * #echo 'prompt #' #→ prompt # %echo 'prompt %' #→ prompt % ##echo 'prompt ##' #→ prompt ## %%echo 'prompt %%' #→ prompt %% #*echo 'prompt #*' #→ prompt #* *#echo 'prompt *#' #→ prompt *# %*echo 'prompt %*' #→ prompt %* *%echo 'prompt *%' #→ prompt *% /// Normal output ?echo 'prompt ?' prompt ? ? *echo 'prompt *' prompt * * #echo 'prompt #' prompt # # %echo 'prompt %' prompt % % ##echo 'prompt ##' prompt ## ## %%echo 'prompt %%' prompt %% %% #*echo 'prompt #*' prompt #* #* *#echo 'prompt *#' prompt *# *# %*echo 'prompt %*' prompt %* %* *%echo 'prompt *%' prompt *% *% clitest-0.0+git20150312.339b2d/dev/test/inline-match-file-error-1.sh0000644000000000000000000000005212500322604022652 0ustar rootroot$ echo 'error: no contents' #→ --file clitest-0.0+git20150312.339b2d/dev/test/lorem-ipsum.txt0000644000000000000000000000030512500322604020557 0ustar rootrootLorem ipsum dolor sit amet, consectetur adipiscing elit. Proin euismod blandit pharetra. Vestibulum eu neque eget lorem gravida commodo a cursus massa. Fusce sit amet lorem sem. Donec eu quam leo. clitest-0.0+git20150312.339b2d/dev/test/option-prefix-glob.sh0000644000000000000000000000146112500322604021631 0ustar rootroot/// Gotcha: glob chars as --prefix: ? * # ## #* *# #*# % %% %* *% %*% /// Inline output ?$ echo 'prefix ?' #→ prefix ? *$ echo 'prefix *' #→ prefix * #$ echo 'prefix #' #→ prefix # %$ echo 'prefix %' #→ prefix % ##$ echo 'prefix ##' #→ prefix ## %%$ echo 'prefix %%' #→ prefix %% #*$ echo 'prefix #*' #→ prefix #* *#$ echo 'prefix *#' #→ prefix *# %*$ echo 'prefix %*' #→ prefix %* *%$ echo 'prefix *%' #→ prefix *% /// Normal output ?$ echo 'prefix ?' ?prefix ? ?$ *$ echo 'prefix *' *prefix * *$ #$ echo 'prefix #' #prefix # #$ %$ echo 'prefix %' %prefix % %$ ##$ echo 'prefix ##' ##prefix ## ##$ %%$ echo 'prefix %%' %%prefix %% %%$ #*$ echo 'prefix #*' #*prefix #* #*$ *#$ echo 'prefix *#' *#prefix *# *#$ %*$ echo 'prefix %*' %*prefix %* %*$ *%$ echo 'prefix *%' *%prefix *% *%$ clitest-0.0+git20150312.339b2d/dev/test/no-nl-file-3.sh0000644000000000000000000000006712500322604020206 0ustar rootroot$ printf '%s\n' 'oneliner, no \n' #→ oneliner, no \nclitest-0.0+git20150312.339b2d/dev/test/inline-match-lines-error-2.sh0000644000000000000000000000006112500322604023046 0ustar rootroot$ echo 'error: negative number' #→ --lines -1 clitest-0.0+git20150312.339b2d/dev/test/ok-100.sh0000644000000000000000000000337012500322604017015 0ustar rootroot$ echo 1 #→ 1 $ echo 2 #→ 2 $ echo 3 #→ 3 $ echo 4 #→ 4 $ echo 5 #→ 5 $ echo 6 #→ 6 $ echo 7 #→ 7 $ echo 8 #→ 8 $ echo 9 #→ 9 $ echo 10 #→ 10 $ echo 11 #→ 11 $ echo 12 #→ 12 $ echo 13 #→ 13 $ echo 14 #→ 14 $ echo 15 #→ 15 $ echo 16 #→ 16 $ echo 17 #→ 17 $ echo 18 #→ 18 $ echo 19 #→ 19 $ echo 20 #→ 20 $ echo 21 #→ 21 $ echo 22 #→ 22 $ echo 23 #→ 23 $ echo 24 #→ 24 $ echo 25 #→ 25 $ echo 26 #→ 26 $ echo 27 #→ 27 $ echo 28 #→ 28 $ echo 29 #→ 29 $ echo 30 #→ 30 $ echo 31 #→ 31 $ echo 32 #→ 32 $ echo 33 #→ 33 $ echo 34 #→ 34 $ echo 35 #→ 35 $ echo 36 #→ 36 $ echo 37 #→ 37 $ echo 38 #→ 38 $ echo 39 #→ 39 $ echo 40 #→ 40 $ echo 41 #→ 41 $ echo 42 #→ 42 $ echo 43 #→ 43 $ echo 44 #→ 44 $ echo 45 #→ 45 $ echo 46 #→ 46 $ echo 47 #→ 47 $ echo 48 #→ 48 $ echo 49 #→ 49 $ echo 50 #→ 50 $ echo 51 #→ 51 $ echo 52 #→ 52 $ echo 53 #→ 53 $ echo 54 #→ 54 $ echo 55 #→ 55 $ echo 56 #→ 56 $ echo 57 #→ 57 $ echo 58 #→ 58 $ echo 59 #→ 59 $ echo 60 #→ 60 $ echo 61 #→ 61 $ echo 62 #→ 62 $ echo 63 #→ 63 $ echo 64 #→ 64 $ echo 65 #→ 65 $ echo 66 #→ 66 $ echo 67 #→ 67 $ echo 68 #→ 68 $ echo 69 #→ 69 $ echo 70 #→ 70 $ echo 71 #→ 71 $ echo 72 #→ 72 $ echo 73 #→ 73 $ echo 74 #→ 74 $ echo 75 #→ 75 $ echo 76 #→ 76 $ echo 77 #→ 77 $ echo 78 #→ 78 $ echo 79 #→ 79 $ echo 80 #→ 80 $ echo 81 #→ 81 $ echo 82 #→ 82 $ echo 83 #→ 83 $ echo 84 #→ 84 $ echo 85 #→ 85 $ echo 86 #→ 86 $ echo 87 #→ 87 $ echo 88 #→ 88 $ echo 89 #→ 89 $ echo 90 #→ 90 $ echo 91 #→ 91 $ echo 92 #→ 92 $ echo 93 #→ 93 $ echo 94 #→ 94 $ echo 95 #→ 95 $ echo 96 #→ 96 $ echo 97 #→ 97 $ echo 98 #→ 98 $ echo 99 #→ 99 $ echo 100 #→ 100 clitest-0.0+git20150312.339b2d/dev/test/inline-match-file.sh0000644000000000000000000000233112500322604021367 0ustar rootroot# Inline matching method: --file # Matches the contents of the informed file # Just inform the file path (no quotes, no escapes) $ printf '$ echo ok\nok\n' #→ --file ok-1.sh # Absolute paths are also supported $ echo 'ok' > /tmp/foo.txt $ echo 'ok' #→ --file /tmp/foo.txt # Syntax: Must be exactly one space before and after --file $ echo 'fail' #→ --file fail-with-2-spaces.txt $ echo 'fail' #→ --file fail-with-tab.txt # Syntax: The extra space after '--file ' is already part of the filename #$ echo 'fail' #→ --file file-with-leading-space-in-name.txt # Syntax: The space after --file is required. # When missing, the '--file' is considered a normal text. $ echo '--file' #→ --file # Syntax: Make sure we won't catch partial matches. $ echo '--filer' #→ --filer # Syntax: To insert a literal text that begins with '--file ' # just prefix it with --text. $ echo '--file is cool' #→ --text --file is cool # Syntax: Empty inline output contents are considered an error # Note: Tested in a separate file: inline-match-file-error-1.sh # # $ echo 'no contents' #→ --file clitest-0.0+git20150312.339b2d/dev/test/inline-match-eval-error-1.sh0000644000000000000000000000005212500322604022662 0ustar rootroot$ echo 'error: no contents' #→ --eval clitest-0.0+git20150312.339b2d/dev/test/inline-match-perl-error-2.sh0000644000000000000000000000074112500322604022703 0ustar rootroot$ echo "error: malformed regex" #→ --perl ( # Some perl errors: # $ echo | perl -0777 -ne 'exit(!/(/)' ; echo $? # Unmatched ( in regex; marked by <-- HERE in m/( <-- HERE / at -e line 1. # 255 # $ echo | perl -0777 -ne 'exit(!/[/)' ; echo $? # Unmatched [ in regex; marked by <-- HERE in m/[ <-- HERE / at -e line 1. # 255 # $ echo | perl -0777 -ne 'exit(!/**/)' ; echo $? # Quantifier follows nothing in regex; marked by <-- HERE in m/* <-- HERE */ at -e line 1. # 255 # $ clitest-0.0+git20150312.339b2d/dev/test/fail-messages.md0000644000000000000000000000066412500322604020617 0ustar rootroot# Fail Messages ## inline text $ echo fail #→ ok $ echo fail #→ --eval echo ok ## normal command $ echo fail ok $ echo fail ok 1 ok 2 ok 3 $ ## inline --file $ echo fail #→ --file lorem-ipsum.txt ## inline --lines $ echo fail #→ --lines 9 ## inline --egrep $ echo fail #→ --egrep ^[0-9]+$ ## inline --perl $ echo fail #→ --perl ^[0-9]+$ ## inline --regex $ echo fail #→ --regex ^[0-9]+$ clitest-0.0+git20150312.339b2d/dev/test/option-prompt-glob-space.sh0000644000000000000000000000146112500322604022746 0ustar rootroot/// Gotcha: glob chars (plus space) as --prompt: ? * # ## #* *# % %% %* *% /// Inline output ? echo 'prompt ? ' #→ prompt ? * echo 'prompt * ' #→ prompt * # echo 'prompt # ' #→ prompt # % echo 'prompt % ' #→ prompt % ## echo 'prompt ## ' #→ prompt ## %% echo 'prompt %% ' #→ prompt %% #* echo 'prompt #* ' #→ prompt #* *# echo 'prompt *# ' #→ prompt *# %* echo 'prompt %* ' #→ prompt %* *% echo 'prompt *% ' #→ prompt *% /// Normal output ? echo 'prompt ? ' prompt ? ? * echo 'prompt * ' prompt * * # echo 'prompt # ' prompt # # % echo 'prompt % ' prompt % % ## echo 'prompt ## ' prompt ## ## %% echo 'prompt %% ' prompt %% %% #* echo 'prompt #* ' prompt #* #* *# echo 'prompt *# ' prompt *# *# %* echo 'prompt %* ' prompt %* %* *% echo 'prompt *% ' prompt *% *% clitest-0.0+git20150312.339b2d/dev/test/fail-1.sh0000644000000000000000000000001712500322604017152 0ustar rootroot$ echo ok fail clitest-0.0+git20150312.339b2d/dev/test/option-prompt-unicode.sh0000644000000000000000000000030112500322604022350 0ustar rootroot# Configurable prompt with --prompt ♥ echo "1" #→ 1 ♥ echo "2" 2 ♥ # Series closed with empty prompt ♥ echo "3" 3 ♥ # Series closed with empty prompt (with no trailing space) clitest-0.0+git20150312.339b2d/dev/test/inline-match-file-error-2.sh0000644000000000000000000000007112500322604022654 0ustar rootroot$ echo "error: file not found" #→ --file XXnotfoundXX clitest-0.0+git20150312.339b2d/dev/test/special-chars.sh0000644000000000000000000001064412500322604020626 0ustar rootroot# ASCII $ echo '0123456789' #→ 0123456789 $ echo 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' #→ ABCDEFGHIJKLMNOPQRSTUVWXYZ $ echo 'abcdefghijklmnopqrstuvwxyz' #→ abcdefghijklmnopqrstuvwxyz $ echo ' ' #→ $ echo '!' #→ ! $ echo '"' #→ " $ echo '#' #→ # $ echo '$' #→ $ $ echo '%' #→ % $ echo '&' #→ & $ echo "'" #→ ' $ echo '(' #→ ( $ echo ')' #→ ) $ echo '*' #→ * $ echo '+' #→ + $ echo ',' #→ , $ echo '-' #→ - $ echo '.' #→ . $ echo '/' #→ / $ echo ':' #→ : $ echo ';' #→ ; $ echo '<' #→ < $ echo '=' #→ = $ echo '>' #→ > $ echo '?' #→ ? $ echo '@' #→ @ $ echo '[' #→ [ $ echo '\' #→ \ $ echo ']' #→ ] $ echo '^' #→ ^ $ echo '_' #→ _ $ echo '`' #→ ` $ echo '{' #→ { $ echo '|' #→ | $ echo '}' #→ } $ echo '~' #→ ~ # !ASCII $ echo '¡' #→ ¡ $ echo '¢' #→ ¢ $ echo '£' #→ £ $ echo '¤' #→ ¤ $ echo '¥' #→ ¥ $ echo '¦' #→ ¦ $ echo '§' #→ § $ echo '¨' #→ ¨ $ echo '©' #→ © $ echo 'ª' #→ ª $ echo '«' #→ « $ echo '¬' #→ ¬ $ echo '­' #→ ­ $ echo '®' #→ ® $ echo '°' #→ ° $ echo '±' #→ ± $ echo '²' #→ ² $ echo '³' #→ ³ $ echo '´' #→ ´ $ echo 'µ' #→ µ $ echo '¶' #→ ¶ $ echo '·' #→ · $ echo '¹' #→ ¹ $ echo 'º' #→ º $ echo '»' #→ » $ echo '¼' #→ ¼ $ echo '½' #→ ½ $ echo '¾' #→ ¾ $ echo '¿' #→ ¿ $ echo 'À' #→ À $ echo 'Á' #→ Á $ echo 'Â' #→  $ echo 'Ã' #→ à $ echo 'Ä' #→ Ä $ echo 'Å' #→ Å $ echo 'Æ' #→ Æ $ echo 'Ç' #→ Ç $ echo 'È' #→ È $ echo 'É' #→ É $ echo 'Ê' #→ Ê $ echo 'Ë' #→ Ë $ echo 'Ì' #→ Ì $ echo 'Í' #→ Í $ echo 'Î' #→ Î $ echo 'Ï' #→ Ï $ echo 'Ð' #→ Ð $ echo 'Ñ' #→ Ñ $ echo 'Ò' #→ Ò $ echo 'Ó' #→ Ó $ echo 'Ô' #→ Ô $ echo 'Õ' #→ Õ $ echo 'Ö' #→ Ö $ echo '×' #→ × $ echo 'Ø' #→ Ø $ echo 'Ù' #→ Ù $ echo 'Ú' #→ Ú $ echo 'Û' #→ Û $ echo 'Ü' #→ Ü $ echo 'Ý' #→ Ý $ echo 'ß' #→ ß $ echo 'à' #→ à $ echo 'á' #→ á $ echo 'â' #→ â $ echo 'ã' #→ ã $ echo 'ä' #→ ä $ echo 'å' #→ å $ echo 'æ' #→ æ $ echo 'ç' #→ ç $ echo 'è' #→ è $ echo 'é' #→ é $ echo 'ê' #→ ê $ echo 'ë' #→ ë $ echo 'ì' #→ ì $ echo 'í' #→ í $ echo 'î' #→ î $ echo 'ï' #→ ï $ echo 'ð' #→ ð $ echo 'ñ' #→ ñ $ echo 'ò' #→ ò $ echo 'ó' #→ ó $ echo 'ô' #→ ô $ echo 'õ' #→ õ $ echo 'ö' #→ ö $ echo '÷' #→ ÷ $ echo 'ø' #→ ø $ echo 'ù' #→ ù $ echo 'ú' #→ ú $ echo 'û' #→ û $ echo 'ü' #→ ü $ echo 'ý' #→ ý $ echo 'ÿ' #→ ÿ $ echo 'Œ' #→ Œ $ echo 'œ' #→ œ $ echo 'Ÿ' #→ Ÿ $ echo 'ƒ' #→ ƒ $ echo 'Α' #→ Α $ echo 'Β' #→ Β $ echo 'Π' #→ Π $ echo 'Σ' #→ Σ $ echo 'Φ' #→ Φ $ echo 'Ω' #→ Ω $ echo 'α' #→ α $ echo 'β' #→ β $ echo 'η' #→ η $ echo 'μ' #→ μ $ echo 'π' #→ π $ echo 'ω' #→ ω $ echo 'ϖ' #→ ϖ $ echo '–' #→ – $ echo '—' #→ — $ echo '‘' #→ ‘ $ echo '’' #→ ’ $ echo '‚' #→ ‚ $ echo '“' #→ “ $ echo '”' #→ ” $ echo '„' #→ „ $ echo '†' #→ † $ echo '‡' #→ ‡ $ echo '•' #→ • $ echo '…' #→ … $ echo '‰' #→ ‰ $ echo '′' #→ ′ $ echo '″' #→ ″ $ echo '‹' #→ ‹ $ echo '›' #→ › $ echo '‾' #→ ‾ $ echo '⁄' #→ ⁄ $ echo '€' #→ € $ echo '™' #→ ™ $ echo '←' #→ ← $ echo '↑' #→ ↑ $ echo '→' #→ → $ echo '↓' #→ ↓ $ echo '↔' #→ ↔ $ echo '↵' #→ ↵ $ echo '⇐' #→ ⇐ $ echo '⇑' #→ ⇑ $ echo '⇒' #→ ⇒ $ echo '⇓' #→ ⇓ $ echo '⇔' #→ ⇔ $ echo '∈' #→ ∈ $ echo '∉' #→ ∉ $ echo '∋' #→ ∋ $ echo '∏' #→ ∏ $ echo '∑' #→ ∑ $ echo '−' #→ − $ echo '∗' #→ ∗ $ echo '√' #→ √ $ echo '∝' #→ ∝ $ echo '∞' #→ ∞ $ echo '∫' #→ ∫ $ echo '∴' #→ ∴ $ echo '∼' #→ ∼ $ echo '≅' #→ ≅ $ echo '≈' #→ ≈ $ echo '≠' #→ ≠ $ echo '≡' #→ ≡ $ echo '≤' #→ ≤ $ echo '≥' #→ ≥ $ echo '⊂' #→ ⊂ $ echo '⊃' #→ ⊃ $ echo '⊄' #→ ⊄ $ echo '⊆' #→ ⊆ $ echo '⊇' #→ ⊇ $ echo '◊' #→ ◊ $ echo '♠' #→ ♠ $ echo '♣' #→ ♣ $ echo '♥' #→ ♥ $ echo '♦' #→ ♦ $ echo '★' #→ ★ clitest-0.0+git20150312.339b2d/dev/test/windows.sh0000644000000000000000000000025012500322604017572 0ustar rootroot$ echo "a file with CRLF line ending" a file with CRLF line ending $ $ echo "inline output" #→ inline output $ echo "inline regex" #→ --regex ^inline regex$ clitest-0.0+git20150312.339b2d/dev/test/fail-2.sh0000644000000000000000000000004412500322604017153 0ustar rootroot$ echo ok fail $ echo ok #→ fail clitest-0.0+git20150312.339b2d/dev/test/no-test-found.sh0000644000000000000000000000034612500322604020610 0ustar rootroot# A file with comments, but no test :( # It raises an error when processed, because the user must # take action in this situation: # # 1) fix the file, adding the tests # or # 2) stop sending this file to the tester clitest-0.0+git20150312.339b2d/dev/test/inline-match-lines-error-3.sh0000644000000000000000000000005712500322604023054 0ustar rootroot$ echo 'error: float number' #→ --lines 1.0 clitest-0.0+git20150312.339b2d/dev/test/inline-match-eval-error-2.sh0000644000000000000000000000005612500322604022667 0ustar rootroot$ echo 'error: syntax error' #→ --eval $( clitest-0.0+git20150312.339b2d/dev/test/inline-match-file-error-3.sh0000644000000000000000000000005512500322604022657 0ustar rootroot$ echo "error: directory" #→ --file /etc/ clitest-0.0+git20150312.339b2d/dev/test/inline.sh0000644000000000000000000000210412500322604017356 0ustar rootroot# The blank space before the #→ marker matters? $ echo 'one space' #→ one space $ echo 'one tab' #→ one tab $ echo 'multi spaces' #→ multi spaces $ echo 'multi tabs' #→ multi tabs $ echo 'mixed' #→ mixed # Blank lines and comments in the middle. # No need to 'close' previous command. # Leading and trailing blank space are preserved? $ echo ' leading space' #→ leading space $ echo ' leading spaces' #→ leading spaces $ printf '\tleading tab\n' #→ leading tab $ printf '\t\tleading tabs\n' #→ leading tabs $ echo 'trailing space ' #→ trailing space $ echo 'trailing spaces ' #→ trailing spaces $ printf 'trailing tab\t\n' #→ trailing tab $ printf 'trailing tabs\t\t\n' #→ trailing tabs # Blank output $ echo ' ' #→ $ echo ' ' #→ $ printf '\t\n' #→ $ printf '\t\t\t\n' #→ $ printf ' \t \t\t \n' #→ # Inline results have precedence over normal results $ echo "both inline and normal output" #→ both inline and normal output Inline wins. The normal output is just ignored. $ clitest-0.0+git20150312.339b2d/dev/test/option-prompt-glob-1.sh0000644000000000000000000000071612500322604022015 0ustar rootroot/// Gotcha: glob chars as --prompt: ? * # % /// Note: These tests are separated from the two chars globs /// to avoid partial matches with wrong output. /// Inline output (one char) ?echo 'prompt ?' #→ prompt ? *echo 'prompt *' #→ prompt * #echo 'prompt #' #→ prompt # %echo 'prompt %' #→ prompt % /// Normal output (one char) ?echo 'prompt ?' prompt ? ? *echo 'prompt *' prompt * * #echo 'prompt #' prompt # # %echo 'prompt %' prompt % % clitest-0.0+git20150312.339b2d/dev/test/option-diff-options.sh0000644000000000000000000000027212500322604022013 0ustar rootroot# Configurable diff options with --diff-options $ echo " diff -w to ignore spaces " diff -w to ignore spaces $ echo " diff -w now inline " #→ diff -w now inline clitest-0.0+git20150312.339b2d/dev/test/empty-file.sh0000644000000000000000000000000012500322604020144 0ustar rootrootclitest-0.0+git20150312.339b2d/dev/test/ok-2.sh0000644000000000000000000000004012500322604016645 0ustar rootroot$ echo ok ok $ echo ok #→ ok clitest-0.0+git20150312.339b2d/dev/test/no-nl-file-2.sh0000644000000000000000000000014512500322604020202 0ustar rootroot$ printf '%s\n' 'another file with no \n at the last line' another file with no \n at the last line $clitest-0.0+git20150312.339b2d/dev/test/blank-output.sh0000644000000000000000000000042712500322604020533 0ustar rootroot$ echo ' ' $ echo ' ' $ printf '\t\n' $ printf '\t\t\t\n' $ printf ' \t \t\t \n' $ printf '\n \n \n \n \n\n' $ printf '\n\t\n\t\t\n\t\t\t\n\t\t\t\t\n\n' $ printf '\n' $ printf '\n\n' $ printf '\n\n\n\n' $ clitest-0.0+git20150312.339b2d/dev/test/exit-code.sh0000644000000000000000000000013412500322604017762 0ustar rootroot$ echo "ok" > /dev/null; echo $? 0 $ cp XXnotfoundXX foo 2> /dev/null; echo $? 1 clitest-0.0+git20150312.339b2d/dev/test/no-nl-command.sh0000644000000000000000000000072412500322604020545 0ustar rootroot# All results assume a trailing newline (\n) at the last line. # Outputs with no \n at the end cannot be tested directly. $ printf 'ok\n' ok $ printf 'fail' fail $ printf 'ok\nok\nfail' ok ok fail $ # The same applies for inline output. $ printf 'ok\n' #→ ok $ printf 'fail' #→ fail # An easy workaround is to add an empty 'echo' at the end: $ printf 'ok'; echo #→ ok # Another workaround is to use --regex $ printf 'ok' #→ --regex ^ok$ clitest-0.0+git20150312.339b2d/dev/test/inline-match-eval.sh0000644000000000000000000000427512500322604021410 0ustar rootroot# Inline matching method: --eval # Matches the text output from an arbitrary shell command # Run a simple command $ folder=$(pwd) $ echo $folder #→ --eval pwd # Read the contents of a variable $ var='abc' $ echo abc #→ --eval echo $var # Use arithmetic expansion $ echo 4 #→ --eval echo $((2+2)) # Run a subshell $ today=$(date +%D) $ echo "Today is $today" #→ --eval echo "Today is $(date +%D)" # You can also match lines without the final \n $ printf 'ok' #→ --eval printf 'ok' # Blanks are preserved $ echo ' leading space' #→ --eval echo ' leading space' $ echo ' leading spaces' #→ --eval echo ' leading spaces' $ printf '\tleading tab\n' #→ --eval printf '\tleading tab\n' $ printf '\t\tleading tabs\n' #→ --eval printf '\t\tleading tabs\n' $ echo 'trailing space ' #→ --eval echo 'trailing space ' $ echo 'trailing spaces ' #→ --eval echo 'trailing spaces ' $ printf 'trailing tab\t\n' #→ --eval printf 'trailing tab\t\n' $ printf 'trailing tabs\t\t\n' #→ --eval printf 'trailing tabs\t\t\n' $ echo ' ' #→ --eval echo ' ' $ echo ' ' #→ --eval echo ' ' $ printf '\t\n' #→ --eval printf '\t\n' $ printf '\t\t\t\n' #→ --eval printf '\t\t\t\n' $ printf ' \t \t\t \n' #→ --eval printf ' \t \t\t \n' # Syntax: Must be exactly one space before and after --eval $ echo 'fail' #→ --eval fail with 2 spaces $ echo 'fail' #→ --eval fail with tab # Syntax: The space after --eval is required. # When missing, the '--eval' is considered a normal text. $ echo '--eval' #→ --eval # Syntax: Make sure we won't catch partial matches. $ echo '--evaluate' #→ --evaluate # Syntax: To insert a literal text that begins with '--eval ' # just prefix it with --text. $ echo '--eval is evil' #→ --text --eval is evil # Syntax: Empty inline output contents are considered an error # Note: Tested in separate files: inline-match-eval-error-?.sh # # $ echo 'no contents' #→ --eval clitest-0.0+git20150312.339b2d/dev/test/multi-commands.sh0000644000000000000000000000025312500322604021034 0ustar rootroot$ echo 1; echo 2; echo 3; echo 4; echo 5 1 2 3 4 5 $ (echo 1; echo 2; echo 3; echo 4; echo 5) | sed -n 3p 3 $ (echo 1; echo 2; echo 3; echo 4; echo 5) | sed -n 3p #→ 3 clitest-0.0+git20150312.339b2d/dev/test/inline-match-egrep-error-1.sh0000644000000000000000000000005312500322604023036 0ustar rootroot$ echo 'error: no contents' #→ --egrep clitest-0.0+git20150312.339b2d/dev/test/no-nl-file-1.sh0000644000000000000000000000012712500322604020201 0ustar rootroot$ printf '%s\n' 'a file with no \n at the last line' a file with no \n at the last lineclitest-0.0+git20150312.339b2d/dev/test/close-command.sh0000644000000000000000000000041012500322604020617 0ustar rootroot# Syntax: Empty prompt (with space) closes the previous command $ echo 1 1 $ # Syntax: Empty prompt (no space) closes the previous command $ echo 2 2 $ # Syntax: Repeated empty prompts are OK $ $ $ # Syntax: End-of-file closes the last command $ echo 3 3 clitest-0.0+git20150312.339b2d/dev/test/option-inline-prefix.sh0000644000000000000000000000021612500322604022161 0ustar rootroot# Configurable prefix with --inline-prefix $ echo "1 space" #==> 1 space $ echo "8 spaces" #==> 8 spaces $ echo "2 tabs" #==> 2 tabs clitest-0.0+git20150312.339b2d/dev/test/stdout-stderr.sh0000644000000000000000000000124712500322604020732 0ustar rootroot# Output from both STDOUT and STDERR are catched by the tester. ### STDOUT # Showing STOUT $ echo "stdout" stdout $ echo "stdout" 2> /dev/null stdout $ # Redirecting STDOUT to STDERR $ echo "stderr" 1>&2 stderr $ # Closing STDOUT $ echo "stdout" > /dev/null $ echo "stdout" 2> /dev/null 1>&2 $ ### STDERR # Showing STDERR $ cp XXnotfoundXX foo cp: XXnotfoundXX: No such file or directory $ cp XXnotfoundXX foo > /dev/null cp: XXnotfoundXX: No such file or directory $ # Redirecting STDERR to STDOUT $ cp XXnotfoundXX foo 2>&1 cp: XXnotfoundXX: No such file or directory $ # Closing STDERR $ cp XXnotfoundXX foo 2> /dev/null $ cp XXnotfoundXX foo > /dev/null 2>&1 $ clitest-0.0+git20150312.339b2d/dev/test/empty-prompts-file.sh0000644000000000000000000000005712500322604021662 0ustar rootroot$ $ $ # Comment $ $ $ # Comment $ $ $ clitest-0.0+git20150312.339b2d/dev/test/inline-match-lines.sh0000644000000000000000000000302112500322604021557 0ustar rootroot# Inline matching method: --lines # Count the number of lines in the output $ a=1 #→ --lines 0 $ echo 'ok' #→ --lines 1 $ printf '1\n2\n3\n' #→ --lines 3 # Lines without the final \n count as one full line $ printf 'no-nl' #→ --lines 1 $ printf '1\n2\nno-nl' #→ --lines 3 # The error message is a short sentence, not a diff # Example: Expected 99 lines, got 1. $ echo 'fail' #→ --lines 99 $ echo 'fail' #→ --lines 0 # Syntax: Must be exactly one space before and after --lines $ echo 'fail' #→ --lines fail with 2 spaces $ echo 'fail' #→ --lines fail with tab # Syntax: The space after --lines is required. # When missing, the '--lines' is considered a normal text. $ echo '--lines' #→ --lines # Syntax: Make sure we won't catch partial matches. $ echo '--linesout' #→ --linesout # Syntax: To insert a literal text that begins with '--lines ' # just prefix it with --text. $ echo '--lines is cool' #→ --text --lines is cool # Note: The following are tested in separate files: # inline-match-lines-error-?.sh # # Syntax: Empty inline output contents are considered an error # # $ echo 'no contents' #→ --lines # # Syntax: Must be an integer number # # $ echo 'fail' #→ --lines -1 # $ echo 'fail' #→ --lines 1.0 # $ echo 'fail' #→ --lines foo clitest-0.0+git20150312.339b2d/dev/test/inline-match-lines-error-1.sh0000644000000000000000000000005312500322604023046 0ustar rootroot$ echo 'error: no contents' #→ --lines clitest-0.0+git20150312.339b2d/dev/test/ok-1.sh0000644000000000000000000000001512500322604016646 0ustar rootroot$ echo ok ok clitest-0.0+git20150312.339b2d/dev/test/ok-10.sh0000644000000000000000000000024212500322604016730 0ustar rootroot$ echo 1 #→ 1 $ echo 2 #→ 2 $ echo 3 #→ 3 $ echo 4 #→ 4 $ echo 5 #→ 5 $ echo 6 #→ 6 $ echo 7 #→ 7 $ echo 8 #→ 8 $ echo 9 #→ 9 $ echo 10 #→ 10 clitest-0.0+git20150312.339b2d/dev/test/inline-match-text.sh0000644000000000000000000000555612500322604021450 0ustar rootroot# Inline matching method: --text # Matches a literal text # This is the default method, the --text part can be omitted. $ echo 'abc' #→ --text abc $ echo 'abc' #→ abc # Special characters as \t and \n are not expanded. $ printf '%s\n' '\t' #→ \t $ printf '%s\n' '\n' #→ \n # Variables and commands are not parsed (see #→ --eval for that). $ echo '$PWD' #→ $PWD $ echo '$(date)' #→ $(date) # It's a literal text, with no special characters. $ echo '$' #→ $ $ echo '>' #→ > $ echo '?' #→ ? $ echo '!' #→ ! $ echo '*' #→ * $ echo '[' #→ [ $ echo '(' #→ ( # For commands that return an empty line, just leave it empty $ echo #→ # But don't forget the blank space after the →, because in this # case the #→ marker will be considered a plain comment and ignored $ echo "not inline output" #→ not inline output $ # Blanks are preserved $ echo '123456789' #→ 123456789 $ echo '1 3 7 9' #→ 1 3 7 9 $ echo ' 5 ' #→ 5 $ echo ' leading space' #→ leading space $ echo ' leading spaces' #→ leading spaces $ printf '\tleading tab\n' #→ leading tab $ printf '\t\tleading tabs\n' #→ leading tabs $ echo 'trailing space ' #→ trailing space $ echo 'trailing spaces ' #→ trailing spaces $ printf 'trailing tab\t\n' #→ trailing tab $ printf 'trailing tabs\t\t\n' #→ trailing tabs $ echo ' ' #→ $ echo ' ' #→ $ printf '\t\n' #→ $ printf '\t\t\t\n' #→ $ printf ' \t \t\t \n' #→ # As seen in all these examples, the final \n is implied. # You can't match lines with no \n. $ printf 'ok\n' #→ ok $ printf 'fail' #→ fail # An easy workaround is to add an empty 'echo' at the end: $ printf 'ok'; echo #→ ok # Syntax: Must be exactly one space before and after --text $ echo 'fail' #→ --text fail with 2 spaces $ echo 'fail' #→ --text fail with tab # Syntax: The extra space after '--text ' is already part of the output $ echo ' ok' #→ --text ok # Syntax: The space after --text is required. # When missing, the '--text' is considered a normal text. $ echo '--text' #→ --text # Syntax: Make sure we won't catch partial matches. $ echo '--textual' #→ --textual # Syntax: To insert a literal text that begins with '--text ' # just prefix it with another --text. $ echo '--text is cool' #→ --text --text is cool clitest-0.0+git20150312.339b2d/dev/test/inline-match-egrep-error-2.sh0000644000000000000000000000051412500322604023041 0ustar rootroot$ echo "error: malformed regex" #→ --egrep ( # Some grep errors: # $ echo | egrep '(' ; echo $? # egrep: parentheses not balanced # 2 # $ echo | egrep '[' ; echo $? # egrep: brackets ([ ]) not balanced # 2 # $ echo | egrep '**' ; echo $? # egrep: repetition-operator operand invalid # 2 # $ echo | egrep '{' ; echo $? # 1 # $ clitest-0.0+git20150312.339b2d/dev/test/option-prefix-tab.sh0000644000000000000000000000164112500322604021454 0ustar rootroot# Test file for the --prefix option # Command blocks here are prefixed by a tab # Run with --prefix '\t' or --prefix tab $ echo "1" #→ 1 $ echo "2" 2 # Any non-prefixed line closes the previous command block. # The empty $ line is not needed. # All other non-indented text is just ignored: $ echo "ignored" # not indented $ echo "ignored" #→ not indented # Lines with the wrong indentation are also ignored $ echo "ignored" # 2 tabs $ echo "ignored" # 8 spaces $ echo "ignored" # 4 spaces # Multiple blocks supported in a single file $ echo "3" 3 # What about prefixed blocks with no commands? Prefixed line with no prompt: ignored. But wait, here comes a command: $ echo "4" 4 $ Last command closed by the empty prompt. $ echo "5" #→ 5 Last command is auto-closed (inline output). # Blank lines in the output are supported $ echo; echo "6"; echo; echo "7" 6 7 # Nice. clitest-0.0+git20150312.339b2d/dev/test/fail-50.sh0000644000000000000000000000173712500322604017250 0ustar rootroot$ echo 1 #→ fail $ echo 2 #→ fail $ echo 3 #→ fail $ echo 4 #→ fail $ echo 5 #→ fail $ echo 6 #→ fail $ echo 7 #→ fail $ echo 8 #→ fail $ echo 9 #→ fail $ echo 10 #→ fail $ echo 11 #→ fail $ echo 12 #→ fail $ echo 13 #→ fail $ echo 14 #→ fail $ echo 15 #→ fail $ echo 16 #→ fail $ echo 17 #→ fail $ echo 18 #→ fail $ echo 19 #→ fail $ echo 20 #→ fail $ echo 21 #→ fail $ echo 22 #→ fail $ echo 23 #→ fail $ echo 24 #→ fail $ echo 25 #→ fail $ echo 26 #→ fail $ echo 27 #→ fail $ echo 28 #→ fail $ echo 29 #→ fail $ echo 30 #→ fail $ echo 31 #→ fail $ echo 32 #→ fail $ echo 33 #→ fail $ echo 34 #→ fail $ echo 35 #→ fail $ echo 36 #→ fail $ echo 37 #→ fail $ echo 38 #→ fail $ echo 39 #→ fail $ echo 40 #→ fail $ echo 41 #→ fail $ echo 42 #→ fail $ echo 43 #→ fail $ echo 44 #→ fail $ echo 45 #→ fail $ echo 46 #→ fail $ echo 47 #→ fail $ echo 48 #→ fail $ echo 49 #→ fail $ echo 50 #→ fail clitest-0.0+git20150312.339b2d/dev/test/ok-50.sh0000644000000000000000000000156212500322604016742 0ustar rootroot$ echo 1 #→ 1 $ echo 2 #→ 2 $ echo 3 #→ 3 $ echo 4 #→ 4 $ echo 5 #→ 5 $ echo 6 #→ 6 $ echo 7 #→ 7 $ echo 8 #→ 8 $ echo 9 #→ 9 $ echo 10 #→ 10 $ echo 11 #→ 11 $ echo 12 #→ 12 $ echo 13 #→ 13 $ echo 14 #→ 14 $ echo 15 #→ 15 $ echo 16 #→ 16 $ echo 17 #→ 17 $ echo 18 #→ 18 $ echo 19 #→ 19 $ echo 20 #→ 20 $ echo 21 #→ 21 $ echo 22 #→ 22 $ echo 23 #→ 23 $ echo 24 #→ 24 $ echo 25 #→ 25 $ echo 26 #→ 26 $ echo 27 #→ 27 $ echo 28 #→ 28 $ echo 29 #→ 29 $ echo 30 #→ 30 $ echo 31 #→ 31 $ echo 32 #→ 32 $ echo 33 #→ 33 $ echo 34 #→ 34 $ echo 35 #→ 35 $ echo 36 #→ 36 $ echo 37 #→ 37 $ echo 38 #→ 38 $ echo 39 #→ 39 $ echo 40 #→ 40 $ echo 41 #→ 41 $ echo 42 #→ 42 $ echo 43 #→ 43 $ echo 44 #→ 44 $ echo 45 #→ 45 $ echo 46 #→ 46 $ echo 47 #→ 47 $ echo 48 #→ 48 $ echo 49 #→ 49 $ echo 50 #→ 50 clitest-0.0+git20150312.339b2d/dev/test/inline-match-perl-error-1.sh0000644000000000000000000000005212500322604022675 0ustar rootroot$ echo 'error: no contents' #→ --perl clitest-0.0+git20150312.339b2d/dev/test/inline-match-lines-error-4.sh0000644000000000000000000000005712500322604023055 0ustar rootroot$ echo 'error: not a number' #→ --lines foo clitest-0.0+git20150312.339b2d/dev/test/empty-prompt-file.sh0000644000000000000000000000000312500322604021466 0ustar rootroot$ clitest-0.0+git20150312.339b2d/LICENSE.txt0000644000000000000000000000207112500322604015635 0ustar rootrootThe MIT License (MIT) Copyright (c) 2013 Aurelio Jargas Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. clitest-0.0+git20150312.339b2d/clitest0000755000000000000000000006103712500322604015416 0ustar rootroot#!/bin/sh # clitest - Tester for Unix command lines # # Author: Aurelio Jargas (http://aurelio.net) # Created: 2013-07-24 # License: MIT # GitHub: https://github.com/aureliojargas/clitest # # POSIX shell script: # This script was coded to be compatible with POSIX shells. # Tested in Bash 3.2, dash 0.5.5.1, ksh 93u 2011-02-08. # Note: Can't set -o posix nor POSIXLY_CORRECT: test env must be intact. # # Exit codes: # 0 All tests passed, or normal operation (--help, --list, ...) # 1 One or more tests have failed # 2 An error occurred (file not found, invalid range, ...) # # Test environment: # By default, the tests will run in the current working directory ($PWD). # You can change to another dir normally using 'cd' inside the test file. # All the tests are executed in the same shell, using eval. Test data # such as variables and working directory will persist between tests. # # Namespace: # All variables and functions in this script are prefixed by 'tt_' to # avoid clashing with test's variables, functions, aliases and commands. tt_my_name="$(basename "$0")" tt_my_version='HEAD' tt_my_version_url="https://github.com/aureliojargas/clitest/tree/$tt_my_version" # Customization (if needed, edit here or use the command line options) tt_prefix='' tt_prompt='$ ' tt_inline_prefix='#→ ' # Problem with Unicode? Use '#=> ' or '### ' tt_diff_options='-u' tt_color_mode='auto' # auto, always, never tt_progress='test' # test, number, dot, none # End of customization # --help message, keep it simple, short and informative tt_my_help="\ Usage: $tt_my_name [options] Options: -1, --first Stop execution upon first failed test -l, --list List all the tests (no execution) -L, --list-run List all the tests with OK/FAIL status -t, --test RANGE Run specific tests, by number (1,2,4-7) -s, --skip RANGE Skip specific tests, by number (1,2,4-7) --pre-flight COMMAND Execute command before running the first test --post-flight COMMAND Execute command after running the last test -q, --quiet Quiet operation, no output shown -V, --version Show program version and exit Customization options: -P, --progress TYPE Set progress indicator: test, number, dot, none --color WHEN Set when to use colors: auto, always, never --diff-options OPTIONS Set diff command options (default: '$tt_diff_options') --inline-prefix PREFIX Set inline output prefix (default: '$tt_inline_prefix') --prefix PREFIX Set command line prefix (default: '$tt_prefix') --prompt STRING Set prompt string (default: '$tt_prompt')" # Temporary files (using files because <(...) is not portable) tt_temp_dir="${TMPDIR:-/tmp}/clitest.$$" tt_temp_file="$tt_temp_dir/temp.txt" tt_test_ok_file="$tt_temp_dir/ok.txt" tt_test_output_file="$tt_temp_dir/output.txt" # Flags (0=off, 1=on), most can be altered by command line options tt_debug=0 tt_use_colors=0 tt_stop_on_first_fail=0 tt_separator_line_shown=0 # The output mode values are mutually exclusive tt_output_mode='normal' # normal, quiet, list, list-run # Globals (all variables are globals, for better portability) tt_nr_files=0 tt_nr_total_tests=0 tt_nr_total_fails=0 tt_nr_total_skips=0 tt_nr_file_tests=0 tt_nr_file_fails=0 tt_nr_file_skips=0 tt_nr_file_ok=0 tt_files_stats= tt_original_dir=$(pwd) tt_pre_command= tt_post_command= tt_run_range= tt_run_range_data= tt_skip_range= tt_skip_range_data= tt_failed_range= tt_test_file= tt_input_line= tt_line_number=0 tt_test_number=0 tt_test_line_number=0 tt_test_command= tt_test_inline= tt_test_mode= tt_test_status=2 tt_test_output= tt_test_diff= tt_test_ok_text= tt_missing_nl=0 # Special handy chars tt_tab=' ' tt_nl=' ' # Handle command line options while test "${1#-}" != "$1" do case "$1" in -1|--first ) shift; tt_stop_on_first_fail=1 ;; -l|--list ) shift; tt_output_mode='list' ;; -L|--list-run ) shift; tt_output_mode='list-run' ;; -q|--quiet ) shift; tt_output_mode='quiet' ;; -t|--test ) shift; tt_run_range="$1"; shift ;; -s|--skip ) shift; tt_skip_range="$1"; shift ;; --pre-flight ) shift; tt_pre_command="$1"; shift ;; --post-flight ) shift; tt_post_command="$1"; shift ;; --debug ) shift; tt_debug=1 ;; -P|--progress ) shift; tt_progress="$1"; tt_output_mode='normal'; shift ;; --color|--colour) shift; tt_color_mode="$1"; shift ;; --diff-options ) shift; tt_diff_options="$1"; shift ;; --inline-prefix ) shift; tt_inline_prefix="$1"; shift ;; --prefix ) shift; tt_prefix="$1"; shift ;; --prompt ) shift; tt_prompt="$1"; shift ;; -h|--help) printf '%s\n' "$tt_my_help" exit 0 ;; -V|--version) printf '%s %s\n%s\n' $tt_my_name $tt_my_version $tt_my_version_url exit 0 ;; --) shift; break ;; *) break ;; esac done # Command line options consumed, now it's just the files tt_nr_files=$# ### Utilities tt_clean_up () { rm -rf "$tt_temp_dir" } tt_message () { test "$tt_output_mode" = 'quiet' && return 0 test $tt_missing_nl -eq 1 && echo printf '%s\n' "$*" tt_separator_line_shown=0 tt_missing_nl=0 } tt_message_part () # no line break { test "$tt_output_mode" = 'quiet' && return 0 printf '%s' "$*" tt_separator_line_shown=0 tt_missing_nl=1 } tt_error () { test $tt_missing_nl -eq 1 && echo printf '%s\n' "$tt_my_name: Error: $1" >&2 tt_clean_up exit 2 } tt_debug () # $1=id, $2=contents { test $tt_debug -ne 1 && return 0 if test INPUT_LINE = "$1" then # Original input line is all blue printf "${tt_color_blue}[%10s: %s]${tt_color_off}\n" "$1" "$2" else # Highlight tabs and inline prefix printf "${tt_color_blue}[%10s:${tt_color_off} %s${tt_color_blue}]${tt_color_off}\n" "$1" "$2" | sed "/LINE_CMD:/ s/$tt_inline_prefix/${tt_color_red}&${tt_color_off}/g" | sed "s/$tt_tab/${tt_color_green}${tt_color_off}/g" fi } tt_separator_line () { printf "%${COLUMNS}s" ' ' | tr ' ' - } tt_list_test () # $1=normal|list|ok|fail { # Show the test command in normal mode, --list and --list-run case "$1" in normal | list) # Normal line, no color, no stamp (--list) tt_message "#${tt_test_number}${tt_tab}${tt_test_command}" ;; ok) # Green line or OK stamp (--list-run) if test $tt_use_colors -eq 1 then tt_message "${tt_color_green}#${tt_test_number}${tt_tab}${tt_test_command}${tt_color_off}" else tt_message "#${tt_test_number}${tt_tab}OK${tt_tab}${tt_test_command}" fi ;; fail) # Red line or FAIL stamp (--list-run) if test $tt_use_colors -eq 1 then tt_message "${tt_color_red}#${tt_test_number}${tt_tab}${tt_test_command}${tt_color_off}" else tt_message "#${tt_test_number}${tt_tab}FAIL${tt_tab}${tt_test_command}" fi ;; esac } tt_parse_range () # $1=range { # Parse numeric ranges and output them in an expanded format # # Supported formats Expanded # ------------------------------------------------------ # Single: 1 :1: # List: 1,3,4,7 :1:3:4:7: # Range: 1-4 :1:2:3:4: # Mixed: 1,3,4-7,11,13-15 :1:3:4:5:6:7:11:13:14:15: # # Reverse ranges and repeated/unordered numbers are ok. # Later we will just grep for :number: in each test. case "$1" in # No range, nothing to do 0 | '') return 0 ;; # Error: strange chars, not 0123456789,- *[!0-9,-]*) return 1 ;; esac # OK, all valid chars in range, let's parse them tt_part= tt_n1= tt_n2= tt_operation= tt_range_data=':' # :1:2:4:7: # Loop each component: a number or a range for tt_part in $(echo "$1" | tr , ' ') do # If there's an hyphen, it's a range case "$tt_part" in *-*) # Error: Invalid range format, must be: number-number echo "$tt_part" | grep '^[0-9][0-9]*-[0-9][0-9]*$' > /dev/null || return 1 tt_n1=${tt_part%-*} tt_n2=${tt_part#*-} tt_operation='+' test $tt_n1 -gt $tt_n2 && tt_operation='-' # Expand the range (1-4 => 1:2:3:4) tt_part=$tt_n1: while test $tt_n1 -ne $tt_n2 do tt_n1=$(($tt_n1 $tt_operation 1)) tt_part=$tt_part$tt_n1: done tt_part=${tt_part%:} ;; esac # Append the number or expanded range to the holder test $tt_part != 0 && tt_range_data=$tt_range_data$tt_part: done test $tt_range_data != ':' && echo $tt_range_data return 0 } tt_reset_test_data () { tt_test_command= tt_test_inline= tt_test_mode= tt_test_status=2 tt_test_output= tt_test_diff= tt_test_ok_text= } tt_run_test () { tt_test_number=$(($tt_test_number + 1)) tt_nr_total_tests=$(($tt_nr_total_tests + 1)) tt_nr_file_tests=$(($tt_nr_file_tests + 1)) # Run range on: skip this test if it's not listed in $tt_run_range_data if test -n "$tt_run_range_data" && test "$tt_run_range_data" = "${tt_run_range_data#*:$tt_test_number:}" then tt_nr_total_skips=$(($tt_nr_total_skips + 1)) tt_nr_file_skips=$(($tt_nr_file_skips + 1)) tt_reset_test_data return 0 fi # Skip range on: skip this test if it's listed in $tt_skip_range_data # Note: --skip always wins over --test, regardless of order if test -n "$tt_skip_range_data" && test "$tt_skip_range_data" != "${tt_skip_range_data#*:$tt_test_number:}" then tt_nr_total_skips=$(($tt_nr_total_skips + 1)) tt_nr_file_skips=$(($tt_nr_file_skips + 1)) tt_reset_test_data return 0 fi case "$tt_output_mode" in normal) # Normal mode: show progress indicator case "$tt_progress" in test) tt_list_test normal ;; number) tt_message_part "$tt_test_number " ;; none) : ;; *) tt_message_part "$tt_progress" ;; esac ;; list) # List mode: just show the command and return (no execution) tt_list_test list tt_reset_test_data return 0 ;; esac #tt_debug EVAL "$tt_test_command" # Execute the test command, saving output (STDOUT and STDERR) eval "$tt_test_command" > "$tt_test_output_file" 2>&1 #tt_debug OUTPUT "$(cat "$tt_test_output_file")" # The command output matches the expected output? case $tt_test_mode in output) printf %s "$tt_test_ok_text" > "$tt_test_ok_file" tt_test_diff=$(diff $tt_diff_options "$tt_test_ok_file" "$tt_test_output_file") tt_test_status=$? ;; text) # Inline OK text represents a full line, with \n printf '%s\n' "$tt_test_inline" > "$tt_test_ok_file" tt_test_diff=$(diff $tt_diff_options "$tt_test_ok_file" "$tt_test_output_file") tt_test_status=$? ;; eval) eval "$tt_test_inline" > "$tt_test_ok_file" tt_test_diff=$(diff $tt_diff_options "$tt_test_ok_file" "$tt_test_output_file") tt_test_status=$? ;; lines) tt_test_output=$(sed -n '$=' "$tt_test_output_file") test -z "$tt_test_output" && tt_test_output=0 test "$tt_test_output" -eq "$tt_test_inline" tt_test_status=$? tt_test_diff="Expected $tt_test_inline lines, got $tt_test_output." ;; file) # If path is relative, make it relative to the test file path, not $PWD if test $tt_test_inline = ${tt_test_inline#/} then tt_test_inline="$(dirname "$tt_test_file")/$tt_test_inline" fi # Abort when ok file not found/readable if test ! -f "$tt_test_inline" || test ! -r "$tt_test_inline" then tt_error "cannot read inline output file '$tt_test_inline', from line $tt_line_number of $tt_test_file" fi tt_test_diff=$(diff $tt_diff_options "$tt_test_inline" "$tt_test_output_file") tt_test_status=$? ;; egrep) egrep "$tt_test_inline" "$tt_test_output_file" > /dev/null tt_test_status=$? # Test failed: the regex not matched if test $tt_test_status -eq 1 then tt_test_diff="egrep '$tt_test_inline' failed in:$tt_nl$(cat "$tt_test_output_file")" # Regex errors are common and user must take action to fix them elif test $tt_test_status -eq 2 then tt_error "check your inline egrep regex at line $tt_line_number of $tt_test_file" fi ;; perl | regex) # Escape regex delimiter (if any) inside the regex: ' => \' if test "$tt_test_inline" != "${tt_test_inline#*\'}" then tt_test_inline=$(printf %s "$tt_test_inline" | sed "s/'/\\\\'/g") fi # Note: -0777 to get the full file contents as a single string perl -0777 -ne "exit(!m'$tt_test_inline')" "$tt_test_output_file" tt_test_status=$? case $tt_test_status in 0) # Test matched, nothing to do : ;; 1) # Test failed: the regex not matched tt_test_diff="Perl regex '$tt_test_inline' not matched in:$tt_nl$(cat "$tt_test_output_file")" ;; 127) # Perl not found :( tt_error "Perl not found. It's needed by --$tt_test_mode at line $tt_line_number of $tt_test_file" ;; 255) # Regex syntax errors are common and user must take action to fix them tt_error "check your inline Perl regex at line $tt_line_number of $tt_test_file" ;; *) tt_error "unknown error when running Perl for --$tt_test_mode at line $tt_line_number of $tt_test_file" ;; esac ;; *) tt_error "unknown test mode '$tt_test_mode'" ;; esac # Test failed :( if test $tt_test_status -ne 0 then tt_nr_file_fails=$(($tt_nr_file_fails + 1)) tt_nr_total_fails=$(($tt_nr_total_fails + 1)) tt_failed_range="$tt_failed_range$tt_test_number," # Decide the message format if test "$tt_output_mode" = 'list-run' then # List mode tt_list_test fail else # Normal mode: show FAILED message and the diff if test $tt_separator_line_shown -eq 0 # avoid dups then tt_message "${tt_color_red}$(tt_separator_line)${tt_color_off}" fi tt_message "${tt_color_red}[FAILED #$tt_test_number, line $tt_test_line_number] $tt_test_command${tt_color_off}" tt_message "$tt_test_diff" | sed '1 { /^--- / { N; /\n+++ /d; }; }' # no ---/+++ headers tt_message "${tt_color_red}$(tt_separator_line)${tt_color_off}" tt_separator_line_shown=1 fi # Should I abort now? if test $tt_stop_on_first_fail -eq 1 then tt_clean_up exit 1 fi # Test OK else test "$tt_output_mode" = 'list-run' && tt_list_test ok fi tt_reset_test_data } tt_process_test_file () { # Reset counters tt_nr_file_tests=0 tt_nr_file_fails=0 tt_nr_file_skips=0 tt_line_number=0 tt_test_line_number=0 # Loop for each line of input file # Note: changing IFS to avoid right-trimming of spaces/tabs # Note: read -r to preserve the backslashes while IFS='' read -r tt_input_line || test -n "$tt_input_line" do tt_line_number=$(($tt_line_number + 1)) #tt_debug INPUT_LINE "$tt_input_line" case "$tt_input_line" in # Prompt alone: closes previous command line (if any) "$tt_prefix$tt_prompt" | "$tt_prefix${tt_prompt% }" | "$tt_prefix$tt_prompt ") #tt_debug 'LINE_$' "$tt_input_line" # Run pending tests test -n "$tt_test_command" && tt_run_test ;; # This line is a command line to be tested "$tt_prefix$tt_prompt"*) #tt_debug LINE_CMD "$tt_input_line" # Run pending tests test -n "$tt_test_command" && tt_run_test # Remove the prompt tt_test_command="${tt_input_line#"$tt_prefix$tt_prompt"}" # Save the test's line number for future messages tt_test_line_number=$tt_line_number # This is a special test with inline output? if printf '%s\n' "$tt_test_command" | grep "$tt_inline_prefix" > /dev/null then # Separate command from inline output tt_test_command="${tt_test_command%"$tt_inline_prefix"*}" tt_test_inline="${tt_input_line##*"$tt_inline_prefix"}" #tt_debug NEW_CMD "$tt_test_command" #tt_debug OK_INLINE "$tt_test_inline" # Maybe the OK text has options? case "$tt_test_inline" in '--egrep '*) tt_test_inline=${tt_test_inline#--egrep } tt_test_mode='egrep' ;; '--regex '*) # alias to --perl tt_test_inline=${tt_test_inline#--regex } tt_test_mode='regex' ;; '--perl '*) tt_test_inline=${tt_test_inline#--perl } tt_test_mode='perl' ;; '--file '*) tt_test_inline=${tt_test_inline#--file } tt_test_mode='file' ;; '--lines '*) tt_test_inline=${tt_test_inline#--lines } tt_test_mode='lines' ;; '--eval '*) tt_test_inline=${tt_test_inline#--eval } tt_test_mode='eval' ;; '--text '*) tt_test_inline=${tt_test_inline#--text } tt_test_mode='text' ;; *) tt_test_mode='text' ;; esac #tt_debug OK_TEXT "$tt_test_inline" # There must be a number in --lines if test "$tt_test_mode" = 'lines' then case "$tt_test_inline" in '' | *[!0-9]*) tt_error "--lines requires a number. See line $tt_line_number of $tt_test_file" ;; esac fi # An empty inline parameter is an error user must see if test -z "$tt_test_inline" && test "$tt_test_mode" != 'text' then tt_error "empty --$tt_test_mode at line $tt_line_number of $tt_test_file" fi # Since we already have the command and the output, run test tt_run_test else # It's a normal command line, output begins in next line tt_test_mode='output' #tt_debug NEW_CMD "$tt_test_command" fi ;; # Test output, blank line or comment *) #tt_debug 'LINE_*' "$tt_input_line" # Ignore this line if there's no pending test test -n "$tt_test_command" || continue # Required prefix is missing: we just left a command block if test -n "$tt_prefix" && test "${tt_input_line#"$tt_prefix"}" = "$tt_input_line" then #tt_debug BLOCK_OUT "$tt_input_line" # Run the pending test and we're done in this line tt_run_test continue fi # This line is a test output, save it (without prefix) tt_test_ok_text="$tt_test_ok_text${tt_input_line#"$tt_prefix"}$tt_nl" #tt_debug OK_TEXT "${tt_input_line#"$tt_prefix"}" ;; esac done < "$tt_temp_file" #tt_debug LOOP_OUT "\$tt_test_command=$tt_test_command" # Run pending tests test -n "$tt_test_command" && tt_run_test } ### Init process # No files? if test $tt_nr_files -eq 0 then tt_error 'no test file informed (try --help)' fi # Handy shortcuts for prefixes case "$tt_prefix" in tab) tt_prefix="$tt_tab" ;; 0) tt_prefix='' ;; [1-9] | [1-9][0-9]) # 1-99 # convert number to spaces: 2 => ' ' tt_prefix=$(printf "%${tt_prefix}s" ' ') ;; *\\*) tt_prefix="$(printf %b "$tt_prefix")" # expand \t and others ;; esac # Validate and normalize progress value if test "$tt_output_mode" = 'normal' then case "$tt_progress" in test) : ;; number | n | [0-9]) tt_progress='number' ;; dot | .) tt_progress='.' ;; none | no) tt_progress='none' ;; ?) # Single char, use it as the progress : ;; *) tt_error "invalid value '$tt_progress' for --progress. Use: test, number, dot or none." ;; esac fi # Will we use colors in the output? case "$tt_color_mode" in always | yes | y) tt_use_colors=1 ;; never | no | n) tt_use_colors=0 ;; auto | a) # The auto mode will use colors if the output is a terminal # Note: test -t is in POSIX if test -t 1 then tt_use_colors=1 else tt_use_colors=0 fi ;; *) tt_error "invalid value '$tt_color_mode' for --color. Use: auto, always or never." ;; esac # Set colors # Remember: colors must be readable in dark and light backgrounds # Customization: tweak the numbers after [ to adjust the colors if test $tt_use_colors -eq 1 then tt_color_red=$( printf '\033[31m') # fail tt_color_green=$(printf '\033[32m') # ok tt_color_blue=$( printf '\033[34m') # debug #tt_color_cyan=$( printf '\033[36m') # not used tt_color_off=$( printf '\033[m') fi # Find the terminal width # The COLUMNS env var is set by Bash (must be exported in ~/.bashrc). # In other shells, try to use 'tput cols' (not POSIX). # If not, defaults to 50 columns, a conservative amount. : ${COLUMNS:=$(tput cols 2> /dev/null)} : ${COLUMNS:=50} # Parse and validate --test option value, if informed tt_run_range_data=$(tt_parse_range "$tt_run_range") if test $? -ne 0 then tt_error "invalid argument for -t or --test: $tt_run_range" fi # Parse and validate --skip option value, if informed tt_skip_range_data=$(tt_parse_range "$tt_skip_range") if test $? -ne 0 then tt_error "invalid argument for -s or --skip: $tt_skip_range" fi # Create temp dir, protected from others umask 077 && mkdir "$tt_temp_dir" || tt_error "cannot create temporary dir: $tt_temp_dir" ### Real execution begins here # Some preparing command to run before all the tests? if test -n "$tt_pre_command" then eval "$tt_pre_command" || tt_error "pre-flight command failed with status=$?: $tt_pre_command" fi # For each input file in $@ for tt_test_file do # Some tests may 'cd' to another dir, we need to get back # to preserve the relative paths of the input files cd "$tt_original_dir" # Abort when test file not found/readable if test ! -f "$tt_test_file" || test ! -r "$tt_test_file" then tt_error "cannot read input file: $tt_test_file" fi # In multifile mode, identify the current file if test $tt_nr_files -gt 1 then case "$tt_output_mode" in normal) # Normal mode, show message with filename case "$tt_progress" in test | none) tt_message "Testing file $tt_test_file" ;; *) test $tt_missing_nl -eq 1 && echo tt_message_part "Testing file $tt_test_file " ;; esac ;; list | list-run) # List mode, show ------ and the filename tt_message $(tt_separator_line | cut -c 1-40) $tt_test_file ;; esac fi # Convert Windows files (CRLF) to the Unix format (LF) # Note: the temporary file is required, because doing "sed | while" opens # a subshell and global vars won't be updated outside the loop. sed "s/$(printf '\r')$//" "$tt_test_file" > "$tt_temp_file" # The magic happens here tt_process_test_file # Abort when no test found (and no active range with --test or --skip) if test $tt_nr_file_tests -eq 0 && test -z "$tt_run_range_data" && test -z "$tt_skip_range_data" then tt_error "no test found in input file: $tt_test_file" fi # Save file stats tt_nr_file_ok=$(($tt_nr_file_tests - $tt_nr_file_fails - $tt_nr_file_skips)) tt_files_stats="$tt_files_stats$tt_nr_file_ok $tt_nr_file_fails $tt_nr_file_skips$tt_nl" # Dots mode: any missing new line? # Note: had to force tt_missing_nl=0, even when it's done in tt_message :/ test $tt_missing_nl -eq 1 && tt_missing_nl=0 && tt_message done tt_clean_up # Some clean up command to run after all the tests? if test -n "$tt_post_command" then eval "$tt_post_command" fi #----------------------------------------------------------------------- # From this point on, it's safe to use non-prefixed global vars #----------------------------------------------------------------------- # Range active, but no test matched :( if test $tt_nr_total_tests -eq $tt_nr_total_skips then if test -n "$tt_run_range_data" && test -n "$tt_skip_range_data" then tt_error "no test found. The combination of -t and -s resulted in no tests." elif test -n "$tt_run_range_data" then tt_error "no test found for the specified number or range '$tt_run_range'" elif test -n "$tt_skip_range_data" then tt_error "no test found. Maybe '--skip $tt_skip_range' was too much?" fi fi # List mode has no stats if test "$tt_output_mode" = 'list' || test "$tt_output_mode" = 'list-run' then if test $tt_nr_total_fails -eq 0 then exit 0 else exit 1 fi fi # Show stats # Data: # $tt_files_stats -> "100 0 23 \n 12 34 0" # $@ -> foo.sh bar.sh # Output: # ok fail skip # 100 0 23 foo.sh # 12 34 0 bar.sh if test $tt_nr_files -gt 1 && test "$tt_output_mode" != 'quiet' then echo printf ' %5s %5s %5s\n' ok fail skip printf %s "$tt_files_stats" | while read ok fail skip do printf ' %5s %5s %5s %s\n' $ok $fail $skip "$1" shift done | sed 's/ 0/ -/g' # hide zeros echo fi # The final message: OK or FAIL? # OK: 123 of 123 tests passed # OK: 100 of 123 tests passed (23 skipped) # FAIL: 123 of 123 tests failed # FAIL: 100 of 123 tests failed (23 skipped) skips= if test $tt_nr_total_skips -gt 0 then skips=" ($tt_nr_total_skips skipped)" fi if test $tt_nr_total_fails -eq 0 then stamp="${tt_color_green}OK:${tt_color_off}" stats="$(($tt_nr_total_tests - $tt_nr_total_skips)) of $tt_nr_total_tests tests passed" test $tt_nr_total_tests -eq 1 && stats=$(echo "$stats" | sed 's/tests /test /') tt_message "$stamp $stats$skips" exit 0 else test $tt_nr_files -eq 1 && tt_message # separate from previous FAILED message stamp="${tt_color_red}FAIL:${tt_color_off}" stats="$tt_nr_total_fails of $tt_nr_total_tests tests failed" test $tt_nr_total_tests -eq 1 && stats=$(echo "$stats" | sed 's/tests /test /') tt_message "$stamp $stats$skips" test $tt_test_file = 'dev/test.md' && tt_message "-t ${tt_failed_range%,}" # XXX dev helper, remove before release exit 1 fi