Testing guides
Testing a command-line tool: arguments, pipes, encodings and exit codes
A CLI is used by scripts more than by people, and scripts are unforgiving. The shell rewrites arguments before your tool sees them, pipes close early, exit codes decide whether a CI job passes, and input text arrives in whatever encoding produced it. Here is what to test, with files for each case.
1. Arguments the shell has already touched
| Input | What happens | Correct behaviour |
|---|---|---|
"spaces in name.txt" | Unquoted, it arrives as three arguments. | Works when quoted; the error for the unquoted form names the missing file. |
A file called -rf | Parsed as an option. | Accepted after --, or as ./-rf. |
*.txt in cmd.exe | Windows' command prompt does not expand wildcards; the tool receives the pattern itself. | The tool expands it, or reports no match, rather than looking for a file called *.txt. |
| A glob that matches nothing | bash passes the pattern through unchanged by default. | A clear "no files match" error. |
| Unicode names | Fine in the shell, broken by code that assumes ASCII or the wrong code page. | Reads and prints them intact. |
For names, use the Filename Compatibility pack: spaces, quotes,
#, %, ;, emoji, Cyrillic, Japanese and a right-to-left override, all in one
archive. Create dash names yourself with touch -- -rf.
2. Pipes and streams
A pipe that closes early
yourtool big.txt | head -1 closes the pipe after one
line. The tool should stop quietly, not print a broken-pipe stack trace. Python, for one, raises BrokenPipeError here.
Reading stdin
cat file | yourtool - and yourtool < file should
behave like a file argument, including for a large file that must be streamed, not read whole.
stdout for data, stderr for noise
Progress and warnings go to stderr, so
yourtool > out.csv produces a clean file. Test by redirecting stdout and reading what landed in it.
No terminal
Piped or run by CI, stdout is not a TTY. Colours and progress bars should
switch off, and NO_COLOR should be honoured. Check the output for escape codes.
3. Exit codes
Scripts and CI read nothing else, so they have to be right every time.
| Code | Meaning by convention | Test |
|---|---|---|
| 0 | Success | Only when everything asked for was done. |
| 1 | General failure | A missing input file, a parse error. |
| 2 | Misuse of the command line | An unknown option or a missing required argument. |
| 130 | Interrupted (128 + SIGINT) | Press Ctrl+C mid-run: no partial output left looking complete. |
Check them with echo $? in bash and zsh, echo %ERRORLEVEL% in cmd.exe, and
$LASTEXITCODE in PowerShell.
4. Text in every encoding
| Input | Where it comes from |
|---|---|
| UTF-16 LE with BOM | Windows PowerShell 5.1 redirection (>). PowerShell 7 writes UTF-8 without a BOM instead. |
| UTF-8 with BOM | Notepad and many Windows tools. The BOM ends up glued to the first field. |
| CRLF line endings | Any Windows editor. A trailing \r on every value breaks comparisons. |
| No trailing newline | The last line is lost by tools that only count complete lines. |
| Latin-1 | Legacy exports. Not valid UTF-8: fail clearly rather than print garbage. |
| A very long line | Line-based readers with a fixed buffer. |
Run it under LC_ALL=C as well, the minimal locale many containers and CI images use, and check
that non-ASCII input still works.
5. Files that are not what the name says
Empty
A 0-byte input should give a defined result, not a crash or a hang waiting for data.
A directory
Pass a folder where a file is expected. The error should say so, not "permission denied".
Damaged
Malformed JSON and CSV should report the line, then exit non-zero.
Need exactly these files in a script? Every fixture has a direct URL and a SHA-256, and generate test files from the command line covers making your own.
Questions
How do I test a file name that starts with a dash?
Create one with touch -- -rf and check your tool accepts it after -- or as ./-rf.
Why does my CLI get *.txt literally on Windows?
cmd.exe does not expand wildcards; the program has to. Test from cmd.exe as well as bash.
What exit codes should a CLI return?
0 for success, non-zero for failure, 2 for command-line misuse; Ctrl+C conventionally gives 130.
Why does my PowerShell output look spaced out?
Windows PowerShell 5.1 redirects as UTF-16 LE. PowerShell 7 writes UTF-8. Test both.
Keep going
Generate files from the command line Character encodings → Desktop app testing →
More from Learning
Guides and references for test data, file handling and AI evals. All free, no sign-up. See the full hub.