Shellcheck says to replace expr ..., as expr is deprecated. For mathematical expressions, I can use $((...)) instead. But what should I use for string expressions?

FizzBuzz example:

mcandre/toys#2 (comment)

This fails, when it should not:
time { true; }

On my machine, cabal install puts the Shellcheck binaries in a weird place:

$ cabal install
...
Installing executable(s) in
/Users/apennebaker/Library/Haskell/ghc-7.6.3/lib/ShellCheck-0.2.0/bin

Not sure if this is an error in Shellcheck's Cabal configuration, or an error in Haskell Platform's Cabal configuration.

In any case, I was able to get around this by manually adding export PATH="$PATH:~/Library/Haskell/ghc-7.6.3/lib/ShellCheck-0.2.0/bin" to my ~/.profile.

System:

$ specs haskell os
Specs:

specs 0.7
https://github.com/mcandre/specs#readme

cabal --version
cabal-install version 1.16.0.2
using version 1.16.0 of the Cabal library 

ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.6.3

ghc-pkg field haskell-platform version
version: 2013.2.0.0

system_profiler SPSoftwareDataType | grep 'System Version'
      System Version: OS X 10.9 (13A603)

This parse error doesn't seem right:

In test.sh line 3:
for (( i=0 ; i<10 ; i++ )) ; do
^-- Couldn't parse this for loop.
^-- Expected 'do'.
^-- Unexpected ";". Fix any mentioned problems and try again.

Apparently, the parser is not expecting the semicolon before 'do', but man bash says it is a correct syntax:

   for (( expr1 ; expr2 ; expr3 )) ; do list ; done
          First, the arithmetic expression expr1 is evaluated according to the rules described below under ARITHMETIC EVALUATION.  The arithmetic expression expr2 is then evaluated repeatedly until it  evaluates  to
          zero.  Each time expr2 evaluates to a non-zero value, list is executed and the arithmetic expression expr3 is evaluated.  If any expression is omitted, it behaves as if it evaluates to 1.  The return value
          is the exit status of the last command in list that is executed, or false if any of the expressions is invalid.

I cloned the tree and did a cabal install and it failed. Here's the relevant error:
[6 of 6] Compiling Main ( shellcheck.hs, dist/build/shellcheck/shellcheck-tmp/Main.o )

shellcheck.hs:159:39:
Not in scope: catch' Perhaps you meantcatch#' (imported from GHC.Exts)

shellcheck.hs:220:5:
Not in scope: catch' Perhaps you meantcatch#' (imported from GHC.Exts)
Failed to install ShellCheck-0.2.0
cabal: Error: some packages failed to install:
ShellCheck-0.2.0 failed during the building phase. The exception was:
ExitFailure 1

Have I screwed up somewhere?

It would be very helpful to turn of warnings in specific cases. Pylint warnings can be silenced with a comment:

pylint: disable-msg=C0103

Something similar would be very useful. Another option would be a config file where you could turn off specific warnings globally.

I plan to run Shellcheck over some zsh, ksh scripts, etc. etc. Would Shellcheck print warnings specific to these shell languages? Would Shellcheck even recognize the syntax?

Shellcheck is awesome, by the way!

1 test=grep "\"" test
^––
SC1009 The mentioned parser error was in this simple command.

               ^––

SC1073 Couldn't parse this double quoted string.

                     ^––

SC1072 Unexpected eof. Fix any mentioned problems and try again

in

cat << EOF > /dev/null
foo
EOF

the here document is parsed as starting with > /dev/null rather than foo

Perl tends to incorrectly trigger the variable-in-single-quotes warning, as in

perl -F: -lane 'print $F[0]' /etc/passwd

It should be added to the accepted list along with awk and trap.

This warning is invalid:

find trunk -type f -name '*.gcda' -print0 |\
^-- Don't use find | xargs cmd. find -exec cmd {} + handles whitespace.

find ... -print0 | xargs -0 ... handles whitespace.
And piping to xargs can be orders of magnitude(!) faster than using -exec {}, which is single-threaded, blocking, and not file system cache friendly:

$ rm -f /tmp/foo; time find /usr/lib64 -type f 2>/dev/null -print0 | xargs -0 md5sum >>/tmp/foo 2>/dev/null

real 0m3.304s
user 0m2.971s
sys 0m0.373s
$ rm -f /tmp/foo; time find /usr/lib64 -type f 2>/dev/null -exec md5sum >>/tmp/foo {} ;

real 0m14.842s
user 0m4.928s
sys 0m5.174s

If anything, the warning should be the other way - never use exec {} if there's an opportunity to use -print0 | xargs -0

Hi,

Running Ubuntu 12.04.2 LTS with the packages given in the README:

ghc6 libghc6-parsec3-dev libghc6-quickcheck2-dev libghc6-json-dev libghc-regex-compat-dev

and also cabal. Current shellcheck HEAD cloned:

~/github/shellcheck$ git log -1 --oneline
de1fa61 Warn about client side expansion in ssh strings/heredocs.

configure seems okay but build fails:

~/github/shellcheck$ cabal configure
Resolving dependencies...
Configuring ShellCheck-0.1.0...
~/github/shellcheck$ cabal build 
Building ShellCheck-0.1.0...
Preprocessing executable 'shellcheck' for ShellCheck-0.1.0...

ShellCheck/Simple.hs:23:8:
    Could not find module `Text.Parsec.Pos'
    It is a member of the hidden package `parsec-3.1.2'.
    Perhaps you need to add `parsec' to the build-depends in your .cabal file.
    Use -v to see a list of the files searched for.
~/github/shellcheck$ 

With -v flag as suggested:

~/github/shellcheck$ cabal build -v 
creating dist/build
creating dist/build/autogen
Building ShellCheck-0.1.0...
Preprocessing executable 'shellcheck' for ShellCheck-0.1.0...
Building executable shellcheck...
creating dist/build/shellcheck
creating dist/build/shellcheck/shellcheck-tmp
/usr/bin/ghc --make -o dist/build/shellcheck/shellcheck -hide-all-packages -fbuilding-cabal-package -package-conf dist/package.conf.inplace -i -idist/build/shellcheck/shellcheck-tmp -i. -idist/build/autogen -Idist/build/autogen -Idist/build/shellcheck/shellcheck-tmp -optP-include -optPdist/build/autogen/cabal_macros.h -odir dist/build/shellcheck/shellcheck-tmp -hidir dist/build/shellcheck/shellcheck-tmp -stubdir dist/build/shellcheck/shellcheck-tmp -package-id base-4.5.0.0-40b99d05fae6a4eea95ea69e6e0c9702 -package-id containers-0.4.2.1-cfc6420ecc2194c9ed977b06bdfd9e69 -package-id directory-1.1.0.2-ebacad9b5233212b1abbebce9b7e6524 -package-id json-0.5-b3efb968dbdfc514365c5250445af3ff -package-id mtl-2.0.1.0-db19dd8a7700e3d3adda8aa8fe5bf53d -package-id parsec-2.1.0.1-defe69eb7a92d23008966c94e32574a7 -package-id regex-compat-0.95.1-851005df9f3cd69b337623025f7c092b -O -XHaskell98 ./shellcheck.hs

ShellCheck/Simple.hs:23:8:
    Could not find module `Text.Parsec.Pos'
    It is a member of the hidden package `parsec-3.1.2'.
    Perhaps you need to add `parsec' to the build-depends in your .cabal file.
    Use -v to see a list of the files searched for.
~/github/shellcheck$ 

I have three parsec-dev packages installed:

un  libghc-parsec-dev                             <none>                                        (no description available)
un  libghc-parsec-dev-2.1.0.1-defe6               <none>                                        (no description available)
un  libghc-parsec-dev-3.1.2-a6715                 <none>                                        (no description available)
ii  libghc-parsec2-dev                            2.1.0.1-6                                     Haskell monadic parser combinator library for GHC
ii  libghc-parsec3-dev                            3.1.2-1                                       Haskell monadic parser combinator library for GHC
ii  libghc6-parsec3-dev                           1:6                                           transitional dummy package

cabal info parsec says:

Versions available: 2.0, 2.1.0.0, 2.1.0.1, (3.0.0), (3.0.1), (3.1.0)
Versions installed: 2.1.0.1, (3.1.2)

(and Text.ParserCombinators.Parsec.Pos is included in the modules listing).

An explicit dependency parsec == 2.1.0.1 does not change the build error
(as I expected). However setting parsec == 3.1.2 reveals:

~/github/shellcheck$ cabal configure
Resolving dependencies...
Configuring ShellCheck-0.1.0...
Warning: This package indirectly depends on multiple versions of the same
package. This is highly likely to cause a compile failure.
package json-0.5 requires parsec-2.1.0.1
package ShellCheck-0.1.0 requires parsec-3.1.2
~/github/shellcheck$ 

No more recent version of json appears to be available in Ubuntu (although there is a slightly older version of 0.5).

Because I'm a WILD AND CRAZY guy I tried cabal build in any case and
that seems to produce a working executable. But obviously something a
little fishy here...

This warning is incorrect:

ls -1N | cat
^-- Don't parse ls output; it mangles filenames.

When used with -N and a pipe, ls does not mangle file names, but passes them on raw. What you have after the pipe might mangle them, of course, but that's a different issue.

cat <<-!SOME_MARKER | xargs echo
test
test2
!SOME_MARKER

1 cat <<-!SOME_MARKER | xargs echo
^––
SC1009 The mentioned parser error was in this simple command.

      ^––

SC1073 Couldn't parse this here document.

         ^––

SC1072 Unexpected keyword/token. Fix any mentioned problems and try again.

2 test
3 test2
4 !SOME_MARKER
5

Like with variable modification, break/continue are ineffectual in subshells:

while true; do { foo || break; } | bar; done

#!/bin/bash
var=ab
echo "${"$(echo "${var//a/a1}")"//b/b1}"
exit 0

this erronous script reports bad substitution when run in bash shell but reports OK with shellcheck.

yum install cabal-install ghc ghc-parsec-devel ghc-QuickCheck-devel ghc-json-devel ghc-regex-compat-devel

Thanks to Norbert Varzariu

If we add shellcheck to Hackage, users can simply cabal install shellcheck.

Hello

It looks like this expression is not parsed correctly.

[ -"${option_type:0:1}" "$value" ]
^-- Couldn't parse this test expression.
^-- Unexpected """. Fix any mentioned problems and try again.

For the record, the line context is available at https://github.com/Anvil/bash-argsparse/blob/master/argsparse.sh#L614

Thank you.

There are some legitimate use cases for variables in printf strings, like

printf "%${pad_len}s\n" "pad me"

We should reduce false positives, e.g. by looking for %s or valid format codes in the printf string, indicating that the user is familiar with them

I'm not entirely sure if this is a bug, so please forgive me if it's a problem with my script instead

shellcheck suggests that i replace "dirname $0" with "${var%/*}"
however, when i run the script with "bash scriptname" or "sh scriptname", the parameter expansion does not result in the same string:
dirname: "."
parameter expansion: "scriptname"

hope that helps

This is questionable, since it ignores the very real cornercase of "$0" being a plain word ("dirname foo" is ".", not "foo").

scriptdir="$(cd "$(dirname "$0")" && pwd)"
                   ^-- Use parameter expansion instead, such as ${var%/*}.

The shorthand for for arg in "$@"; do, for arg do (note: without the requirement of a semicolon), is reported as a violation of SC107{3,2}, but it is explicitly permitted by POSIX, and in fact is more portable than for arg; do (with semicolon).

A script that triggers this:

for arg do
    echo "$arg"
done

Add a option to create shell and html output compareable to http://www.shellcheck.net/
and create checkstyle output.

Example:
$ shellcheck --recursive --outputdir=target/shellcheck --checkstyle-file target/checkstyle.xml target/ shellscriptfolder

This example would:

• recursively search the folder "shellscriptfolder" for shellscripts
• creates html output in the folder "target/shellcheck"
  (one outputfile for every shellscript, same directory structure like in "shellscriptfolder)
• creates a output file compatible to the well known "checkstyle" tool
  (http://checkstyle.sourceforge.net/)

This might be very useful for automatic build environments.
Jenkins/Hudson provide plugins for publishing html outout or for interpreting checkstyle reports (provide a awesome presentation and statistics from the checkstyle files to the build metrics of the project)

On OS X, the shellcheck compiled output collides with the ShellCheck directory due to case insensitivity. I don't know the convention in Haskell, but perhaps you could output the binary in a build directory (ghc --make shellcheck -o build/shellcheck). Or maybe use cabal.

$ make
: Conditionally compiling shellcheck
ghc -O9 --make shellcheck
[1 of 5] Compiling ShellCheck.AST   ( ShellCheck/AST.hs, ShellCheck/AST.o )
[2 of 5] Compiling ShellCheck.Parser ( ShellCheck/Parser.hs, ShellCheck/Parser.o )
[3 of 5] Compiling ShellCheck.Analytics ( ShellCheck/Analytics.hs, ShellCheck/Analytics.o )
[4 of 5] Compiling ShellCheck.Simple ( ShellCheck/Simple.hs, ShellCheck/Simple.o )
[5 of 5] Compiling Main             ( shellcheck.hs, shellcheck.o )
Linking shellcheck ...
ld: can't open output file for writing: shellcheck, errno=21 for architecture x86_64
collect2: error: ld returned 1 exit status
make: *** [shellcheck] Error 1

This example script converts first letter to capital of first argument passed to it.
Shellcheck reports array "small" is not being used despite it is getting processed inside printf expression.

#!/bin/bash

small=( a b c d e f g h i j k l m n o p q r s t u v w x y z )
capital=( A B C D E F G H I J K L M N O P Q R S T U V W X Y Z )

pos() {
declare -a my_array=("${!2}")
for (( i = 0; i < ${#my_array[@]}; i++ )); do
   if [ "${my_array[$i]}" = "${1}" ]; then
       echo $i;
   fi
done
}

printf "%s%s" "${capital[$(pos "${1::1}" "small[@]")]}" "${1#?}"

exit 0

Compare:

foo() { typeset lul; lul=42; }; foo; echo $lul

function foo { typeset lul; lul=42; }; foo; echo $lul

... and ? are valid function names in bash, however shellcheck chokes on them and stops parsing

I like to run shellcheck *.sh to scan a whole directory at once. But the helpful messages are often hidden inside long lists of No comments for <file>. Could we turn this message off by default, so that shellcheck only shows warnings, like how CheckStyle works?

Hi, would it be possible that you tag a stable release of the software?

That would help getting the package accepted in Homebrew: Homebrew/legacy-homebrew#21231

Also, being a person who's experienced with Haskell, could you please see the discussion at the aforementioned ticket and add your thoughts about the Cabal issues we're having? Namely preventing it from writing into the user's HOME directory (~/.cabal)

If we add shellcheck to Homebrew, users can simply brew install shellcheck.

In rename_files_from_cue.sh line 14:

if (($#!=2)) || [[ ! -f "$2" ]] || [[ ! "$1" =~ ogg|flac ]]; then
^-- The mentioned parser error was in this if expression.
^-- Couldn't parse this test expression.
^-- Unexpected keyword/token. Fix any mentioned problems and try again.

Thanks to Norbert Varzariu for reporting.

Hi,

ShellCheck is awesome, but it fails to parse this code:

for i do
  echo $i
done

It works only when the do statement is clearly separated:

for i; do
  echo $i
done

# or

for i
do
  echo $i
done

All shells (except [t]csh) I have tried support all three syntaxes. POSIX [1] doesn't clearly state how the do reserved word is supposed to delimit the loop's body.

Best Regards,
Dridi

[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_04_03

Consider:
foo=$(something)
rm -r "/usr/$foo"

The assignment could fail due to program errors or ulimit, and rm would then delete /usr/

Is there a way to warn about these things that's useful and not full of false positives?

To wit:

case $var in
a )
foo
b )
bar
esac

There should be a way to disable false positives in comments, e.g.

# shellcheck disable-msg SC2086
echo $1

These should be scoped for structure for which they appear, or the entire file if at the top.

The following test case was created from /etc/cron.daily/prelink on RHEL6:

#!/bin/sh
[ "`find /var/lib/prelink/quick -mtime -${PRELINK_NONRPM_CHECK_INTERVAL:-7} 2>/dev/null`" \
   -a -f /var/lib/rpm/Packages \
   -a /var/lib/rpm/Packages -ot /var/lib/prelink/quick ] && exit 0

# shellcheck testcase.sh 

In testcase.sh line 2:
[ "`find /var/lib/prelink/quick -mtime -${PRELINK_NONRPM_CHECK_INTERVAL:-7} 2>/dev/null`" \
^-- Couldn't parse this test expression.


In testcase.sh line 3:
   -a -f /var/lib/rpm/Packages \
         ^-- Unexpected "/". Fix any mentioned problems and try again.

Here's a slightly modified version with an explicit "-n" which doesn't show the parsing problem:

#!/bin/sh
[ -n "`find /var/lib/prelink/quick -mtime -${PRELINK_NONRPM_CHECK_INTERVAL:-7} 2>/dev/null`" \
   -a -f /var/lib/rpm/Packages \
   -a /var/lib/rpm/Packages -ot /var/lib/prelink/quick ] && exit 0

# shellcheck testcase2.sh 

In testcase2.sh line 2:
[ -n "`find /var/lib/prelink/quick -mtime -${PRELINK_NONRPM_CHECK_INTERVAL:-7} 2>/dev/null`" \
     ^-- Use $(..) instead of deprecated `..`

I.e. there is a parsing problem when the explicit "-n" is missing. According to the man page test(1) both variants should be equivalent:

      -n STRING
             the length of STRING is nonzero

      STRING equivalent to -n STRING

Hi,

I have packaged ShellCheck for Fedora, it should be available for Fedora 19+ in less than a week :)

Now this is my duty to get a man page in the package, but even if it weren't for Fedora I would've asked. Of course I can help with that, if you're not familiar with it. I'd recommend a markup language such as rst (with rst2man, my favorite one) or markdown (with ronn), but man pages are written in (not so) plain text.

There may also be an existing tool in the Haskell ecosystem that would fit better, I don't know.

VERSION=$(expr "$VERSION" : '.*"\(1.[0-9\.]*\)["_]')
          ^-- expr is antiquated. Consider rewriting this using $((..)), ${} or [[ ]].

In this case the regular expression cannot be handled by the suggested mechanisms.

expr "$VERSION" \< 1.2 >/dev/null && continue
^-- expr is antiquated. Consider rewriting this using $((..)), ${} or [[ ]].

expr can compare floats

Interesting project.

I am wondering if it would be possible to use the AST to generate correct shell scripts from Haskell. The use case I have in mind is what Ansible is doing: managing remote hosts with SSH and shell scripts.

I prefer this construct a lot over having an obscure redirect after "done" to feed the while loop:

cat "$pkglist" | while read pkg; do
    ^-- Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead.