weird behaviour of wildcharacter * in shell script

Question

In my shell script I have following code

echo * | tr ' ' '\n'

Here I noticed that although I was using * , it was skipping hidden files(.*) After that I tried an obvious change

echo .* | tr ' ' '\n'

This solved my hidden files problem. But I am just curious regarding this weird behaviour of *

Because .* is subset of *
The desirable output of

1. echo * -> All file including hidden files

2. echo .* -> All hidden files

3. echo [^.]* -> All non-hidden files(currently echo *)

Hence echo * is behaving like echo [^.]*

How do I get entire list of files including hidden files using echo. Similar was the output for ls and dir, although ls -a was giving desirable outputs

Answer 1

The shell glob * is defined as ignoring hidden files, so works as expected. And .* will expand only to hidden files.

Some shells allow to change this behavior via an option. E.g. the zsh allows setopt dotglob to become non-conforming (to POSIX) and also glob dotfiles by default. For bash you can use shopt -s dotglob. But beware that scripts may malfunction as they usually assume POSIX behavior for globbing.

Your best bet to get all files is to not use echo with globbing, but for example find . -maxdepth 1 (if you're desparate for echo, maybe echo * .* will do, but it has problems if either glob does not match any file, in which case the glob pattern may be retained).

Answer 2

This is covered in Filename Expansion

When a pattern is used for filename expansion, the character ‘.’ at the start of a filename or immediately following a slash must be matched explicitly, unless the shell option dotglob is set. When matching a file name, the slash character must always be matched explicitly. In other cases, the ‘.’ character is not treated specially.

If you want to include .* along with *, you must give both parameters to echo

echo .* * | tr ' ' '\n'

or use dotglob

shopt -s dotglob
echo * | tr ' ' '\n'

which still excludes . and .. though.