DOS batch FOR loop with FIND.exe is stripping out blank lines?

Question

This DOS batch script is stripping out the blank lines and not showing the blank lines in the file even though I am using the TYPE.exe command to convert the file to make sure the file is ASCII so that the FIND command is compatible with the file. Can anyone tell me how to make this script include blank lines?

@ECHO off
FOR /F "USEBACKQ tokens=*" %%A IN (`TYPE.exe "build.properties" ^| FIND.exe /V ""`) DO (
  ECHO --%%A--
)
pause

Answer 1

That is the designed behavior of FOR /F - it never returns blank lines. The work around is to use FIND or FINDSTR to prefix the line with the line number. If you can guarantee no lines start with the line number delimiter, then you simply set the appropriate delimiter and keep tokens 1* but use only the 2nd token.

::preserve blank lines using FIND, assume no line starts with ]
::long lines are truncated
for /f "tokens=1* delims=]" %%A in ('type "file.txt" ^| find /n /v ""') do echo %%B

::preserve blank lines using FINDSTR, assume no line starts with :
::long lines > 8191 bytes are lost
for /f "tokens=1* delims=:" %%A in ('type "file.txt" ^| findstr /n "^"') do echo %%B

::FINDSTR variant that preserves long lines
type "file.txt" > "file.txt.tmp"
for /f "tokens=1* delims=:" %%A in ('findstr /n "^" "file.txt.tmp"') do echo %%B
del "file.txt.tmp"

I prefer FINDSTR - it is more reliable. For example, FIND can truncate long lines - FINDSTR does not as long as it reads directly from a file. FINDSTR does drop long lines when reading from stdin via pipe or redirection.

If the file may contain lines that start with the delimiter, then you need to preserve the entire line with the line number prefix, and then use search and replace to remove the line prefix. You probably want delayed expansion off when transferring the %%A to an environment variable, otherwise any ! will be corrupted. But later within the loop you need delayed expansion to do the search and replace.

::preserve blank lines using FIND, even if a line may start with ]
::long lines are truncated
for /f "delims=" %%A in ('type "file.txt" ^| find /n /v ""') do (
  set "ln=%%A"
  setlocal enableDelayedExpansion
  set "ln=!ln:*]=!"
  echo(!ln!
  endlocal
)

::preserve blank lines using FINDSTR, even if a line may start with :
::long lines >8191 bytes are truncated
for /f "delims=*" %%A in ('type "file.txt" ^| findstr /n "^"') do (
  set "ln=%%A"
  setlocal enableDelayedExpansion
  set "ln=!ln:*:=!"
  echo(!ln!
  endlocal
)

::FINDSTR variant that preserves long lines
type "file.txt" >"file.txt.tmp"
for /f "delims=*" %%A in ('findstr /n "^" "file.txt.tmp"') do (
  set "ln=%%A"
  setlocal enableDelayedExpansion
  set "ln=!ln:*:=!"
  echo(!ln!
  endlocal
)
del "file.txt.tmp"

If you don't need to worry about converting the file to ASCII, then it is more efficient to drop the pipe and let FIND or FINDSTR open the file specified as an argument, or via redirection.

There is another work around that completely bypasses FOR /F during the read process. It looks odd, but it is more efficient. There are no restrictions with using delayed expansion, but unfortunately it has other limitations.

1) lines must be terminated by <CR><LF> (this will not be a problem if you do the TYPE file conversion)

2) lines must be <= 1021 bytes long (disregarding the <CR><LF>)

3) any trailing control characters are stripped from each line.

4) it must read from a file - you can't use a pipe. So in your case you will need to use a temp file to do your to ASCII conversion.

setlocal enableDelayedExpansion
type "file.txt">"file.txt.tmp"
for /f %%N in ('find /c /v "" ^<"file.txt.tmp"') do set cnt=%%N
<"file.txt.tmp" (
  for /l %%N in (1 1 %cnt%) do(
    set "ln="
    set /p "ln="
    echo(!ln!
  )
)
del "file.txt.tmp"

Answer 2

I wrote a very simple program that may serve as replacement for FIND and FINDSTR commands when they are used for this purpose. My program is called PIPE.COM and it just insert a blank space in empty lines, so all the lines may be directly processed by FOR command with no further adjustments (as long as the inserted space don't cares). Here it is:

@ECHO off
if not exist pipe.com call :DefinePipe
FOR /F "USEBACKQ delims=" %%A IN (`pipe ^< "build.properties"`) DO (
  ECHO(--%%A--
)
pause
goto :EOF

:DefinePipe
setlocal DisableDelayedExpansion
set pipe=´)€ì!Í!ŠÐŠà€Ä!€ü.t2€ü+u!:æu8²A€ê!´#€ì!Í!².€ê!´#€ì!Í!²+€ê!´#€ì!Í!Šò€Æ!´,€ì!Í!"Àu°´LÍ!ëÒ
setlocal EnableDelayedExpansion
echo !pipe!>pipe.com
exit /B

EDIT: Addendum as answer to new comment

The code at :DefinePipe subroutine create a 88 bytes program called pipe.com, that basically do a process equivalent to this pseudo-Batch code:

set "space= "
set line=
:nextChar
   rem Read just ONE character
   set /PC char=
   if %char% neq %NewLine% (
      rem Join new char to current line
      set line=%line%%char%
   ) else (
      rem End of line detected
      if defined line (
         rem Show current line
         echo %line%
         set line=
      ) else (
         rem Empty line: change it by one space
         echo %space%
      )
   )
goto nextChar

This way, empty lines in the input file are changed by lines with one space, so FOR /F command not longer omit they. This works "as long as the inserted space don't cares" as I said in my answer.

Note that the pipe.com program does not work in 64-bits Windows versions.

Antonio