Issue with special characters in path (exclamation point !, carrot ^, etc) using batch Delayed Expansion

Question

I have looked around and not been able to find anything to get my script working correctly with special characters (such as ! or ; or ^) in the file path or file name.

My script does work, but only if the above characters are not in any of the scanned folders or file names. If any folders or files have those characters, then the script breaks down. I need help figuring out how to make my script work with special characters (like above) within the path or file name. Here is my script:

set srcdir=%~dp0%src

set desdir=%~dp0%des

setlocal EnableDelayedExpansion
for /r "%srcdir%" %%f in ("*.txt") do (
    set "subdir=%%~f"
    set "subdir=!subdir:%srcdir%=%desdir%!"
    echo !subdir!
    pause
)
endlocal

Thanks for any and all assistance!

Answer 1

Exclamation marks get clobbered when delayed expansion is enabled while you set a variable. You can avoid this by waiting to delay expansion until you retrieve a variable value. Sometimes this takes some acrobatics to make it work. In this case, it's probably easier just to leave delayed expansion disabled and use call to delay expansion.

@echo off
setlocal

set "srcdir=%~dp0%src"
set "desdir=%~dp0%des"

for /r "%srcdir%" %%f in ("*.txt") do (
    set "subdir=%%~f"

    rem // use call to avoid delayed expansion and preserve exclamation marks
    call set "subdir=%%subdir:%srcdir%=%desdir%%%"

    rem // use set /p rather than echo to exploit quotation marks and preserve carets
    call set /P "=%%subdir%%"<NUL & echo;
    pause
)

Or if you prefer delayed expansion, one trick I like to use to toggle delayed expansion for one line is to use a for loop like this:

@echo off
setlocal

set "srcdir=%~dp0%src"
set "desdir=%~dp0%des"

for /r "%srcdir%" %%f in ("*.txt") do (
    set "subdir=%%~f"
    setlocal enabledelayedexpansion
    for %%I in ("!subdir:%srcdir%=%desdir%!") do endlocal & set "subdir=%%~I" & echo(%%~I
    pause
)

Answer 2

1. Put all paths in between "".
2. Always use syntax set "VAR=Value".
3. Toggle delayed expansion: when expanding %%~F, disable it; afterwards, enable it.

Here is the fixed code:

setlocal DisableDelayedExpansion

set "srcdir=%~dp0src"
set "desdir=%~dp0des"

for /R "%srcdir%" %%F in ("*.txt") do (
    set "subdir=%%~F"
    setlocal EnableDelayedExpansion
    set "subdir=!subdir:%srcdir%=%desdir%!"
    echo(!subdir!
    pause
    endlocal
)
endlocal

This only works as long as the directory where the batch file is stored does not contain exclamation marks. If it does, let me know...

---

Amendment

Sub-string substitution using also variables for the search and replace strings is never safe against all characters in general.

Imagine you have something like this:

echo(!VAR:%SEARCH%=%REPLACE%!

This means to replace every occurrence of %SEARCH% by %REPLACE% (in a case-insensitive manner).

But if %SEARCH% contains a =, the behaviour is changed: for instance, %SEARCH% is a=b and %REPLACE% is cd, the immediately expanded version is !VAR:a=b=cd!, so every a is going to be replaced by b=cd.

A leading * in %SEARCH% changes the behaviour: replace everything up to and including the rest of %SEARCH% by %REPLACE%. (An asterisk cannot occur within a path, of cource.)

A leading ~ in %SEARCH% changes the the behaviour from sub-string substitution to sub-string expansion, that is, expansion of a string portion given by character position an length; if the syntax is violated, the non-expanded string !VAR:~a=b! will be returned literally, supposing ~a and b are the search and replace strings, respectively.

Finally, if %SEARCH% and/or replace contain a !, this is going to be taken as the closing ! for the delayed expansion, so !VAR:a!=b! is seen as !VAR:a!, which is invalid syntax and will be kept as is.