Why can't I access a variable named __CD__ on Windows 7?

Question

Note - this question is based on behavior observed on Windows 7. I believe the behavior applies to all other versions from Vista onward. Based on MC ND's answer, and Foxidrive's comments, it does not apply to XP.

I can use the pseudo dynamic variable %CD% to get the current directory, without a trailing \. If I define a true variable named CD using set "CD=someValue", then %CD% returns the value I assigned, and not the current directory.

A lesser known dynamic variable is %__CD__%, which is the same as %CD%, except it includes the trailing \. But if I do set "__CD__=someValue", I cannot access my assigned value! I can use set __CD__ to see that my variable exists with my assigned value, but %__CD__% always returns the current directory!

Does anyone have any idea how this %__CD__% mechanism works (why it is different than any other dynamic variable)?

Can someone come up with a way to access my __CD__ variable using nothing but standard batch commands (without parsing the output of set __CD__)?

This is more an academic question, not a practical one. No one should be defining variables with names that match dynamic variables.

---

The reason I do not want a routine that parses the results of set __CD__ is because values can have line feeds in them. Someone could define two variables, __CD__ and __CD__2, or they could define a single __CD__ value with a value containing a line feed followed by __CD__2=.... It would be impossible to differentiate between those two scenarios. (As I said, this is an academic question, not a practical one!)

Answer 1

I have a theory as to how and why __CD__ behaves so differently than any of the other pseudo environment variable.

I wrote a simple ENV.JS script to probe the process environment.

var env=WScript.CreateObject("WScript.Shell").Environment("Process");
WScript.echo(env(WScript.Arguments.Item(0)));

ENV.JS expects an environment variable name as the one and only argument, and simply prints the value of the variable. I did my testing on a Windows 7 machine. The script can be run from within a CMD.EXE console, or it can be run directly via a shortcut that defines the argument.

I will classify the various dynamic variables, showing contrasting behaviors, along with a theory as to the mechanism by which they work.

There are a number of dynamic variables that can be divided into three classes:

1) Normal looking "variables" without any prefix or suffix

CD              current directory
DATE            current date
TIME            current time
ERRORLEVEL      current errorlevel
RANDOM          random integer between 0 and 32767
CMDEXTVERSION   current extension level (only available if extensions are enabled)
CMDCMDLINE      the command line that invoked the current CMD.EXE level

The dynamic values are only available via expansion within CMD.EXE, and only when command extensions are enabled. They are not available via SET, or by ENV.JS. The dynamic values can be overridden by explicitly defining a static value using SET. The override values are available via SET and ENV.JS.

C:\test>echo %cmdcmdline%
"C:\Windows\system32\cmd.exe"

C:\test>set cmdcmdline
Environment variable cmdcmdline not defined

C:\test>cscript //nologo env.js cmdcmdline
cmdcmdline=

C:\test>set cmdcmdline=override

C:\test>echo %cmdcmdline%
override

C:\test>set cmdcmdline
cmdcmdline=override

C:\test>cscript //nologo env.js cmdcmdline
cmdcmdline=override

C:\test>cmd /e:off
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\test>echo %cmdcmdline%
%cmdcmdline%

The dynamic values are not really environment variables at all, which is why they are not accessible to the SET command or to ENV.JS. The variable expansion code in CMD.EXE first checks the process environment variables for the variable, and only if not found does it compare the name against the list of special dynamic names. Special CMD.EXE code must exist for each dynamic "variable" to derive the value from the appropriate source.

The behavior of this class of dynamic variables is described by Microsoft's Raymond Chen within his blog - The Old New Thing

2) Dynamic variables prefixed with =

=ExitCode
=ExitCodeAscii
=C:
=D:
...
=Z:

Each of these dynamic values is undefined until some command is executed that defines it. For example, a brand new CMD.EXE console session starts with =ExitCode undefined. It becomes defined once an external command is executed that set a return code.

There is one exception in that the ={driveLetter}: variable corresponding to the current directory will always be defined, even when CMD.EXE first starts up.

It is impossible to use SET to define any of these variables because SET does not allow = in a variable name. But the underlying process environment variable space does allow = in variable names. It just must be done outside of the CMD.EXE context.

I've written an additional TEST.JS to help test these variables:

var shell=WScript.CreateObject("WScript.Shell");
var env=shell.Environment("Process");
WScript.echo('Within JScript: ExitCode='+env("=ExitCode"));
env("=ExitCode") = "override";
WScript.echo('Within JScript after override: ExitCode='+env("=ExitCode"));
WScript.echo('Within JScript test.bat return code = '+shell.run("cmd /c test.bat",10,1));
WScript.echo('Within JScript after test.bat: ExitCode='+env("=ExitCode"));

In turn, TEST.JS calls TEST.BAT:

@echo off
echo Within test.bat: ExitCode=%=ExitCode%
cmd /c exit 10
echo =Within test.bat: ExitCode=%=ExitCode%
pause
exit %errorlevel%

Here are some test results using =ExitCode, starting with a brand new CMD.EXE session:

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\test>echo %=ExitCode%
%=ExitCode%

C:\test>set "=ExitCode=override"
The syntax of the command is incorrect.

C:\test>cscript //nologo env.js =ExitCode
=ExitCode=

C:\test>REM cscript is an external command that set the value!

C:\test>echo %=ExitCode%
00000000

C:\test>set =
The syntax of the command is incorrect.

C:\test>set ""|findstr /b =ExitCode
=ExitCode=00000000

C:\test>cscript //nologo env.js =ExitCode
=ExitCode=00000000

C:\test>cscript //nologo test.js
Within JScript: ExitCode=00000000
Within JScript after override: ExitCode=override
Within JScript test.bat return code = 10
Within JScript after test.bat: ExitCode=override

C:\test>

Here are the results of TEST.BAT that TEST.JS launched in a new Window:

Within test.bat: ExitCode=override
Within test.bat: ExitCode=0000000A
Press any key to continue . . .

I believe that these dynamic variables are true environment variables, which is why both SET and JScript can access them. (SET can only access the value using the special SET "" syntax.) The variables are dynamically defined (or updated) by CMD.EXE each time a relavent command is executed. ENV.JS and TEST.JS can see the value that was set by the calling CMD.EXE session. The TEST.BAT cmd session could see the inherited override value that TEST.JS set. But JScript continued to get the override value after TEST.BAT exited because CMD.EXE was not there to update the value with the return code.

This class of dynamic variables is available regardless whether command extensions are enabled or disabled. The dynamic values are maintained even when extensions are disabled, as evidenced below:

C:\test>cmd /e:off
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\test>echo %=c:%
C:\test

C:\test>cd test

C:\test\test>echo %=c:%
C:\test\test

3) __CD__ - A special case all unto itself!
EDIT - Actually, __APPDIR__ works the same way

This dynamic variable is always available to both CMD.EXE and JScript. An override value can be defined, but neither CMD.EXE nor JScript can see the override value, except the SET command can list the override value. (Also, not shown, but jeb discovered SET /A can read the override value if it is numeric).

I wrote yet another TEST2.JS to probe this variable.

var shell=WScript.CreateObject("WScript.Shell");
var env=shell.Environment("Process");
WScript.echo('Within JScript: __CD__='+env("__CD__"));
env("__CD__") = "JS override";
WScript.echo('Within JScript after override: __CD__='+env("__CD__"));
shell.run('cmd /c "set __CD__&pause",1,0');

Here are the results of some tests:

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\test>echo %__CD__%
C:\test\

C:\test>set __CD__
Environment variable __CD__ not defined

C:\test>set "__CD__=Batch override"

C:\test>echo %__CD__%
C:\test\

C:\test>set __CD__
__CD__=Batch override

C:\test>cscript //nologo test2.js
Within JScript: __CD__=C:\test\
Within JScript after override: __CD__=C:\test\

C:\test>

Here is the result of the CMD.EXE window that TEST2.JS opened up:

__CD__=JS override
Press any key to continue . . .

If I define a shortcut to ENV.JS as follows:

Target:   C:\test\env.js __CD__
Start in: C:\test\xyz

Then when I click on it, I get an alert box stating:

__CD__=C:\test\xyz\

I find these results fascinating. The dynamic value must not be a true environment variable. Presumably there is a low level OS environment variable access routine that automatically returns the process's current directory whenever it is asked to return the value of __CD__. It does so, even if a true static environment variable named __CD__ is defined.

The CMD.EXE SET command must access the environment variable differently than most other contexts. I imagine a C program could be written to get a pointer to the process environment memory and parse any true user defined __CD__ value, much as the SET command.

Given that it is a low operating system routine that provides this value, it is not surprising that %__CD__% is available even when command extensions are disabled.

C:\test>cmd /e:off
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\test>echo %__cd__%
C:\test\

C:\test>cd test

C:\test\test>echo %__cd__%
C:\test\test\

I suppose the concept of a current directory is critical for every process from an OS perspective, and MS decided to give a process access to the value via a dynamic virtual environment variable. The XP OS allows expansion of any user defined __CD__ override value. Perhaps that caused problems for some applications, and MS decided to modify the environment access routines (starting with Vista perhaps?) to always return the true current directory, regardless of any user defined __CD__ variable.

So, based on the theory that it is a low level routine that is returning the current directory, I now believe it is impossible to use native batch commands to reliably get any user defined value for __CD__.

Answer 2

[This should be a comment, but it would be a very large one!]

I would like to add that %CD% variable is also a special case for the following reason: if a setlocal command is executed and then the current directory is changed, a posterior endlocal command change the current directory back to the one active when setlocal was executed:

@echo off

echo Original: %CD%
setlocal
md newdir
cd newdir
echo In newdir: %CD%
endlocal
echo After endlocal: %CD%

This behavior indicate that %CD% dynamic variable is saved with setlocal's and restored with (explicit or implicit) endlocal's, in the same way of normal variables. However, a weird point is that this mechanism works even if a user CD variable is defined before or after the setlocal! The conclusion is that setlocal command saves the current directory in an area separated from the environment variables, and that endlocal command restore the current directory from that area.