Popen with conflicting executable/path

Question

I'd like to call the "convert" utility from ImageMagick from my Python script using Popen, like so:

Popen(["convert", input_path, "-flop", output_file_path])

(The above example simply reverses the image horizontally)

The problem is that when I run the script in Windows, it mistakenly calls the convert.exe utility that ships with Windows to convert FAT partitions to NTFS! (located in \Windows\system32)

Now, if I randomly open a command prompt at any directory other than system32, and type "convert", it correctly runs the ImageMagick executable. So, this implies that Popen is automatically looking in system32. How can I make it not look in system32, and run the correct executable?

Answer 1

Search for a program is not trivial. I'd specify the full path to the convert.exe executable explicitly instead.

subprocess uses CreateProcess on Windows that looks in system32 directory even before any other directory in %PATH%:

... If the file name does not contain an extension, .exe is appended. Therefore, if the file name extension is .com, this parameter must include the .com extension. If the file name ends in a period (.) with no extension, or if the file name contains a path, .exe is not appended. If the file name does not contain a directory path, the system searches for the executable file in the following sequence:

1. The directory from which the application loaded.
2. The current directory for the parent process.
3. The 32-bit Windows system directory. Use the GetSystemDirectory function to get the path of this directory.
4. The 16-bit Windows system directory. There is no function that obtains the path of this directory, but it is searched. The name of this directory is System.
5. The Windows directory. Use the GetWindowsDirectory function to get the path of this directory.
6. The directories that are listed in the PATH environment variable. Note that this function does not search the per-application path specified by the App Paths registry key. To include this per-application path in the search sequence, use the ShellExecute function.

Therefore convert is equivalent to convert.exe in this case. It first looks in a directory that contains sys.executable e.g., C:\Python27. Then in the current directory: where you started the Python script from. Then in system32 where it finds convert.exe (filesystem utility, not imagemagick).

You could try to remove system32 directory from os.environ['PATH'] it may(?) suppress checking it: Popen(cmd, env=no_system32_environ) but it is fragile (worse than the explicit path).

There is a related issue on Python bug tracker: "Subprocess picks the wrong executable on Windows."

---

cmd.exe (the shell) uses different algorithm. See How does Windows locate files input in the shell?

If you set shell=True then the search sequence for convert program:

1. convert is not an internal shell command
2. there is no explicit path, so the search continues
3. search the current directory
4. search each directory specified by the PATH environment variable, in the order listed

%PATHEXT% defines which file extensions are checked and in what order e.g., convert.com, convert.exe, convert.bat, convert.cmd if %PATHEXT% is .com;.exe;.bat;.cmd.

Answer 2

As a completely different approach, you may want to try out PythonMagick, a Python wrapper for ImageMagick. This way you can access convert's functions from within Python, and you won't have to spawn outside processes.