hide command prompt when using subprocess.Popen with shell=False on Windows

[iMath]

BPO	30082
Nosy	@pfmoore, @tjguk, @zware, @eryksun, @zooba, @ohn0, @ammgws
PRs	bpo-30082 hide command prompt when using subprocess.Popen with shell=False on Windows #3013 bpo-30082 Popen: add kwarg to hide process window on Windows #19014
Files	test.py: Script to test if subprocess spawns additional windows

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-04-16.17:26:26.499>
labels = ['easy', 'type-feature', 'library', 'OS-windows', '3.11']
title = 'hide command prompt when using subprocess.Popen with shell=False on Windows'
updated_at = <Date 2021-11-09.13:15:20.475>
user = 'https://bugs.python.org/iMath'

bugs.python.org fields:

activity = <Date 2021-11-09.13:15:20.475>
actor = 'swgmma'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'Windows']
creation = <Date 2017-04-16.17:26:26.499>
creator = 'iMath'
dependencies = []
files = ['48938']
hgrepos = []
issue_num = 30082
keywords = ['patch', 'easy']
message_count = 30.0
messages = ['291760', '291815', '329979', '330091', '362978', '362979', '363046', '363060', '363116', '363143', '364227', '403897', '404094', '404095', '404122', '404261', '404264', '405151', '405166', '405177', '405205', '405269', '405285', '405298', '405302', '405307', '405309', '405362', '405576', '406019']
nosy_count = 8.0
nosy_names = ['paul.moore', 'tim.golden', 'zach.ware', 'eryksun', 'steve.dower', 'iMath', 'ohno', 'swgmma']
pr_nums = ['3013', '19014']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue30082'
versions = ['Python 3.11']

[iMath]

First, It is nearly useless for the command prompt to pop up during the running time of subprocess.Popen with shell=False.
Second, the popping up command prompt would interrupt users and do bad to user experience of GUI applications.
Third, I found QProcess within Qt won't pop up the command prompt in using.

It would be convenient to add an argument to suppress the command prompt from popping up when using subprocess.Popen with shell=False on Windows, many users are missing the feature and these are many similar feature request questions like the following
http://stackoverflow.com/questions/7006238/how-do-i-hide-the-console-when-i-use-os-system-or-subprocess-call
http://stackoverflow.com/questions/1765078/how-to-avoid-console-window-with-pyw-file-containing-os-system-call/12964900#12964900
http://stackoverflow.com/questions/1016384/cross-platform-subprocess-with-hidden-window

[eryksun]

Hiding the console is required often enough that a keyword-only parameter for this would be useful. Other platforms could simply ignore it, in contrast to writing platform-dependent code that passes startupinfo. Note that this option would also hide the window of GUI programs that do not (intentionally) ignore SW_HIDE, such as notepad.

[ohn0]

Can I work on this? I'm not sure of it's status, though.

[zooba]

Looks like it's available.

It will be a new subprocess option to not create a new window. The underlying issue is that some applications always request a console, and if your own application doesn't have one then Windows will create it. Normally you would fix this in the target application, but we can offer an option to try and force any new windows to be hidden (though as Eryk points out, it may cause unexpected behaviour and so should not be the default).

[ammgws]

I would like to pick up work on this (have started at link below), however I would like some help:

For testing purposes, could someone please provide a minimum working example of using subprocess that causes a command prompt to pop up?

I have not been able to reproduce it (using Python 3.7), though I vaguely remember encountering it in the past.

ammgws@03b9dac

[zooba]

If you use subprocess to launch any process that is marked SUBSYSTEM_CONSOLE, Windows will create a new console for it. For example, "python.exe" is such a process.

Eryk's answer in the very first StackOverflow link shows how to create a STARTUPINFO parameter with the correct settings. We just want to add a Boolean argument to subprocess.Popen.__init__() that can add that setting automatically.

[ammgws]

Sorry, perhaps I did not word it clearly. Indeed the code to implement this looks trivial, however the part I am having trouble with is reproducing the reported issue in the first place.

For example, running the attached file from a command prompt (python test.py) does not result in any additional windows popping up.

Either my test case is wrong or there is something else going on.

[zooba]

Try running that script with pythonw.exe instead.

[iMath]

To reproduce the reported issue, one could also test with ffmpeg.exe

[zooba]

It'll have to be a program that cannot inherit the active console. Ffmpeg normally can, but running the first script with pythonw will make sure there is no console to inherit, and a new one will pop up for a regular Python process.

[ammgws]

Try running that script with pythonw.exe instead.

That did the trick. Confirmed that the changes are working as intended.

Running the test script posted earlier and adding the new force_hide kwarg to the subprocess call:

With force_hide=False, a command prompt window pops up.
With force_hide=True, no command prompt window pops up.

[ammgws]

Ping in case anyone from the windows team is available to review the PR

[zooba]

Had a look. The change looks fine, just needs a NEWS entry, which I suggested as:

Adds new `force_hide` argument to :mod:`subprocess` functions. This passes ``SW_HIDE`` to the new process, which most applications will use to not display any window even if they normally would.

[zooba]

That 'force_hide' in the NEWS shouldn't have single backticks. Make them double, or the docs won't compile.

[eryksun]

The force_hide option could also force the use of a new hidden window, even if the current process has a console. That way the output is always hidden. For example:

    if force_hide or shell:
        if force_hide and not (creationflags & _winapi.DETACHED_PROCESS):
            creationflags |= _winapi.CREATE_NEW_CONSOLE
        startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW
        startupinfo.wShowWindow = _winapi.SW_HIDE

One can also use CREATE_NO_WINDOW to create a console that has no window, as opposed to a hidden window. For example:

    if force_no_window:
        if creationflags & _winapi.CREATE_NEW_CONSOLE:
            raise ValueError('force_no_window cannot be used with '
                             'CREATE_NEW_CONSOLE')
        creationflags |= _winapi.CREATE_NO_WINDOW

[ammgws]

I implemented your first suggestion for force_hide.

Should we add your force_no_window suggestion as well?

[eryksun]

I was intending force_no_window as an alternative to adding force_hide, since CREATE_NO_WINDOW only affects console applications. Maybe a better name is force_hide_console.

[ammgws]

Just added a commit implementing force_hide_console.

What is the most trivial way to test it's behaviour?

[eryksun]

What is the most trivial way to test it's behaviour?

With CREATE_NO_WINDOW, the child process is attached to a console that has no window. Thus calling GetConsoleWindow() in the child returns NULL without an error. OTOH, if the child has no console (e.g. DETACHED_PROCESS or executing "pythonw.exe"), then GetConsoleWindow() returns NULL with the last error set to ERROR_INVALID_HANDLE (6). For example:

    script = r'''
    import sys
    import ctypes
    kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)
    kernel32.GetConsoleWindow.restype = ctypes.c_void_p
    ctypes.set_last_error(0)
    result = kernel32.GetConsoleWindow()
    status = bool(result or ctypes.get_last_error())
    sys.exit(status)
    '''
    args = [sys.executable, '-c', script]

The child's exit status will be 0 if CREATE_NO_WINDOW works as expected. Otherwise the exit status will be 1.

[ammgws]

For users who simply want to hide a process' window and do not have intimate knowledge of how the window is created by Windows, is it feasible to just have one option that handles all cases? Or do we have to implement both force_hide and force_hide_console and let users figure out which to use from the docs?

[eryksun]

is it feasible to just have one option that handles all cases?

I'm in favor of whatever has broad support and is least surprising for users. I like CREATE_NO_WINDOW, which is restricted to console applications. The alternative is hiding the window, like what we do with shell=True. But the new option would also hide the main window of many GUI apps, which may come as a surprise and is hardly ever desired behavior.

[zooba]

I'd rather only have one force_hide option that does its best to hide everything.

Yes, it's probably unintuitive for GUI apps, it might even be maliciously misused, but it does have occasional legitimate uses. And besides, an app can easily ignore SW_HIDE if it wants.

Bear in mind that the test suite should (mostly) operate without ctypes. So if you use ctypes to verify the test, make it a separate test that skips if ctypes cannot be imported, and have a test that runs it without verifying the window is actually hidden. That way we always at least _run_ the code, even if we only (usually) verify that it's actually hiding the window.

[eryksun]

If only force_hide is implemented, based on wShowWindow=SW_HIDE, please ensure that it's clearly documented that this option hides the main window (i.e. first top-level window) of any application that doesn't bypass the default behavior of the window manager.

This means a script can't simply use force_hide=True universally when running arbitrary commands, since hiding the main window of a GUI app is probably undesired behavior. If it's unknown in advance whether an executable is a console or GUI app, it can be determined via SHGetFileInfoW(). For example:

    import ctypes
    shell32 = ctypes.WinDLL('shell32', use_last_error=True)
    SHGFI_EXETYPE = 0x2000

    def is_gui_exe(path):
        result = shell32.SHGetFileInfoW(path, 0, None, 0, SHGFI_EXETYPE)
        # the high word is non-zero for a GUI app
        return (result >> 16) > 0

    >>> is_gui_exe(r'C:\Windows\pyw.exe')
    True
    >>> is_gui_exe(r'C:\Windows\py.exe')
    False

Regarding the effect of wShowWindow:

If the STARTUPINFO record defines wShowWindow, the value gets used as the normal show command for the first top-level window that's shown in the process. In particular, it overrides the first use (and only the first use) of the commands SW_SHOW, SW_SHOWNORMAL, and SW_SHOWDEFAULT (yes, only the first use of the latter; the documentation is wrong). This affects showing the first top-level window regardless of whether it's explicit via ShowWindow() or implicit via the WS_VISIBLE window style. For the latter, the implicit command is SW_SHOW, which one can override in the CreateWindowW() call by passing x=CW_USEDEFAULT and y as the show command to use.

To completely ignore the STARTUPINFO wShowWindow value for the main window, an application can use an initial show command that wShowWindow doesn't override, such as SW_HIDE, SW_RESTORE (behaves like SW_SHOWNORMAL on first use), SW_MINIMIZE, or SW_MAXIMIZE. For example, for the first command use ShowWindow(hwnd, SW_HIDE). Subsequently calling ShowWindow(hwnd, SW_SHOWNORMAL) will show the window normally (i.e. active and restored) instead of using the STARTUPINFO wShowWindow command.

[ammgws]

What if we change force_hide from True/False to something like "off" | "console" | "all" (defaulting to "off")?

[zooba]

There's nothing gained by complicating this API with more options.

Document it as "Passing *force_hide* as True attempts to start the application without creating or showing any windows. Some applications may ignore this request, and applications that are hidden often cannot be used or exited by users."

We don't have to specify all the nuances here, especially since we may come up with a more reliable way to do this in the future. Specifying too much detail in the documentation will prevent us from improving it without breaking stuff.

[eryksun]

There's nothing gained by complicating this API with more options.

Yes, both options is too much. I suggested force_hide_console as an alternative to force_hide, not for both to be implemented. It would be the same as CreateNoWindow in .NET ProcessStartInfo [1], but ProcessStartInfo also has WindowStyle, and one or the other is ignored depending on the value of UseShellExecute. I see a lot of confusion online regarding these .NET ProcessStartInfo properties, which is something I'd like to avoid.

A benefit of force_hide based on wShowWindow=SW_HIDE is feature parity with ShellExecuteExW(), if subprocess ever supports the shell API. ShellExecuteExW() does not support CREATE_NO_WINDOW, but it supports a show-window command and CREATE_NEW_CONSOLE (i.e. the inverse of SEE_MASK_NO_CONSOLE).

---
[1] https://docs.microsoft.com/en-us/dotnet/api/system.diagnostics.processstartinfo?view=net-5.0

[ammgws]

Thanks all for the guidance. Have gone back to the original force_hide with the suggested documentation.

[eryksun]

Here's an example test that calls WinAPI GetConsoleWindow() and IsWindowVisble() in a child process.

    if mswindows:
        try:
            import ctypes
        except ImportError:
            ctypes = None

    # added in Win32ProcessTestCase
    def test_force_hide(self):
        if ctypes:
            script = textwrap.dedent(r'''
                import sys, ctypes
                GetConsoleWindow = ctypes.WinDLL('kernel32').GetConsoleWindow
                IsWindowVisible = ctypes.WinDLL('user32').IsWindowVisible
                sys.exit(IsWindowVisible(GetConsoleWindow()))
            ''')
        else:
            script = 'import sys; sys.exit(0)'
        rc = subprocess.call([sys.executable, '-c', script], force_hide=True)
        self.assertEqual(rc, 0)

This test will work reliably even if the user has Windows Terminal set as the default terminal (available with recent builds of Windows and Windows Terminal Preview). The proxy does not hand off to Windows Terminal if the STARTUPINFO wShowWindow is SW_HIDE or SW_SHOWMINIMIZED, or if the process is created with the CREATE_NO_WINDOW flag. In these cases, it just creates a system conhost.exe session.

[ammgws]

Anything else left to do?

[ammgws]

Just squashed all the commits.

[ammgws]

Reckon there is a chance that this make it in before the 3.11 feature freeze?