Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The I/O operation has been aborted because of either a thread exit or an application request #5633

Closed
bityob opened this issue Jul 22, 2020 · 56 comments

Comments

@bityob
Copy link

bityob commented Jul 22, 2020

Environment

Windows build number:  Microsoft Windows NT 10.0.20175.0
Your Distribution version:  Ubuntu (can't run because failed to run the bash)
Whether the issue is on WSL 2 and/or WSL 1: WSL1

Steps to reproduce

Just opening the WSL (Ubuntu) after a windows update (insider build).

image

lxcore details logs -

lxcore-files.zip

Expected behavior

Should work

Actual behavior

Error:
The I/O operation has been aborted because of either a thread exit or an application request

@nathpete-msft
Copy link
Member

I'm hitting the same issue, and am on the same Windows build with WSL1.
Even just trying to list distributions hangs and fails like this: wsl --list --all

@benhillis
Copy link
Member

@bityob - Interesting, I'm able to repro this but this didn't hit in our automated testing. I'll figure out what's going on. In the meantime I'd suggest rolling back to the previous version or using WSL2. Looks like conversion is working ok.

@benhillis benhillis self-assigned this Jul 22, 2020
@benhillis benhillis added the bug label Jul 22, 2020
@benhillis
Copy link
Member

Ok... this issue seems to be related to when the init daemon receives a larger than expected message from our service and has to resize our buffer. You may be able to work around this by temporarily reducing the length of your windows %PATH% variable.

@benhillis
Copy link
Member

Alright, I've fixed this and written a test so this kind of thing won't happen again.

Sorry about the disruption this caused... I'll try to get the fix into an Insider Fast build ASAP.

@benhillis
Copy link
Member

@nathpete-msft - the list issue you mention looks different. Can you file a new issue with more details, please?

@bityob
Copy link
Author

bityob commented Jul 23, 2020

@benhillis

You may be able to work around this by temporarily reducing the length of your windows %PATH% variable.

Thanks, this did solve the issue -

set PATH=
C:\Windows\System32\wsl.exe

or

set PATH=C:\Windows\System32
wsl

@dcharlespyle
Copy link

Yep. I'm definitely affected by this bug. I started shortening my %PATH%, one program path string at a time until I got down to 989 characters, including spaces (so, assuming a limit of 1024 characters or just below). Once I got it pared down to 989 characters, WSL started working again as normally. But my machine where I test preview builds also has Windows Kits, developer environments, multimedia software, diagnostics, and so forth, all of which need to be in the path to work properly. Why was WSL limited to a certain path length? Any way to work around this without shortening the path in a cmd prompt before running WSL? I have use it to run Linux software side-by-side with Windows on the fly. Haven't been able to do this since updating to Windows 10 build 20175.

@microsoft microsoft deleted a comment from dcharlespyle Jul 24, 2020
@dcharlespyle
Copy link

dcharlespyle commented Jul 24, 2020

Temporary workaround I started using today is to launch terminal from PowerShell:

c:\windows\system32\cmd.exe /c "path %SystemRoot%\System32; && %USERPROFILE%\AppData\Local\Microsoft\WindowsApps\wt"

Alternatively, a shortcut can be created with the following:

C:\WINDOWS\system32\cmd.exe /c "path %SystemRoot%\System32; && %USERPROFILE%\AppData\Local\Microsoft\WindowsApps\wt"

Or, you can use:

C:\Windows\System32\cmd.exe /c "path C:\Windows\system32; && %USERPROFILE%\AppData\Local\Microsoft\WindowsApps\wt.exe"

Set the "Run:" property on the Shortcut tab to Minimized and click apply. So, until the above bug is fixed, this is what I have started doing for the time being. Similar can be done with individual Linux apps, but with the wt.exe line replaced with the path to your Linux program.

Note: If you also use PowerShell through Windows Terminal, you'll also have to add the path to your PowerShell installation in addition, like this:

C:\Windows\System32\cmd.exe /c "path C:\Windows\system32;C:\WINDOWS\System32\WindowsPowerShell\v1.0\; && %USERPROFILE%\AppData\Local\Microsoft\WindowsApps\wt.exe"

@bityob
Copy link
Author

bityob commented Jul 25, 2020

@dcharlespyle

I started shortening my %PATH%, one program path string at a time until I got down to 989 characters, including spaces (so, assuming a limit of 1024 characters or just below). Once I got it pared down to 989 characters, WSL started working again as normally

I wrote a simple python script to check the limit in my machine.

In my case the limit was not always the same (805/817/878 characters).

The script needs Python 3.6+ to run.

import os
from subprocess import STDOUT, check_output
import platform

# Get the path to the wsl executable, it's different if python is 32bit or 64bit
# See here - https://stackoverflow.com/a/41631476
system32 = 'SysNative' if platform.architecture()[0] == '32bit' else 'System32'
wsl_command = os.path.join(os.environ['SystemRoot'], system32, 'wsl.exe')

min_num = 800
max_num = 2000
path = ""

try:
    for i in range(min_num, max_num):
        path = "A" * i

        curr_command = f"set PATH={path} && {wsl_command} uptime"

        print(f"Length of path: {len(path)}") 
        
        # Raise exception in case of error
        output = check_output(curr_command, stderr=STDOUT, timeout=60, shell=True)
        
        print(output.decode('utf8').strip())
except Exception as ex:
    print(f"FAILED. Output: {ex.output.decode('utf8')}")

@iccfish
Copy link

iccfish commented Jul 26, 2020

I've shorten my PATH to 770 characters, but still can not start WSL with extractly same error.

@jonnyasmar
Copy link

jonnyasmar commented Jul 26, 2020

If you're running into the issue with too long of a PATH and you're using Windows Terminal, you can set up a profile with the following commandline property to automatically clear your PATH before running WSL. This will leave PATH in tact for other shells.

"commandline": "cmd.exe /C \"set PATH= && \"C:\\WINDOWS\\System32\\wsl.exe\"\"",

If you're using VSCode, you can resolve this by using the following settings:

  "terminal.integrated.shell.windows": "C:\\WINDOWS\\System32\\wsl.exe",
  "terminal.integrated.env.windows": {
    "PATH": ""
  },

@iccfish
Copy link

iccfish commented Jul 26, 2020

If you're running into the issue with too long of a PATH and you're using Windows Terminal, you can set up a profile with the following commandline property to automatically clear your PATH before running WSL. This will leave PATH in tact for other shells.

"commandline": "cmd.exe /C \"set PATH= && \"C:\\WINDOWS\\System32\\wsl.exe\"\"",

If you're using VSCode, you can resolve this by using the following settings:

  "terminal.integrated.shell.windows": "C:\\WINDOWS\\System32\\wsl.exe",
  "terminal.integrated.env.windows": {
    "PATH": ""
  },

Not working for me 😢

@dcharlespyle
Copy link

dcharlespyle commented Jul 26, 2020

If you're running into the issue with too long of a PATH and you're using Windows Terminal, you can set up a profile with the following commandline property to automatically clear your PATH before running WSL. This will leave PATH in tact for other shells.

"commandline": "cmd.exe /C \"set PATH= && \"C:\\WINDOWS\\System32\\wsl.exe\"\"",

If you're using VSCode, you can resolve this by using the following settings:

  "terminal.integrated.shell.windows": "C:\\WINDOWS\\System32\\wsl.exe",
  "terminal.integrated.env.windows": {
    "PATH": ""
  },

The first one actually worked for me, although I applied a minor tweak to the code of the first example. This is what I put into my profile settings in Windows Terminal for use with WSL Ubuntu:

        // Disabled as workaround for current path length bug; replaced with "commandline" property; when bug is fixed, enable the below line and disable the other.
        // "source": "Windows.Terminal.Wsl",
        "commandline": "cmd.exe /C \"set PATH=C:\\WINDOWS\\System32;C:\\WINDOWS;C:\\;C:\\Program Files;C:\\Program Files (x86) && \"C:\\WINDOWS\\System32\\wsl.exe\"\"",

That worked and it also allows me to run Windows executables from the Linux command line.

@ertwro
Copy link

ertwro commented Jul 27, 2020

I've reduced the path characters to 456, I might break things in a really bad way if I keep trying to make it short and still no success. When is the next build coming out?

@bityob
Copy link
Author

bityob commented Jul 27, 2020

@ertwro

Did you try the python script above to check the limit in your machine?

I guess that empty path is working for you, right?

@ertwro
Copy link

ertwro commented Jul 27, 2020

@bityob Yes, empty path is working. The limit using the python script and lowering the minimum variable is 760-767. Unfortunately that is not true. The real limit is somewhere around 250 in my case.

@andreluizgit
Copy link

andreluizgit commented Jul 27, 2020

Even with PATH cleared (set PATH =), I can NOT run wsl:
"The I/O operation has been aborted because of either a thread exit or an application request" BUG

@bityob
Copy link
Author

bityob commented Jul 27, 2020

@andreluizgit

Even with PATH cleared (set PATH =), I can NOT run wsl:
"The I/O operation has been aborted because of either a thread exit or an application request" BUG

set PATH=
C:\Windows\System32\wsl.exe

This doesn't work for you? (from cmd)

@dcharlespyle
Copy link

I've reduced the path characters to 456, I might break things in a really bad way if I keep trying to make it short and still no success. When is the next build coming out?

I hope you weren't shortening your actual path environment. That could be bad for some programs. If you are doing it in the command line, however, it's only temporary for that session. So there is nothing to risk if doing it that way.

@dcharlespyle
Copy link

Even with PATH cleared (set PATH =), I can NOT run wsl:
"The I/O operation has been aborted because of either a thread exit or an application request" BUG

If you actually put a space between PATH and the = sign (like you did above), that probably is why it didn't work for you. There should be no space between the words PATH and =. It should be set PATH=.

@iccfish
Copy link

iccfish commented Jul 28, 2020

Even with PATH cleared (set PATH =), I can NOT run wsl:
"The I/O operation has been aborted because of either a thread exit or an application request" BUG

If you actually put a space between PATH and the = sign (like you did above), that probably is why it didn't work for you. There should be no space between the words PATH and =. It should be set PATH=.

I'm also can not run wsl even set PATH to empty.

@dcharlespyle
Copy link

Even with PATH cleared (set PATH =), I can NOT run wsl:
"The I/O operation has been aborted because of either a thread exit or an application request" BUG

If you actually put a space between PATH and the = sign (like you did above), that probably is why it didn't work for you. There should be no space between the words PATH and =. It should be set PATH=.

I'm also can not run wsl even set PATH to empty.

Could you post exactly what command line string it is that you are using to try to start and use WSL? And, where you are trying to launch WSL from?

@iccfish
Copy link

iccfish commented Jul 28, 2020

If you actually put a space between PATH and the = sign (like you did above), that probably is why it didn't work for you. There should be no space between the words PATH and =. It should be set PATH=.

I'm also can not run wsl even set PATH to empty.

Could you post exactly what command line string it is that you are using to try to start and use WSL? And, where you are trying to launch WSL from?

I've tried in many ways, such as:

  • launch from cmd
  • launch from powershell
  • launch from Windows Termial

And I've tried many ways to launch:

  • modify Windows Termial profile as you suggested previously
  • launch wsl after set PATH=
  • shorten my PATH environment from 1k+ chars to <800byte

None of them works for me 😭
So at last I rollback my laptop, cause I uses WSL often.
And I leave my PC on 20175 for testing.

@dcharlespyle
Copy link

@dcharlespyle thanks for your patience.
The command line or profile I use copy & paste, not typing by hand 😅

Command line. Easier to make sure the window is on top, and then hit Alt + PrintScr (or, "Print Screen"). Then, paste the screen shot into your comment. No need for typing or copy-pasting, if you do that. :-)

Here you go .... 😄

image

Wow. Not sure what is going on there, to be honest. But I am reasonably sure your system has more problems than just with WSL. This time, run the command cmd /c "set path=C:\WINDOWS\System32;C:\WINDOWS; && wsl" and post a screenshot. I want to see if a null path also will cause an issue with WSL on a system running in another language, before looking at other things.

@henris42
Copy link

Hi, same issue here. The set PATH= workaround works.
Any instruction on how to do workaround with the Windows Terminal? Running under CMD messes up everything. I also need to have the VS Code "code" command working, or this will be total pain.

@dcharlespyle
Copy link

dcharlespyle commented Jul 28, 2020

Hi, same issue here. The set PATH= workaround works.
Any instruction on how to do workaround with the Windows Terminal? Running under CMD messes up everything. I also need to have the VS Code "code" command working, or this will be total pain.

Open up your Terminal settings.json file in the Run dialog:

notepad %USERPROFILE%\AppData\Local\Packages\Microsoft.WindowsTerminal_8wekyb3d8bbwe\LocalState\settings.json

Here is what I put into mine to workaround the problem in Windows Terminal:

        "name": "Ubuntu",
        // Disabled as workaround for current path length bug; replaced with "commandline" property; when bug is fixed, enable the below line and disable the other.
        // "source": "Windows.Terminal.Wsl",
        "commandline": "cmd.exe /C \"set PATH=C:\\WINDOWS\\System32;C:\\WINDOWS;C:\\;C:\\Program Files;C:\\Program Files (x86) && \"C:\\WINDOWS\\System32\\wsl.exe\"\"",

Each property must be on its own line, and must end with a comma, and you must have the proper number of escape characters, as above. Okay, installed VS Code and tried out the code that was given above. I modified it to agree with my current situation.

Open (or create) the following by executing the following command:
notepad %USERPROFILE%\AppData\Roaming\Code\User\settings.json

Paste the following into the file to adjust WSL:

{  "terminal.integrated.shell.windows": "C:\\WINDOWS\\System32\\wsl.exe",
    "terminal.integrated.env.windows": {
      "PATH": "C:\\WINDOWS\\System32;C:\\WINDOWS;C:\\;"
    },
}

Save the file. So far, whenever I open WSL as the default Terminal in VS Code, I get the following:

image

@henris42
Copy link

henris42 commented Jul 28, 2020

Hi,

that works. "code" command is in user app data, So my path line is now like:
"commandline": "cmd.exe /C \"set PATH=C:\\WINDOWS\\System32;C:\\WINDOWS;C:\\;C:\\Program Files;C:\\Program Files (x86);C:\\Users\\MyUserName\\AppData\\Local\\Programs\\Microsoft VS Code\\bin&& \"C:\\WINDOWS\\System32\\wsl.exe\"\""

It is indeed picky on correct formatting. You don't need comma at the end of on the last entry, but there may not be space after \bin! I tried %USERPROFILE%, that did not resolve to anything useful.

@dcharlespyle
Copy link

Seems to be another issue with WSL that may or may not be related to the above. WSL isn't converting spaces in paths properly. Seeing the following in BASH:

$PATH
-bash: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/WINDOWS/System32:/mnt/c/WINDOWS:/mnt/c/:/mnt/c/Program: No such file or directory

Escaping is properly set in the path command in the settings.json file, according to VS Code and Visual Studio 2019. No errors reported but WSL truncates the path when converting it and then BASH reports "No such file or directory."

@henris42
Copy link

Hmm that thing I submited above works for me:
hs@simu:/mnt/c/Users/Henri$ echo $PATH /home/hs/.cargo/bin:/home/hs/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/WINDOWS/System32:/mnt/c/WINDOWS:/mnt/c/:/mnt/c/Program Files:/mnt/c/Program Files (x86):/mnt/c/Users/Henri/AppData/Local/Programs/Microsoft VS Code/bin:/snap/bin:/home/hs/.cargo/bin:/home/hs/.local/bin:/usr/local/bin

The spaces are as they should, and "code" command opens VS Code as expected.

@dcharlespyle
Copy link

dcharlespyle commented Jul 28, 2020

Things just keep getting more bizarre! Doesn't matter where I put the double ampersand operator. I still get a path that is truncated, and my code is identical to yours with the exception of the extra ones you appended to yours. But we get two different results in WSL. Really strange! So, I added spaces as Unicode escape sequences and I now see the entire path as set.

@henris42
Copy link

Do you use WSL1 or WSL2? I use WSL1.

@dcharlespyle
Copy link

dcharlespyle commented Jul 28, 2020

Do you use WSL1 or WSL2? I use WSL1.

I also am using WSL1. WSL2 is too buggy for my purposes at the moment (until they make improvements in networking interoperability).

wsl --list -v
  NAME      STATE           VERSION
* Ubuntu    Running         1

@henris42
Copy link

@dcharlespyle that kinda sounds like a locale issue, not WSL related. Might be worth a ticket with Windows Terminal folks?

@dcharlespyle
Copy link

@dcharlespyle that kinda sounds like a locale issue, not WSL related. Might be worth a ticket with Windows Terminal folks?

I thought that this was a distinct possibility. However, on further tinkering around, it isn't just in Windows Terminal. I found the same issue running direct through command line using cmd, and also in PowerShell. It definitely seems to be something with WSL, and not isolated to Windows Terminal. Not quite sure how or where to narrow it down at this moment, but hoping to figure it out sooner than later. They'll probably have it fixed before I do, though. :-)

@iccfish
Copy link

iccfish commented Jul 29, 2020

@dcharlespyle thanks for your patience.
The command line or profile I use copy & paste, not typing by hand 😅

Command line. Easier to make sure the window is on top, and then hit Alt + PrintScr (or, "Print Screen"). Then, paste the screen shot into your comment. No need for typing or copy-pasting, if you do that. :-)

Here you go .... 😄
image

Wow. Not sure what is going on there, to be honest. But I am reasonably sure your system has more problems than just with WSL. This time, run the command cmd /c "set path=C:\WINDOWS\System32;C:\WINDOWS; && wsl" and post a screenshot. I want to see if a null path also will cause an issue with WSL on a system running in another language, before looking at other things.

My WSL was OK when I using Win10 Insider Preview 20170(Intel) on laptop & 20160 on my PC(AMD). But when I upgrade them to 20175 last week, WSL1 on my laptop and PC all stop working. Cause I use my laptop often, so I rollback it to win10 20170, WSL1 works again.

And set path to C:\WINDOWS\System32;C:\WINDOWS; I've tried before, same situation, not working. I can not provide a screenshot right now.

@dcharlespyle
Copy link

@dcharlespyle thanks for your patience.
The command line or profile I use copy & paste, not typing by hand 😅

Command line. Easier to make sure the window is on top, and then hit Alt + PrintScr (or, "Print Screen"). Then, paste the screen shot into your comment. No need for typing or copy-pasting, if you do that. :-)

Here you go .... 😄
image

Wow. Not sure what is going on there, to be honest. But I am reasonably sure your system has more problems than just with WSL. This time, run the command cmd /c "set path=C:\WINDOWS\System32;C:\WINDOWS; && wsl" and post a screenshot. I want to see if a null path also will cause an issue with WSL on a system running in another language, before looking at other things.

My WSL was OK when I using Win10 Insider Preview 20170(Intel) on laptop & 20160 on my PC(AMD). But when I upgrade them to 20175 last week, WSL1 on my laptop and PC all stop working. Cause I use my laptop often, so I rollback it to win10 20170, WSL1 works again.

And set path to C:\WINDOWS\System32;C:\WINDOWS; I've tried before, same situation, not working. I can not provide a screenshot right now.

That's just it, though. A lot of WSL1 users are having problems with WSL in build 20175 (including myself), but your system on that build was a special case because nothing worked with WSL, not even the workarounds that worked around the issue for everybody else. That was my point. What is working for everybody else, however, didn't work for your machine--at all. Something in 20175 breaks WSL, but in your case even more was wrong with that build on your computer.

@henris42
Copy link

@dcharlespyle I'm still wondering on that unicode. Do you have beta UTF-8 setting on? Settings-Region-Additional Settings- Administrative-Change System Locale. There is a tick for beta UTF-8. Mine is off.

@dcharlespyle
Copy link

@dcharlespyle I'm still wondering on that unicode. Do you have beta UTF-8 setting on? Settings-Region-Additional Settings- Administrative-Change System Locale. There is a tick for beta UTF-8. Mine is off.

Mine also is off.
image

@henris42
Copy link

and the locale in Ubuntu? I have LANG="C.UTF-8"
Yep this is mysterious.

@dcharlespyle
Copy link

dcharlespyle commented Jul 29, 2020

and the locale in Ubuntu? I have LANG="C.UTF-8"
Yep this is mysterious.

Here is what I get:

$ locale
LANG=C.UTF-8
LANGUAGE=
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=

@henris42
Copy link

henris42 commented Jul 29, 2020

I have 100% same.. I really think that this workaround might have revealed a new issue. Maybe worth a new ticket.

@dcharlespyle
Copy link

I agree that this is mysterious. Another fellow who has posted here has had not one workaround work for him on build 20175. That is when my problems began, but I am holding out from rolling back this time around. I almost rolled back when I started noticing the problem with WSL on this build.

@carolinaisslaying
Copy link

All of the workarounds above used to work for me, but now they do not.

@henris42
Copy link

I don't know how to make a poll here, but are the effected all WSL1, or is WSL2 the same thing?
Myself I cannot use WSL2 as I need VMWare.

@iccfish
Copy link

iccfish commented Jul 29, 2020

@dcharlespyle FYI, WSL2 works ok on my 20175. I just not want to use it, it doesn't make sense to me

@blumu
Copy link

blumu commented Jul 29, 2020

Unregistering (using wsl command-line) and reinstalling Ubuntu WSL from the app store fixed the problem for me. Caveat: I had forgotten that unregistering a WSL distrib completely deletes the file system so I lost all my files in the process, ouch! It would have been wiser for me to first backup the file system using wsl --export and then reimport it using wsl --import or make a copy of the .vhdx file.

@dcharlespyle
Copy link

Unregistering (using wsl command-line) and reinstalling Ubuntu WSL from the app store fixed the problem for me. Caveat: I had forgotten that unregistering a WSL distrib completely deletes the file system so I lost all my files in the process, ouch! It would have been wiser for me to first backup the file system using wsl --export and then reimport it using wsl --import or make a copy of the .vhdx file.

I have no .vhdx file that I am aware of. Went searching for it, too, but turned up nothing.

@microsoft microsoft locked and limited conversation to collaborators Jul 30, 2020
@benhillis
Copy link
Member

Use the path workaround, WSL2, or roll back to previous insider build. Fix is inbound.

@benhillis
Copy link
Member

benhillis commented Aug 12, 2020

/fixed 20190.

@ghost ghost closed this as completed Aug 12, 2020
@ghost ghost added fixedininsiderbuilds and removed fixinbound labels Aug 12, 2020
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests