Investigating strange PWD behaviour on ksh93
While trying out illumos, I ran into a problem that boiled down
to “sh sometimes doesn’t update PWD when it starts”.
#!/usr/bin/python3
from subprocess import run
command = ["sh", "-c", "echo $PWD"]
# The parent script is run from /export/home/user/foo
run(command) #=> /export/home/user/foo (ok)
run(command, cwd="/export/home/user") #=> /export/home/user (ok)
run(command, cwd="/") #=> / (ok)
run(command, cwd="/export") #=> /export/home/user/foo (wrong)
run(command, cwd="/tmp") #=> /export/home/user/foo (wrong)
The pwd command works as expected, and Bash doesn’t seem to be affected by the issue.
run(["sh" "-c", "pwd"], cwd="/tmp") #=> /tmp (ok)
run(["bash" "-c", "echo $PWD"], cwd="/tmp") #=> /tmp (ok)
§Is this a bug?
The crux of the problem is that, in all of the failing cases, the current working directory is changed in a way
that does not change PWD. The relevant part of POSIX
says:
If a value for PWD is passed to the shell in the environment when it is executed, the value is an absolute pathname of the current working directory that is no longer than {PATH_MAX} bytes including the terminating null byte, and the value does not contain any components that are dot or dot-dot, then the shell shall set PWD to the value from the environment. Otherwise, if a value for PWD is passed to the shell in the environment when it is executed, the value is an absolute pathname of the current working directory, and the value does not contain any components that are dot or dot-dot, then it is unspecified whether the shell sets PWD to the value from the environment or sets PWD to the pathname that would be output by pwd -P. Otherwise, the sh utility sets PWD to the pathname that would be output by pwd -P.
That’s… yeah, I’ll try to unpack that. There are three sentences here, in the form “If (A) then {AA}. Else if (B) then {BB}. Else {CC}.” Both A and B are false because “the value is an absolute pathname of the current working directory”, which occurs on both clauses, is false in our case. This leaves us with {CC}:
the sh utility sets PWD to the pathname that would be output by pwd -P.
In simpler terms, when sh launches, if PWD is not equivalent to the current working directory, it must
be set to the current working directory with symlinks resolved.
illumos sh fails to do this unless the current directory is / or the user’s home directory, so I
believe this is indeed a bug. Interestingly enough, PWD does get updated after a single
pwd call.
§Minimal reproducer
The code at the beginning of this post shows one way to demonstrate the problem: changing the working directory
without changing PWD. However, the same issue can be reproduced by doing it the other way around:
changing PWD without actually changing the working directory.
cd /tmp
PWD=/dev sh -c 'echo $PWD'
- Expected result:
/tmp. - Actual result:
/dev.
This is a lot simpler and is what I use in my subsequent tests.
§Which shells and OSes are affected?
In illumos, sh is a symlink to ksh93 (KornShell 93). ksh93 has a weird history and there are now multiple variants
of it, but KSH_VERSION shows version 93u+, which greatly narrows down the options.
AT&T ksh93u+ is the last released version of the original KornShell line published by AT&T. A bit of searching led me to the illumos fork, vendored into the main illumos repository at contrib/ast.
To make sure that this is not an illumos-only problem, I also tried reproducing it on Linux. Attempting to build ksh93u+ on my Linux machine led to compile errors that didn’t look trivial to fix. Eventually I had to run Debian 11 (Bullseye), the last Debian version that packaged ksh93u+, and—yes—the problem is reproducible there.
I also tested some ksh93 forks (ksh2020 and ksh93u+m), as well as other KornShell implementations (mksh and oksh). None of these exhibit the problem.
Out of curiosity, I tried the same on a lot of non-KornShell Unix shells and found one where the problem is reproducible: OSH (from the oils-for-unix project).
§What now?
AT&T ksh93 is not maintained anymore so I don’t think reporting a bug there will be useful. I've filed a bug report on illumos but I don’t expect them to have the interest or manpower to fix this niche problem.
As for OSH, I found an open bug report that described a symptom of the same problem, so I added a comment explaining the likely underlying bug. As it turned out, it had been fixed in the development version not long before my comment.
(This post was mostly written in May 2025, when I did all this investigation and bug reporting; I only cleaned up and finished it nine months later.)