2.4 alpha2 failing on windows 10

Fri Feb 12 15:01:19 UTC 2021

On Fri, 12 Feb 2021 at 15:50, Thibaut Cuvelier <tcuvelier at lyx.org> wrote:

> On Fri, 12 Feb 2021 at 14:36, Thibaut Cuvelier <tcuvelier at lyx.org> wrote:
>
>> On Fri, 12 Feb 2021 at 06:46, Richard Kimberly Heck <rikiheck at lyx.org>
>> wrote:
>>
>>> On 2/11/21 11:19 PM, Thibaut Cuvelier wrote:
>>>
>>> On Thu, 11 Feb 2021 at 20:52, Thibaut Cuvelier <tcuvelier at lyx.org>
>>> wrote:
>>>
>>>> On Thu, 11 Feb 2021 at 20:05, José Abílio Matos <jamatos at lyx.org>
>>>> wrote:
>>>>
>>>>> On Thursday, February 11, 2021 4:02:19 PM WET Yu Jin wrote:
>>>>> > Will be pointless imo, it's not the script's fault that python is
>>>>> not found.
>>>>> > The fault is to be searched in the part of code which adds the
>>>>> prefix path
>>>>> > to the environment, which part would that be? After all when
>>>>> installing,
>>>>> > the installer adds the prefix path to its environment too on runtime
>>>>> while
>>>>> > installing and then calls the script, it is successful then.
>>>>>
>>>>> Does Tools->Reconfigure works after adding the python path in the
>>>>> environment
>>>>> variable?
>>>>>
>>>>> I expect it to work. If it works that means that the configure.py
>>>>> changes are
>>>>> not the culprit.
>>>>>
>>>>
>>>> In my case, LyX can reconfigure once its PATH includes Python.
>>>>
>>>
>>> I just checked that the situation is quite weird: in Tools > Preferences
>>> > Paths, I have "C:\Program Files\MiKTeX
>>> 2.9\miktex\bin\x64;$LyXDir\bin;$LyXDir\Python;$LyXDir\Python\Lib;$LyXDir\imagemagick;$LyXDir\ghostscript\bin",
>>> meaning that LyX should really consider its built-in version of Python, but
>>> (re)configure always fails if Python is not in the environment variable
>>> PATH does not contain Python before LyX starts.
>>>
>>> Does it fail if you try to reconfigure once LyX has launched? If so, can
>>> you please open View> Messages, then activate "Files Used By LyX", and
>>> reconfigure? I see, at the top
>>>
>>> 00:43:30.779: python3 -tt "/usr/local/share/lyx/configure.py"
>>> --binary-dir="/usr/local/bin/"
>>>
>>> That is the command LyX is using to try to reconfigure.
>>>
>>> Maybe we need also to check what the environment is at that point?
>>>
>>
>>  Here is what I get (a full LyX session with two calls to reconfigure):
>>
>> 14:30:52.923: Running configure...
>>>
>>> 14:30:56.919: python -tt "C:/Program Files/LyX
>>> 2.4/Resources/configure.py" --binary-dir="C:/Program Files/LyX 2.4/bin/"
>>>
>>> 14:30:57.187: Python was not found; run without arguments to install
>>> from the Microsoft Store, or disable this shortcut from Settings > Manage
>>> App Execution Aliases.
>>>
>>> 14:30:57.187: Reloading configuration...Manually looking for python in
>>> PATH ...
>>>
>>> Examining "C:/Program Files/MiKTeX 2.9/miktex/bin/x64/pythontex.exe"
>>>
>>> Examining
>>> "C:/Users/Thibaut/AppData/Local/Microsoft/WindowsApps/python.exe"
>>>
>>> Examining
>>> "C:/Users/Thibaut/AppData/Local/Microsoft/WindowsApps/python3.exe"
>>>
>>> Examining
>>> "C:/Users/Thibaut/AppData/Local/Microsoft/WindowsApps/python.exe"
>>>
>>> Examining
>>> "C:/Users/Thibaut/AppData/Local/Microsoft/WindowsApps/python3.exe"
>>>
>>> Warning: No python v2.x or 3.x binary found.
>>>
>>> LyX: reconfiguring user directory
>>>
>>> support\Systemcall.cpp (291): Systemcall: 'python -tt "C:/Program
>>> Files/LyX 2.4/Resources/configure.py" --binary-dir="C:/Program Files/LyX
>>> 2.4/bin/"' finished with exit code 9009
>>>
>>> LyX: Done!
>>>
>>> 14:31:06.641: (reconfigure)
>>>
>>> 14:31:21.627: Running configure...
>>>
>>> 14:31:21.766: python -tt "C:/Program Files/LyX
>>> 2.4/Resources/configure.py" --binary-dir="C:/Program Files/LyX 2.4/bin/"
>>>
>>> 14:31:22.099: Python was not found; run without arguments to install
>>> from the Microsoft Store, or disable this shortcut from Settings > Manage
>>> App Execution Aliases.
>>>
>>> 14:31:22.103: Reloading configuration...LyX: reconfiguring user directory
>>>
>>> support\Systemcall.cpp (291): Systemcall: 'python -tt "C:/Program
>>> Files/LyX 2.4/Resources/configure.py" --binary-dir="C:/Program Files/LyX
>>> 2.4/bin/"' finished with exit code 9009
>>>
>>> LyX: Done!
>>>
>>> 14:31:23.273: (reconfigure)
>>>
>>> 14:32:30.743: (dialog-toggle progress)
>>>
>>
>> It really seems like LyX is getting the "standard" python.exe that just
>> shows a message and returns an error instead of its own Python (or any
>> other one from the path).
>>
>
> I can still reproduce the problem with master: the major difference is
> that I can use a debugger on that build. Digging in the call stack, it
> looks like the PATH environment variable is filled with strange things: the
> letter e is being replaced by $ for the part that LyX prepends! In
> environment.cpp, getEnv("PATH") returns the following string (truncated),
> when called from Tools > Reconfigure (find_python_binary in os.cpp):
>
> "C:\\Program Fil$s\\MiKT$X
>> 2.9\\mikt$x\\bin\\x64;D:\\Thibaut\\LyX\\cmak$-build-d$bug-visual-studio\\bin;D:\\Thibaut\\LyX\\cmak$-build-d$bug-visual-studio\\Python;D:\\Thibaut\\LyX\\cmak$-build-d$bug-visual-studio\\Python\\Lib;D:\\Thibaut\\LyX\\cmak$-build-d$bug-visual-studio\\imag$magick;D:\\Thibaut\\LyX\\cmak$-build-d$bug-visual-studio\\ghostscript\\bin;C:\\Program
>> Files\\Common
>> Files\\Oracle\\Java\\javapath;C:\\gurobi911\\win64\\bin;C:\\Program
>> Files\\IBM\\ILOG\\CPLEX_Studio201\\opl\\bin\\x64_win64 […]
>>
>
> Only the folders that LyX prepends have Es replaced. In a normal shell or
> other programs run in my debugger, there is no such thing.
>
> Digging more into it, the first call to getEnv("PATH") returns a perfectly
> normal value, when called from prependEnvPath in environment.cpp. It's the
> loop in prependEnvPath that does garbage. In my LyX folder in my user
> folder, the preferences file has the right line for the path prefix:
>
> \path_prefix "C:/Program Files/MiKTeX
>> 2.9/miktex/bin/x64;$LyXDir/bin;$LyXDir/Python;$LyXDir/Python/Lib;$LyXDir/imagemagick;$LyXDir/ghostscript/bin"
>>
>
> I therefore suspect that replaceEnvironmentPath in filetools.cpp is the
> source of the problem.
>
>    - I believe this function is wrong on Windows, as environment
>    variables are supposed to be wrapped in %, like %PATH% (and, by the way,
>    environment-variable names are case-insensitive on Windows). (! is only
>    used in Batch files, this symbol should not be interpreted here.) Only the
>    paths that are understood by LyX are transformed (like LyXDir), not all
>    environment variables. Some software adds itself to the path using
>    environment variables, i.e. add something like "%CUDA_PATH%" in the PATH
>    environment variable that just refers to the CUDA_PATH environment variable
>    (set to some perfectly valid path).
>    - The line subst(result, '\e', '$') is the most likely culprit for
>    this error. Maybe some difference in implementation for regexes?
>
> Actually, it looks like \e is a GCC-only extension that is not
well-understood by MSVC. The attached patch solves the problem for me by
replacing \e by another nonprintable character.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lyx.org/pipermail/lyx-devel/attachments/20210212/f32e78a2/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: unnamed1.patch
Type: application/octet-stream
Size: 1022 bytes
Desc: not available
URL: <http://lists.lyx.org/pipermail/lyx-devel/attachments/20210212/f32e78a2/attachment.obj>