lyx2lyx error: \begin{english} is inside a bold layout instead of outside

Tue Mar 3 19:12:11 UTC 2020

On Tue, Mar 03, 2020 at 06:42:24PM +0100, Kornel Benko wrote:
> Am Tue, 3 Mar 2020 16:32:08 -0000 (UTC)
> schrieb Guenter Milde <milde at users.sf.net>:
> 
> > Dear Scott, dear LyX developers,
> > 
> > thanks for the failure-report, analysis and patience.
> > I believe to have found the reason, it has been a long and tricky journey:
> > 
> > On 2020-03-02, Scott Kostyshak wrote:
> > > On Sun, Mar 01, 2020 at 09:43:52PM -0000, Guenter Milde wrote:  
> > >> On 2020-02-23, Scott Kostyshak wrote:  
> > >> > On Fri, Feb 21, 2020 at 11:26:08AM -0500, Scott Kostyshak wrote:  
> > >> >> The following tests are failing on master:  
> > 
> > >> >>   export/examples/ko/Welcome_lyx22 (Failed)
> > >> >>   export/examples/ko/Welcome_lyx23 (Failed)  
> > 
> > ...
> > 
> > >> >> This happens because the ERT "\begin{English}" is inside the bold
> > >> >> layout. Either we need to end it at the end of the bold layout, or we
> > >> >> need to move the "\begin{English}" before the bold starts.  
> > 
> > ...
> > 
> > >> > Bisect gives the following:  
> > 
> > >> >   0d49918c0c815ead4ab70128fd6677118bf08c1e is the first bad commit
> > >> >   commit 0d49918c0c815ead4ab70128fd6677118bf08c1e
> > >> >   Author: Günter Milde <milde at lyx.org>
> > >> >   Date:   Thu Jul 4 20:49:06 2019 +0200  
> > 
> > >> >       Refactor lyx2lyx for speed and simplicity.  
> > 
> > >> > Günter can you reproduce?  
> > 
> > >> I can reproduce the problem but not the bisect result:  
> > 
> > >> Converting an MWE with lyx2lyx from commit fc99ef2ab3e60b4 28.05.2019,
> > >> I get the same nesting mix-up and compilation error.  
> > 
> > ...
> > 
> > > When I use 0d49918c, compared to 0d49918c^, I get different lyx2lyx
> > > output. The compilation succeeds with 0d49918c^ and fails with 0d49918c.
> > > Attached is the diff. From what I understand from the commit message, it
> > > is not expected that 0d49918c changed the lyx2lyx output.  
> > 
> > Actually, my MWE used Russian (Petrine orthography) and/or Church
> > Slavonic instead of Korean. With these languages, the problem of a
> > misplaced ERT "\begin{English}" can be reproduced also with
> > earlier versions of master.
> > 
> > So the next try was with lib/examples/ko/Welcome.lyx:
> > 
> >   #> git checkout 0d49918c  
> > 
> >   #> lyx2lyx ../examples/ko/Welcome.lyx -V 2.3 > /tmp/Welcome.23-after.lyx  
> > 
> >   #> git checkout 0d49918c^  
> > 
> >   #> lyx2lyx ../examples/ko/Welcome.lyx -V 2.3 > /tmp/Welcome.23-before.lyx  
> > 
> > There was no difference between Welcome.23-after.lyx and
> > Welcome.23-before.lyx and both failed to compile with "missing glyphs" !?
> > 
> > It took some time and tests to realise, that the revert_language() function
> > was not called at all, LyX-2.3 used Babel as language package (with document
> > language "english" and without any language switch commands.
> > This means the font-setup for Polyglossia in the latex-preamble had no
> > effect leading to the missing glyphs.
> > 
> > 
> > It took some more time to find out that I can reproduce Scott's findings 
> > setting a higher version number:
> > 
> >   --- a/lib/examples/ko/Welcome.lyx
> >   +++ b/lib/examples/ko/Welcome.lyx
> >   @@ -1,5 +1,5 @@
> >    #LyX 2.4 created this file. For more info see https://www.lyx.org/
> >   -\lyxformat 575
> >   +\lyxformat 576
> >    \begin_document
> >    \begin_header
> > 
> > Now, the re-created "Welcome.23-after.lyx" and "Welcome.23-before.lyx"
> > differed and "Welcome.23-before.lyx" compiled fine while
> > "Welcome.23-after.lyx" failed due to the miss-placed ERT-language-switch.
> > 
> > So, indeed there is an unannounced change in lyx2lyx output after
> > 0d49918c. It went unnoticed and did not trigger a lyx2lyx error at this
> > time, because "ko/Welcome.lyx" was still an older file version and did not
> > trigger ``revert_language("korean", "", "korean")``.
> > 
> > 
> > But why does language-reversion with *bold text* at the start of a
> > paragraph lead to compilation failure with all languages except Korean?
> > 
> > * Korean is special, as it is "half-supported" by LyX for 8-bit TeX via
> >   CJK. (CJK supports Korean *script* but not Korean *language*.)
> > 
> > * When using LyX 2.3 with Korean and non-TeX fonts, it does not write
> >   Korean language tags/switches to the LaTeX source. Documents containing
> >   Korean text are compiled without error if a font containing required
> >   glyphs is defined.
> > 
> > * This kind of worked with the pre-0d49918c conversion: the reversion
> >   added preamble code to load Polyglossia, however LyX also loaded Babel.
> >   
> >   However, loading both, Babel and Polyglossia does not seem correct and
> >   indeed, a language switch in the middle of a paragraph leads to missing
> >   output or errors. 
> >   
> >   How to reproduce:
> >   
> >    * Convert ko/Welcome.lyx to 2.3 format with the pre-0d49918c lyx2lyx.
> >    
> >    * Open in LyX 2.3 - compilation works.
> >    
> >    * Open the code preview: there are \usepackage calls for Polyglossia and
> >      Babel!
> >    
> >    * Set a word in the English paragraph to a different language (German,
> >      say): This word will not appear in the output, because the third
> >      language is only set up for Babel (as global argument "ngerman") but
> >      not for Polyglossia!
> >      
> >    * Insert an English word in the Korean text part and set the language
> >      to English: Here, I get the error:
> >      
> >        ! LaTeX Error: Environment CJK undefined.
> >      
> >      because the enumeration starts with "\begin{CJK}{KS}{mj}" in the LaTeX
> >      source.
> > 
> > 
> > So, yes 0d49918c changed the lyx2lyx output removing an exception for
> > Korean which in some cases leads to compilation errors and in some cases
> > fixes compilation errors or missing output.
> 
> Wow!
> Nice investigation.

+1 thanks for the explanation.

> > I propose leaving it this way and gradually improve revert_language().

I'm fine with whatever you suggest since I don't understand these issues
well.

Scott
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.lyx.org/pipermail/lyx-devel/attachments/20200303/ab7870db/attachment-0001.asc>