Layout file format change proposal

[Lorenzo Bertini]

Il giorno gio 28 mar 2024 alle ore 11:40 José Matos
<jaomatos at gmail.com> ha scritto:
>
> On Wed, 2024-03-27 at 11:02 -0400, Scott Kostyshak wrote:
> > Cool idea, Lorenzo! There have been some discussions in the past on
> > using standard formats, even for the .lyx file itself. I can see the
> > advantages of that but I don't know what the cost of converting to
> > them
> > is.
> >
> > Scott
>
> Typical answer to this:
> https://xkcd.com/927/
>
> :-)
>
> It makes sense for us to use a standard format.
> Some of the questions then are:
>  * which one is chosen (what is the most appropriate for our needs)?
>  * what the are advantages and drawbacks from this change?
>  * what is the transition plan?
>  * what is the effort required for this transition?
>
> Best regards,
> --
> José Abílio
> --

Ah, a classic xkcd. For once, however, the talk is about "removing"
one standard :)

* Which one to choose?
See research below.

* The advantages and drawbacks
As already mentioned, using a more common format means
  - people generally more familiar with the format
  - having more tools able to produce them, easing the efforts of
publishers trying to provide layouts
  - having parsers already available, for LyX and for people wanting
to manipulate them externally
The drawbacks that i can think of are
  - migration always takes developer effort
  - migration can take user effort too, to adapt to the new format
(this can be greatly mitigated by which format is chosen)

* The transition plan:
The new format is introduced *alongside* the old format. The LyX
version that brings the format will have all layouts converted, but
will accept the old format too. This "transition period" can last
indefinitely.

* The efforts required for the transition
We need to:
  - write or choose a parser for the new format that maps a file to a
LyX's params
  - write a converter from the old format to the new one
  - convert all the files to the new formats and test them

Here is some info I dug up regarding configuration formats. For pros
and cons I tried to limit them to one single aspect where they excel
or fail
- JSON:
  . pros: most widely supported
  . cons: not concise or readable, very distant from .layout
- TOML:
  . pros: very easy to read and write, even for humans
  . cons: not widespread, distant from .layout
- YAML:
  . pros: very easy to read and write, very close to .layout
  . cons: indentation is prone to syntax errors
- XML:
  . pros: the one and only, supports validation through schemas,
widely supported
  . cons: hard to read and write, very distant from .layout

I asked a chatbot to convert LyX's following layout extract into those formats:

LYX'S LAYOUT

InsetLayout Marginal
    LabelString           Margin
    LatexType             command
    Font
      Size                Small
    EndFont
    LabelFont
      Color               marginlabel
      Size                Small
    EndFont
    HTMLStyle
        div.marginal {
            border: 2px solid black;
            font-style: normal;
        }
    EndHTMLStyle
End

JSON:

{
  "InsetLayout": {
    "LabelString": "Margin",
    "LatexType": "command",
    "Font": {
      "Size": "Small"
    },
    "LabelFont": {
      "Color": "marginlabel",
      "Size": "Small"
    },
    "HTMLStyle": {
      "div.marginal": {
        "border": "2px solid black",
        "font-style": "normal"
      }
    }
  }
}

TOML:

[InsetLayout]
  LabelString = "Margin"
  LatexType = "command"

  [InsetLayout.Font]
    Size = "Small"

  [InsetLayout.LabelFont]
    Color = "marginlabel"
    Size = "Small"

  [InsetLayout.HTMLStyle.div.marginal]
    border = "2px solid black"
    font-style = "normal"

YAML:

InsetLayout:
  LabelString: Margin
  LatexType: command
  Font:
    Size: Small
  LabelFont:
    Color: marginlabel
    Size: Small
  HTMLStyle:
    div.marginal:
      border: 2px solid black
      font-style: normal

XML:

<InsetLayout>
  <LabelString>Margin</LabelString>
  <LatexType>command</LatexType>
  <Font>
    <Size>Small</Size>
  </Font>
  <LabelFont>
    <Color>marginlabel</Color>
    <Size>Small</Size>
  </LabelFont>
  <HTMLStyle>
    <div.marginal>
      <border>2px solid black</border>
      <font-style>normal</font-style>
    </div.marginal>
  </HTMLStyle>
</InsetLayout>

Obviously there are errors because it considered the CSS inside it as
part of the format. But I hope this gives you an impression of what
each format can give us. Personally, and for what it counts (very
little), my pick would the the closest to the original format, and
that would be YAML.

Let me know what you think.
Lorenzo