Making Paragraph::latex() thread safe?

[Scott Kostyshak]

If you're not curious about the background, skip to the part below starting with "The main problem". In short, I want to learn more about some code to try to make it thread-safe.

The last few days I've been experimenting with using parallelization in part of LyX's code.

I'm currently using a for_each loop inside of latexParagraphs() to parallelize over TeXOnePar(). I'm shocked that it actually runs in parallel. The reason I'm shocked is because I don't know what I'm doing. My current code uses a lot of ugly hacks and is not usable for anything other than curiosity. If you really want to see it, I can share it. But I really wouldn't recommend it :)

I used the hacks just to get an idea of whether such parallelization would actually lead to a speed-up. It does. For example, on an 8-core machine, the document files (user guide, etc.) export to .tex about 4x faster. This isn't that helpful from a user perspective, because the main bottle-neck is compiling to PDF, not the LaTeX generation. But it's still cool to see that it actually does have a real effect. I think the speedup will be even faster if we could make Paragraph::latex() thread-safe. Currently I protect part of the code with a mutex, which I ask a question about below.

I don't think my work is likely to be ready for upstream for a while, and even then I'm not sure what others would think. In the short-run I'm motivated by a personal use case, which is that I have a document with 20 children and each of those children takes considerable time to be exported to TeX because they each use knitr. I would like to do those exports in parallel. I don't imagine this is a common use case.

The main problem I currently face is the following code inside Paragraph::latex():

d->latexInset(bparams, os, rp, running_font,
                basefont, real_outerfont, open_font,
                runningChange, style, i, column, fontswitch_inset,
                closeLanguage, (lang_switched_at_inset || langClosed));

If I protect it with a mutex, then things work much better. But that's exactly the code that I need to run in parallel for my knitr child documents to export in parallel. Could someone explain intuitively why this code is not thread-safe? Is there any hope of making it thread-safe without major surgery?

In addition to that code, there's other parts that are not thread-safe. I know this because when I export to LaTeX, LyX crashes once every 5 times. I haven't put much effort into finding the other parts, but I think I could make progress if we could address the d->latexInset() issue I ask about above.

Thanks,
Scott
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.lyx.org/pipermail/lyx-devel/attachments/20220829/e6dbbdbf/attachment-0001.asc>