RFC: concurrent forks of graphics conversions

[Scott Kostyshak]

Pavel gave some helpful direction regarding where concurrency might be
especially helpful: graphics conversion.

Attached is a work-in-progress patch, as well as an example file.

When testing with the attached document, remember to clear the cache in
your user directory before opening LyX. For example, I do the following:

  rm -rf ~/.lyx/cache/* && lyx

The modified code is just about LyX's display, not about converters used
for PDF export.

While testing, it works surprisingly smoothly and the improvement is
quite noticeable. I'm surprised that there are no known *observable*
problems I've encountered, although I still consider the patch hackish
at its current state.

The concurrency design of the patch is not good. It was just the most
minimal changes I could make to the current code to get a simple
prototype. I have not worked on thread safety. To do that I need to
study more about the signals mechanism that's being used.

I also need to test problems like what happens if a process exits with
error, what if it hangs, etc.

The reason I send this preliminary patch in is to (1) see if there is at
least preliminary support for this kind of code change; (2) see if
anyone has intuition for whether I am on the right track (i.e., is this
the right place in the code to introduce this functionality?); (3) any
thoughts on how I can stress-test this patch, and future patches?; (4)
what should the default be for number of concurrent processes forked and
what interface could I provide to change the default? e.g., default to
current master behavior and allow for a hidden preference?

Regarding (3), how can I force a bunch of conversions that take a long
time? Maybe a svg that is very small (or scalled down?) but is a large
file size? Currently these conversions are all quick, so quick that the
bars on my 'htop' don't even dance.

Thanks,
Scott
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Working-on-concurrent-forks.patch
Type: text/x-diff
Size: 2646 bytes
Desc: not available
URL: <http://lists.lyx.org/pipermail/lyx-devel/attachments/20220923/c647ba22/attachment-0001.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graphics-conversions.lyx
Type: application/x-lyx
Size: 21928 bytes
Desc: not available
URL: <http://lists.lyx.org/pipermail/lyx-devel/attachments/20220923/c647ba22/attachment-0001.lyx>