RFC: concurrent forks of graphics conversions

[Kornel Benko]

Am Fri, 23 Sep 2022 15:43:05 -0400
schrieb Scott Kostyshak <skostysh at lyx.org>:

> Pavel gave some helpful direction regarding where concurrency might be
> especially helpful: graphics conversion.
> 
> Attached is a work-in-progress patch, as well as an example file.
> 
> When testing with the attached document, remember to clear the cache in
> your user directory before opening LyX. For example, I do the following:
> 
>   rm -rf ~/.lyx/cache/* && lyx
> 
> The modified code is just about LyX's display, not about converters used
> for PDF export.
> 
> While testing, it works surprisingly smoothly and the improvement is
> quite noticeable. I'm surprised that there are no known *observable*
> problems I've encountered, although I still consider the patch hackish
> at its current state.
> 
> The concurrency design of the patch is not good. It was just the most
> minimal changes I could make to the current code to get a simple
> prototype. I have not worked on thread safety. To do that I need to
> study more about the signals mechanism that's being used.
> 
> I also need to test problems like what happens if a process exits with
> error, what if it hangs, etc.
> 
> The reason I send this preliminary patch in is to (1) see if there is at
> least preliminary support for this kind of code change; (2) see if
> anyone has intuition for whether I am on the right track (i.e., is this
> the right place in the code to introduce this functionality?); (3) any
> thoughts on how I can stress-test this patch, and future patches?; (4)
> what should the default be for number of concurrent processes forked and
> what interface could I provide to change the default? e.g., default to
> current master behavior and allow for a hidden preference?
> 
> Regarding (3), how can I force a bunch of conversions that take a long
> time? Maybe a svg that is very small (or scalled down?) but is a large
> file size? Currently these conversions are all quick, so quick that the
> bars on my 'htop' don't even dance.
> 
> Thanks,
> Scott

Works nice. I tried to make multiple (== same) images to stress the concurrent handling,
but lyx resisted and did not crash :)

It may be too early to get it for the release, OTOH it is huge improvement at lyx-start.
We could limit the number of processes to #cores - 1, if the #cores > 1.

	Kornel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 488 bytes
Desc: Digitale Signatur von OpenPGP
URL: <http://lists.lyx.org/pipermail/lyx-devel/attachments/20220924/71aca8b6/attachment.asc>