Build script to help bisecting by cherry-picking compiler fixes

[Scott Kostyshak]

On Tue, Nov 19, 2024 at 08:54:50AM +0100, Kornel Benko wrote:
> Am Mon, 18 Nov 2024 11:52:22 +0100
> schrieb Pavel Sanda <sanda at lyx.org>:
> 
> > 2) Demands:
> > a) How long takes one test? (Instead of "many hours" which can drastically change
> >    with different cpu's can you estimate ratio to e.g. total compilation time?)
> > b) Can your test work in parallel manner for 4 or 8 cores?
> 
> I am running the tests mostly as 24 jobs parallel (I have 8 cpu's with 2 cores each).
> The individual export tests may vary between < 1 second and > 1800 seconds.
> Total time for export tests here
>  $ ctest -j24 --timeout 1800 -L export -E '(xhtml|lyx..)$'
> is about 1 hour for 5608 tests.
> 
> > I think if we could get with an hour or half an hour of compute for parallel regime
> > that's negligible enough and we might not even ask.
> 
> We could limit the tests so that we could alternatively test only html exports, or pdf2
> exports or docbook5 or ...

True. Alternating could be good idea. My first thought was we should
run, e.g., all the systemF tests one day and all of the other tests a
different day. But perhaps it's better to have a more balanced split,
like running half of the systemF tests and half of the other tests one
day, and the rest the next day. This way, if there is a problem with a
recent lyx2lyx commit, then it will likely be picked up with either set
of tests.

Alternating might lead to a bit of confusion though. e.g., one day the
tests give errors; and maybe the next day the (other set) tests give no
errors so it is tempting to think the problem has been fixed.

Scott
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.lyx.org/pipermail/lyx-devel/attachments/20241119/f7ee8561/attachment.sig>