ctests: How to protect against tests accidentally being ignored? i.e., ctest -N changing

Scott Kostyshak skostysh at lyx.org
Thu Nov 26 18:18:53 UTC 2020


On Thu, Nov 26, 2020 at 07:11:14PM +0100, Kornel Benko wrote:
> Am Thu, 26 Nov 2020 13:03:41 -0500
> schrieb Scott Kostyshak <skostysh at lyx.org>:
> 
> > On Thu, Nov 26, 2020 at 05:32:41PM +0000, José Abílio Matos wrote:
> > > On Thursday, November 26, 2020 5:30:10 PM WET Kornel Benko wrote:  
> > > > We could output (at configure time) how many tests are ignored.
> > > > 
> > > >         Kornel  
> > > 
> > > That seems a nice first step. :-)  
> > 
> > Good idea.
> > 
> > A further step might be to assert that the number of ignored tests
> > equals a saved number. Whenever we updated the ignored tests, we need to
> > update the number. i.e., if I add a regex to ignoredTests that I expect
> > will ignore 5 tests, I increase the number by 5.
> > 
> > What are your thoughts on that? The disadvantage to this workflow is
> > that if we do not remember to update the number, the CMake command would
> > fail. This could be annoying. I think it would be worth the annoyance
> > and that we would get used to remembering to update the number of
> > ignored tests, but I would understand if others disagreed.
> > 
> > Scott
> 
> Possible, and not even difficult. The saved value would reside in the 'Testing' directory.
> We could also accumulate this file with entries like date-stamp or commit.

Nice. I like this idea. Let's wait though to hear from Thibaut and
Günter. I'm curious about their thoughts.

Scott
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.lyx.org/pipermail/lyx-devel/attachments/20201126/05a7886c/attachment.asc>


More information about the lyx-devel mailing list