ctests: How to protect against tests accidentally being ignored? i.e., ctest -N changing

Kornel Benko kornel at lyx.org
Thu Nov 26 18:19:55 UTC 2020


Am Thu, 26 Nov 2020 19:11:14 +0100
schrieb Kornel Benko <kornel at lyx.org>:

> Am Thu, 26 Nov 2020 13:03:41 -0500
> schrieb Scott Kostyshak <skostysh at lyx.org>:
> 
> > On Thu, Nov 26, 2020 at 05:32:41PM +0000, José Abílio Matos wrote:  
> > > On Thursday, November 26, 2020 5:30:10 PM WET Kornel Benko wrote:    
> > > > We could output (at configure time) how many tests are ignored.
> > > > 
> > > >         Kornel    
> > > 
> > > That seems a nice first step. :-)    
> > 
> > Good idea.
> > 
> > A further step might be to assert that the number of ignored tests
> > equals a saved number. Whenever we updated the ignored tests, we need to
> > update the number. i.e., if I add a regex to ignoredTests that I expect
> > will ignore 5 tests, I increase the number by 5.
> > 
> > What are your thoughts on that? The disadvantage to this workflow is
> > that if we do not remember to update the number, the CMake command would
> > fail. This could be annoying. I think it would be worth the annoyance
> > and that we would get used to remembering to update the number of
> > ignored tests, but I would understand if others disagreed.
> > 
> > Scott  
> 
> Possible, and not even difficult. The saved value would reside in the 'Testing'
> directory. We could also accumulate this file with entries like date-stamp or commit.
> 
> 	Kornel

I misread your mail. What I meant was
1.) Protocol the date, commit, and number of ignored tests in a file
so we can always check when the count made a big step forward.

	Kornel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 488 bytes
Desc: Digitale Signatur von OpenPGP
URL: <http://lists.lyx.org/pipermail/lyx-devel/attachments/20201126/2daf2b64/attachment-0001.asc>


More information about the lyx-devel mailing list