Link error when compiling with -fsanitize=undefined

[Scott Kostyshak]

On Mon, Sep 20, 2021 at 06:30:14PM +0200, Kornel Benko wrote:
> Am Mon, 20 Sep 2021 11:48:37 +0200
> schrieb Kornel Benko <kornel at lyx.org>:
> 
> > Am Sun, 19 Sep 2021 20:19:15 -0400
> > schrieb Scott Kostyshak <skostysh at lyx.org>:
> > 
> > > On Sun, Sep 19, 2021 at 08:37:16PM +0200, Kornel Benko wrote:  
> > > > Am Sun, 19 Sep 2021 14:15:21 -0400
> > > > schrieb Scott Kostyshak <skostysh at lyx.org>:
> > > >     
> > > > > On Sun, Sep 19, 2021 at 02:00:33PM +0200, Kornel Benko wrote:    
> > > > > > Am Sun, 19 Sep 2021 07:45:19 -0400
> > > > > > schrieb Scott Kostyshak <skostysh at lyx.org>:
> > > > > >       
> > > > > > > > > > So inserting this include cured the compilation for me.          
> > > > > > > > > 
> > > > > > > > > That's great you figured it out and it doesn't add too much complexity to
> > > > > > > > > the code. Are you sure it ended up using the -fsanitize=undefined flag ?
> > > > > > > > > If you do
> > > > > > > > > 
> > > > > > > > >   ldd build-dir/bin/lyx | grep ubsan
> > > > > > > > > 
> > > > > > > > > Does it show up?
> > > > > > > > > 
> > > > > > > > > Scott        
> > > > > > > > 
> > > > > > > > 	libubsan.so.1 => /usr/lib/x86_64-linux-gnu/libubsan.so.1
> > > > > > > > (0x00007f49cdfd2000)        
> > > > > > > 
> > > > > > > Great, I will experiment next week with it.
> > > > > > > 
> > > > > > > Thank you,
> > > > > > > 
> > > > > > > Scott
> > > > > > >       
> > > > > > 
> > > > > > BTW, how got you the output showing errors? My lyx does not print any error
> > > > > > messages. (I mean:
> > > > > > 'ColorCode' /home/scott/lyxbuilds/master/repo/src/MetricsInfo.cpp:174:6: runtime
> > > > > > error: load of value 128, which is not a valid value for type
> > > > > > ...
> > > > > > )      
> > > > > 
> > > > > It seems that it is triggered with a lot of branches (or perhaps more generally
> > > > > any different kind of insets?). Attached is an LFUN sequence that reproduces it.
> > > > > Does it trigger the error for you?
> > > > > 
> > > > > Scott    
> > > > 
> > > > Not on empty document.    
> > > 
> > > For me it works on an empty document. I do use a default.lyx document (this is what
> > > saves when you click on "save class defaults"), but I can reproduce if I remove that
> > > template. I attach both MWEs; there's not much difference between them. For me, it is
> > > sufficient to just open them and I get the messages in the terminal.
> > > 
> > > Not sure if it's relevant, but perhaps it could be differences with GCC version:
> > > 
> > >   g++ --version
> > >   g++ (Ubuntu 10.3.0-1ubuntu1) 10.3.0
> > > 
> > > Scott  
> > 
> > My fail (Using LYX_DEBUG_SANITIZE too late). I get the messages now too.
> > 
> > BTW, I wonder, if if LYX_ASAN and LYX_DEBUG_SANITIZE are eventually exclusive?
> > 
> > 	Kornel
> 
> Some debugging lead to following observation:
> Each inserted branch add a color to the list. The original enum defined colors (about
> 102), last Color_ignore.
> So each branch will now get colors 103, 104, ..., and so on, so that the original enum
> (probably used as 'char') cannot contain the value beyond 127).
> 
> Expanding the colors in src/ColorCode.h as in attached, removes the sanitize messages.

Nice, that makes sense. 500 seems to be a reasonable ceiling. I suppose an alternative would be to have the code stop assigning new colors past the max. That is, I imagine they could just assign the max color value to multiple insets?

I didn't understand how JMarc's suggestion to change the type to int would solve the (non) issue.

Scott
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.lyx.org/pipermail/lyx-devel/attachments/20210921/8bb6f054/attachment.asc>