Link error when compiling with -fsanitize=undefined

[Scott Kostyshak]

On Fri, Nov 05, 2021 at 09:18:05PM -0400, Scott Kostyshak wrote:
> On Wed, Sep 22, 2021 at 07:08:48PM +0200, Kornel Benko wrote:
> > Am Wed, 22 Sep 2021 10:28:44 +0200
> > schrieb Jean-Marc Lasgouttes <lasgouttes at lyx.org>:
> > 
> > > Le 21/09/2021 à 16:52, Scott Kostyshak a écrit :
> > > >> Some debugging lead to following observation:
> > > >> Each inserted branch add a color to the list. The original enum defined colors (about
> > > >> 102), last Color_ignore.
> > > >> So each branch will now get colors 103, 104, ..., and so on, so that the original
> > > >> enum (probably used as 'char') cannot contain the value beyond 127).
> > > >>
> > > >> Expanding the colors in src/ColorCode.h as in attached, removes the sanitize
> > > >> messages.  
> > > > 
> > > > Nice, that makes sense. 500 seems to be a reasonable ceiling. I suppose an
> > > > alternative would be to have the code stop assigning new colors past the max. That
> > > > is, I imagine they could just assign the max color value to multiple insets?  
> > > 
> > > But why the 127? Is it that the underlying type of the enum is a signed 
> > > char? What does sizeof(ColorCode) return?
> > 
> > That is not important from my POV. Sanitize is checking also for portability, so if
> > there is one compiler which would use 'char' for this enum, that would justify the
> > message. Just my 2 cents.
> > 
> > > If this is the case, using
> > >    enum ColorCode : int {
> > > should help. But I am surprised because the underlying type should be 
> > > int (unless -fshort-enums is used):
> > > https://stackoverflow.com/questions/1122096/what-is-the-underlying-type-of-a-c-enum
> > 
> > 'int' type may be too big (think on 64-bit (or more) intergers).
> > 
> > > > I didn't understand how JMarc's suggestion to change the type to int would solve the
> > > > (non) issue.  
> > > 
> > > I was under the impression that the code would check that the enum value 
> > > was one of the declared values. But this is not the case, since a number 
> > > of synthetic values between 100 and 127 seem to be valid.
> > > 
> > > I also wonder how you triggered it initially. Is it that you really had 
> > > 25+ branches, or is it that the code "leaks" color code and creates too 
> > > many of them?
> > > 
> > > JMarc
> 
> JMarc any objection to Kornel's patch?

Ping to JMarc.

Scott
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.lyx.org/pipermail/lyx-devel/attachments/20220120/fe4afbb0/attachment-0001.asc>