Python compiler infects "immutable" bundle - how to avoid it?

[Guenter Milde]

On 2020-02-23, Stephan Witt wrote:
> Am 22.02.2020 um 20:13 schrieb Guenter Milde <milde at users.sf.net>:
>> On 2020-02-22, Stephan Witt wrote:
>>> Am 22.02.2020 um 03:35 schrieb Richard Kimberly Heck <rikiheck at lyx.org>:
>>>> On 2/21/20 12:08 PM, Stephan Witt wrote:
>>>>> Am 21.02.2020 um 12:08 schrieb Stephan Witt <st.witt at gmx.net>:
>>>>>> Hi pythonists,

>>>>>> I’m trying to make ready for code signing on Mac.

>>>>>> The idea of code signing is to ship the package with a digital
>>>>>> signature to guarantee the integrity of the software.

>>>>>> https://developer.apple.com/library/archive/documentation/Security/Conceptual/CodeSigningGuide/Introduction/Introduction.html

>>>>>> The problem now is arising: the python scripts LyX is using
>>>>>> are compiled on the fly and the result is placed inside the
>>>>>> package. That way the package looses its integrity.

>>>>>> Is there any suggestion how to handle this?

>> Under Linux, AFAIK, the *.pyc files are not shipped with the package but
>> created by a post-install script. This way, the package can be checked
>> against the signature (before installation) but the byte-compiled files are
>> there once users start to use LyX.

> Yes, after the first run the signature is valid but the contents isn’t
> clean anymore. That’s not nice, IMO.

You are right, if it is required to have the package matching the
signature key also after installation on a mac, you must either ship the
byte-compiled *pyc files or block their creation.

>>>>> ... ATM I cannot see any performance issues.

>> One of the more critical cases would be opening a small but really old file
>> so that lyx2lyx needs most of the rather big modules. Of course, the effect
>> will be more visible with a HDD than with an SSD.

> Frankly, I think it’s more a matter of CPU power than disk performance.

> On my system the call "python -m py_compile configure.py“ lasts about
> 50 msecs real time and the whole configure process is much longer. 

Yes, I believe that configure.py is not a problematic case. It is rather
seldom used and users expect a configuration to take some time.

> The lyx_2_4.py compilation takes 70 msecs.

With pre-compiled files, converting the small test file

  /usr/local/src/lyx/autotests/export/lyx2lyx> time lyx2lyx lyx_2_0_test.lyx 
  
resulted in

  real	0m0,059s
  user	0m0,041s
  sys	0m0,009s

I don't know how much of this is due to loading the script and modules...


>>>> On Windows, we compile the Python files at installation. I don't know if
>>>> that could help.

>>> On Mac I'd prefer to avoid that. The python executable is not part of the
>>> software bundle and therefore its version at runtime is unknown.

>> But one compatible Python version should be installed before installing
>> LyX, right?

> Python 2.7 is part of the OS. And this is subject to change.

Then, you could consider shipping byte-compiled files for 2.7 or for
both, 2.7 and the current stable version or all supported versions¹ and
set the directories readonly. Python will detect any version mismatch and
just use the source files instead.²


Günter

¹ Python 3 will select the pyc file matching its version
  https://docs.python.org/3/tutorial/modules.html#compiled-python-files
² https://www.python.org/dev/peps/pep-3147/