[pkg-discuss] .pyc file stability

[Danek Duvall]

On Wed, Dec 24, 2008 at 08:41:30PM -0500, Mike Meyer wrote:

> In that case, the problem is with your packaging system. If python
> isn't supposed to be mucking around in python's library directories,
> then who is?

Absolutely nothing.  Once a package is installed to a directory under /usr,
/usr needs to be treated as read-only, as it very well may be mounted that
way.

> pyc files are *not* some kind of compiled file you can use like elc or
> jar files. They're a caching mechanism used by the python vm/compiler
> combo to improve performance.

If they're truly cache files, then they should go in a directory guaranteed
to be writable, such as something under /var.

> Once you bundle in the pyc files, it's no longer python version
> independent - the pyc files won't work on other versions of python, and
> someone with write permission running an other version will cause it to
> be rewritten, making the checksums wrong, etc.

Which is fine for the way we deliver python programs on Solaris, where
everything is rebuilt with the python executable used to run the program,
and the shebang statement points to a specific version of the python
executable.  I understan

> trying to treat pyc files as deliverables causes a build-time dependency
> on the run-time python to appear. Unless you can guarantee that the only
> run-time, then they're going to create problems

I think you're missing a verb and object in there.  But like I said, we do
guarantee that components built with a certain version of python are also
executed with it -- we don't just run /usr/bin/python and hope it's the
right version.

> Failing that, not delivering pyc files and having a way for the
> packaging system to clean up files that it didn't deliver but might
> have been created as part of the normal course of running the
> application is probably second best.

We already do this, but it's ugly, and if files are left behind in a
directory the removed package didn't own exclusively, then there's no way
to know that the files can be gotten rid of.  I don't know whether "import
foo" works if foo.pyc exists but not foo.py, but if so, then that's
obviously bad behavior in such a situation.

> As a final option, if ON includes a recent enough version of Python
> (the one they include is so old I no longer remember if it could do
> this or not) is to put all the py files into a zip file that gets
> properly added to the path, as python won't try and create pyc files
> for files imported from a zip file.

We could do that, but having the .py files out in the open is usually very
useful, particularly when something goes awfully wrong, and you can help
someone past the problem with a text editor.

Danek