Report forwarded to debian-bugs-dist@lists.debian.org, Debian Security Team <team@security.debian.org>, Robert Collins <robertc@robertcollins.net>: Bug#484305; Package bicyclerepair.
(full text, mbox, link).
Acknowledgement sent to Thomas Arendsen Hein <thomas@intevation.de>:
New Bug report received and forwarded. Copy sent to Debian Security Team <team@security.debian.org>, Robert Collins <robertc@robertcollins.net>.
(full text, mbox, link).
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: bicyclerepair: bike.vim imports untrusted python files from cwd
Date: Tue, 3 Jun 2008 17:40:03 +0200
Package: bicyclerepair
Version: 0.9-4.1
Severity: critical
Tags: security
Justification: root security hole
# pwd
/tmp/roundup-1.3.3/roundup
# vim /tmp/whatever
Error detected while processing /usr/share/vim/addons/plugin/bike.vim:
line 110:
Traceback (most recent call last):
File "<string>", line 6, in ?
File "/usr/lib/python2.4/site-packages/bike/__init__.py", line 10, in ?
from bikefacade import init, NotAPythonModuleOrPackageException, CouldntLoca
teASTNodeFromCoordinatesException, UndoStackEmptyException
File "/usr/lib/python2.4/site-packages/bike/bikefacade.py", line 3, in ?
import compiler
File "__init__.py", line 24, in ?
File "compiler/transformer.py", line 1348, in ?
AttributeError: 'module' object has no attribute 'LESS'
Press ENTER or type command to continue
bicyclerepair contains /usr/share/vim/addons/plugin/bike.vim which is
automatically executed, at least in etch. I don't know about lenny/sid,
see #464817 (bicyclerepair: Conform with Vim addon policy)
It imports (i.e. runs) python code it finds in the current working
directory, in my example from the extracted roundup tarball.
I set Severity to "critical" instead of "grave", because the user who
reported the traceback to me on a multi-user system does not use
bicyclerepair, but just vim. Reportbug forced me to set "root security
hole", because everyone using vim is affected (including root) and
the Justification 5 "unknown / something else" would downgrade the
Severity to "normal". The description for "grave" said, that it only
applies if the security problem affects people actually using the package.
-- System Information:
Debian Release: 4.0
APT prefers stable
APT policy: (500, 'stable')
Architecture: i386 (i686)
Shell: /bin/sh linked to /bin/bash
Kernel: Linux 2.6.24.3-id1-k8-2
Locale: LANG=en_US, LC_CTYPE=de_DE@euro (charmap=ISO-8859-15)
Versions of packages bicyclerepair depends on:
ii python 2.4.4-2 An interactive high-level object-o
ii python-central 0.5.12 register and build utility for Pyt
bicyclerepair recommends no packages.
-- no debconf information
--
thomas@intevation.de - http://intevation.de/~thomas/ - OpenPGP key: 0x5816791A
Intevation GmbH, Osnabrueck - Register: Amtsgericht Osnabrueck, HR B 18998
Geschaeftsfuehrer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner
Information forwarded to debian-bugs-dist@lists.debian.org, Robert Collins <robertc@robertcollins.net>: Bug#484305; Package bicyclerepair.
(full text, mbox, link).
Acknowledgement sent to Steffen Joeris <steffen.joeris@skolelinux.de>:
Extra info received and forwarded to list. Copy sent to Robert Collins <robertc@robertcollins.net>.
(full text, mbox, link).
severity 484305 important
thanks
Hi
I have troubles reproducing your problem and I've tried an etch and a lenny
chroot.
white@security:/tmp/roundup-1.2.1/roundup$ pwd
/tmp/roundup-1.2.1/roundup
white@security:/tmp/roundup-1.2.1/roundup$ vim /tmp/foo
white@security:/tmp/roundup-1.2.1/roundup$ apt-cache policy bicyclerepair
bicyclerepair:
Installed: 0.9-4.1
Candidate: 0.9-4.1
Version table:
*** 0.9-4.1 0
500 http://ftp.no.debian.org etch/main Packages
100 /var/lib/dpkg/status
and
white@security:/tmp/roundup-1.4.4/roundup$ pwd
/tmp/roundup-1.4.4/roundup
white@security:/tmp/roundup-1.4.4/roundup$ vim /tmp/foo
white@security:/tmp/roundup-1.4.4/roundup$ apt-cache policy bicyclerepair
bicyclerepair:
Installed: 0.9-4.2
Candidate: 0.9-4.2
Version table:
*** 0.9-4.2 0
500 http://ftp.no.debian.org lenny/main Packages
100 /var/lib/dpkg/status
Also, I can't find /usr/share/vim/addons/plugin/bike.vim. The package
bicyclerepair only contains /usr/share/vim/addons/ftplugin/python_bike.vim,
however I am not so firm with vim plugins after all.
I am using a standard vim config file.
Cheers
Steffen
Severity set to `important' from `critical'
Request was from Steffen Joeris <steffen.joeris@skolelinux.de>
to control@bugs.debian.org.
(Sun, 06 Jul 2008 09:06:03 GMT) (full text, mbox, link).
Information forwarded to debian-bugs-dist@lists.debian.org, Robert Collins <robertc@robertcollins.net>: Bug#484305; Package bicyclerepair.
(full text, mbox, link).
Acknowledgement sent to Thomas Arendsen Hein <thomas@intevation.de>:
Extra info received and forwarded to list. Copy sent to Robert Collins <robertc@robertcollins.net>.
(full text, mbox, link).
severity 484305 critical
thanks
* Steffen Joeris <steffen.joeris@skolelinux.de> [20080706 11:15]:
> severity 484305 important
> thanks
Please do not downgrade severity without providing a reason. As I
wrote in my original report, this should not be less than "grave":
| I set Severity to "critical" instead of "grave", because the user who
| reported the traceback to me on a multi-user system does not use
| bicyclerepair, but just vim. Reportbug forced me to set "root security
| hole", because everyone using vim is affected (including root) and
| the Justification 5 "unknown / something else" would downgrade the
| Severity to "normal". The description for "grave" said, that it only
| applies if the security problem affects people actually using the package.
> I have troubles reproducing your problem and I've tried an etch and a lenny
> chroot.
>
> white@security:/tmp/roundup-1.2.1/roundup$ pwd
> /tmp/roundup-1.2.1/roundup
> white@security:/tmp/roundup-1.2.1/roundup$ vim /tmp/foo
> white@security:/tmp/roundup-1.2.1/roundup$ apt-cache policy bicyclerepair
> bicyclerepair:
> Installed: 0.9-4.1
> Candidate: 0.9-4.1
> Version table:
> *** 0.9-4.1 0
> 500 http://ftp.no.debian.org etch/main Packages
> 100 /var/lib/dpkg/status
>
> Also, I can't find /usr/share/vim/addons/plugin/bike.vim. The package
> bicyclerepair only contains /usr/share/vim/addons/ftplugin/python_bike.vim,
> however I am not so firm with vim plugins after all.
> I am using a standard vim config file.
On etch:
$ dpkg -l bicyclerepair|grep ^i
ii bicyclerepair 0.9-4.1 A refactoring tool for python
$ dpkg -L bicyclerepair|grep vim
/usr/share/doc/bicyclerepair/README.vim
/usr/share/vim
/usr/share/vim/vim62
/usr/share/vim/vim62/plugin
/usr/share/vim/vim62/plugin/bike.vim
/usr/share/vim/vim63
/usr/share/vim/vim63/plugin
/usr/share/vim/vim63/plugin/bike.vim
/usr/share/vim/addons
/usr/share/vim/addons/plugin
/usr/share/vim/addons/plugin/bike.vim
Maybe (I haven't verified) you need:
/etc/alternatives/vim -> /usr/bin/vim.python
On lenny the plugin is indeed stored in
/usr/share/vim/addons/ftplugin/python_bike.vim
Regards,
Thomas
--
thomas@intevation.de - http://intevation.de/~thomas/ - OpenPGP key: 0x5816791A
Intevation GmbH, Osnabrueck - Register: Amtsgericht Osnabrueck, HR B 18998
Geschaeftsfuehrer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner
Severity set to `critical' from `important'
Request was from Thomas Arendsen Hein <thomas@intevation.de>
to control@bugs.debian.org.
(Sun, 06 Jul 2008 20:30:10 GMT) (full text, mbox, link).
Information forwarded to debian-bugs-dist@lists.debian.org, Robert Collins <robertc@robertcollins.net>: Bug#484305; Package bicyclerepair.
(full text, mbox, link).
Acknowledgement sent to Nico Golde <nion@debian.org>:
Extra info received and forwarded to list. Copy sent to Robert Collins <robertc@robertcollins.net>.
(full text, mbox, link).
severity 484305 grave
thanks
Hi Thomas,
* Thomas Arendsen Hein <thomas@intevation.de> [2008-07-06 22:53]:
> * Steffen Joeris <steffen.joeris@skolelinux.de> [20080706 11:15]:
> > severity 484305 important
> > thanks
>
> Please do not downgrade severity without providing a reason.
"critical makes unrelated software on the system (or the whole system)
break, or causes serious data loss, or introduces a security
hole on systems where you install the package."
I had a look at the issue now and this is not the case
because you have to a) install vim-python and bicyclerepair
together and b) set vim.python as the vim alternative.
Thus downgrading this bug.
> As I
> wrote in my original report, this should not be less than "grave":
>
> | I set Severity to "critical" instead of "grave", because the user who
> | reported the traceback to me on a multi-user system does not use
> | bicyclerepair, but just vim. Reportbug forced me to set "root security
> | hole", because everyone using vim is affected (including root) and
> | the Justification 5 "unknown / something else" would downgrade the
> | Severity to "normal".
I think that this is more like a user security hole because
the security issue itself doesn't automatically result in
root access. root security hole fit better to issues
included in a daemon running as root for example. But I
doubt discussing this gets us anywhere and I personally
don't care about this tag in this case :)
[...]
> On etch:
>
> $ dpkg -l bicyclerepair|grep ^i
> ii bicyclerepair 0.9-4.1 A refactoring tool for python
>
> $ dpkg -L bicyclerepair|grep vim
> /usr/share/doc/bicyclerepair/README.vim
> /usr/share/vim
> /usr/share/vim/vim62
> /usr/share/vim/vim62/plugin
> /usr/share/vim/vim62/plugin/bike.vim
> /usr/share/vim/vim63
> /usr/share/vim/vim63/plugin
> /usr/share/vim/vim63/plugin/bike.vim
> /usr/share/vim/addons
> /usr/share/vim/addons/plugin
> /usr/share/vim/addons/plugin/bike.vim
>
> Maybe (I haven't verified) you need:
> /etc/alternatives/vim -> /usr/bin/vim.python
Indeed, this is needed (+ installation of vim-python).
So to sum up you need to install vim-python and set the
alternative to vim.python. I am not sure about the status of
this in unstable, at least I could not reproduce this on
unstable but vim.python is also no longer available there,
a lot in the vim structure changed since then and I don't
really have an idea about the scripting support of vim.
That's why I Cc'ed the vim maintainers. Do you think this
should also work in the same way in unstable/testing?
I am also not really sure what is causing the automatic
import.
To reproduce this on stable:
cd /tmp && apt-get source roundup && roundup-1.2.1/roundup/
apt-get install vim-python bicyclerepair
update-alternatives --set vim /usr/bin/vim.python
and edit some random file (e.g. vim /tmp/foobar).
I found out that the file that causes this is token.py in
the roundup sources. Another way to reproduce this would be
to create a file named fcntl:
cat >> fcntl.py << EOF
print "FOOOOBAR"
EOF
This file is also automatically imported besides the files
bike.py, compiler.py, parser.py, symbol.py, token.py, struct.py
cStringIO.py, dis.py, opcode.py, new.py, re.py, sre.py
sre_compile.py, sre_constants.py, sre_parse.py, __future__.py
string.py, strop.py, tempfile.py, random.py, math.py, binascii.py
_random.py and fcntl.py
Something should prepend '.' to syspath but I don't see
anything doing this :/
Kind regards
Nico
--
Nico Golde - http://www.ngolde.de - nion@jabber.ccc.de - GPG: 0x73647CFF
For security reasons, all text in this mail is double-rot13 encrypted.
Severity set to `grave' from `critical'
Request was from Nico Golde <nion@debian.org>
to control@bugs.debian.org.
(Mon, 07 Jul 2008 17:15:06 GMT) (full text, mbox, link).
Information forwarded to debian-bugs-dist@lists.debian.org, Robert Collins <robertc@robertcollins.net>: Bug#484305; Package bicyclerepair.
(full text, mbox, link).
Acknowledgement sent to James Vega <jamessan@debian.org>:
Extra info received and forwarded to list. Copy sent to Robert Collins <robertc@robertcollins.net>.
(full text, mbox, link).
On Mon, Jul 07, 2008 at 07:11:10PM +0200, Nico Golde wrote:
> Hi Thomas,
> * Thomas Arendsen Hein <thomas@intevation.de> [2008-07-06 22:53]:
> > * Steffen Joeris <steffen.joeris@skolelinux.de> [20080706 11:15]:
> > > severity 484305 important
> > > thanks
> >
> > Please do not downgrade severity without providing a reason.
>
> "critical makes unrelated software on the system (or the whole system)
> break, or causes serious data loss, or introduces a security
> hole on systems where you install the package."
>
> I had a look at the issue now and this is not the case
> because you have to a) install vim-python and bicyclerepair
> together and b) set vim.python as the vim alternative.
> Thus downgrading this bug.
In Lenny/Sid, all Vim packages except vim-tiny and vim contain the
python support, so it is more likely that a user will have a vim binary
that can be scripted via Python.
On the other hand, in Lenny/Sid the path that plugins used to be
installed to (/usr/share/vim/addons) is no longer automatically included
in Vim's runtimepath. This means that manual work is required to enable
the plugin.
> [...]
> > On etch:
> >
> > $ dpkg -l bicyclerepair|grep ^i
> > ii bicyclerepair 0.9-4.1 A refactoring tool for python
> >
> > $ dpkg -L bicyclerepair|grep vim
> > /usr/share/doc/bicyclerepair/README.vim
> > /usr/share/vim
> > /usr/share/vim/vim62
> > /usr/share/vim/vim62/plugin
> > /usr/share/vim/vim62/plugin/bike.vim
> > /usr/share/vim/vim63
> > /usr/share/vim/vim63/plugin
> > /usr/share/vim/vim63/plugin/bike.vim
> > /usr/share/vim/addons
> > /usr/share/vim/addons/plugin
> > /usr/share/vim/addons/plugin/bike.vim
> >
> > Maybe (I haven't verified) you need:
> > /etc/alternatives/vim -> /usr/bin/vim.python
>
> Indeed, this is needed (+ installation of vim-python).
> So to sum up you need to install vim-python and set the
> alternative to vim.python. I am not sure about the status of
> this in unstable, at least I could not reproduce this on
> unstable but vim.python is also no longer available there,
> a lot in the vim structure changed since then and I don't
> really have an idea about the scripting support of vim.
>
> That's why I Cc'ed the vim maintainers. Do you think this
> should also work in the same way in unstable/testing?
See above explanation.
Also, taking a look at the current bicyclerepair package, the addon is
now installed to /usr/share/addons/vim/ftplugin/python_bike.vim. This
means that the functionality will only be used when editing python files
(once the user has enabled the plugin) instead of when editing any file,
as was the case when it was installed to plugins/bike.vim.
> I am also not really sure what is causing the automatic
> import.
Python, by default, has '' as the initial item in its sys.path list
$ python
Python 2.5.2 (r252:60911, Jun 25 2008, 17:58:32)
[GCC 4.3.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', '/usr/lib/python2.5/lib-dynload', '/usr/local/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages/Numeric', '/usr/lib/python2.5/site-packages/gst-0.10', '/var/lib/python-support/python2.5', '/var/lib/python-support/python2.5/gtk-2.0']
This means that anything in the current directory is first priority when
trying to use/import a module.
/usr/lib/pythonX.Y/compiler/transformer.py has the following lines that
are run when the module is imported
import token
...
_cmp_types = {
token.LESS : '<',
token.GREATER : '>',
token.EQEQUAL : '==',
token.EQUAL : '==',
token.LESSEQUAL : '<=',
token.GREATEREQUAL : '>=',
token.NOTEQUAL : '!=',
}
When the bike Vim plugin is loaded, it imports the bike python module,
which imports compiler (and therefore compiler.transformer). Since
Vim's current working directory is roundup-X.Y/roundup, the above lines
from transformer.py combined with '' being the first item in sys.path
cause python to load the token module in the current working directory
(from roundup's source) instead of using /usr/lib/pythonX.Y/token.py.
--
James
GPG Key: 1024D/61326D40 2003-09-02 James Vega <jamessan@debian.org>
Information forwarded to debian-bugs-dist@lists.debian.org, Robert Collins <robertc@robertcollins.net>: Bug#484305; Package bicyclerepair.
(full text, mbox, link).
Acknowledgement sent to Nico Golde <nion@debian.org>:
Extra info received and forwarded to list. Copy sent to Robert Collins <robertc@robertcollins.net>.
(full text, mbox, link).
To: Thomas Arendsen Hein <thomas@intevation.de>, 484305@bugs.debian.org,
pkg-vim-maintainers@lists.alioth.debian.org,
Steffen Joeris <steffen.joeris@skolelinux.de>
Subject: Re: Bug#484305: PoC not working for bicyclerepair
Hi James,
* James Vega <jamessan@debian.org> [2008-07-07 20:11]:
> On Mon, Jul 07, 2008 at 07:11:10PM +0200, Nico Golde wrote:
> > * Thomas Arendsen Hein <thomas@intevation.de> [2008-07-06 22:53]:
> > > * Steffen Joeris <steffen.joeris@skolelinux.de> [20080706 11:15]:
[...]
> In Lenny/Sid, all Vim packages except vim-tiny and vim contain the
> python support, so it is more likely that a user will have a vim binary
> that can be scripted via Python.
>
> On the other hand, in Lenny/Sid the path that plugins used to be
> installed to (/usr/share/vim/addons) is no longer automatically included
> in Vim's runtimepath. This means that manual work is required to enable
> the plugin.
Ok, thanks for this information!
[...]
> > That's why I Cc'ed the vim maintainers. Do you think this
> > should also work in the same way in unstable/testing?
>
> See above explanation.
>
> Also, taking a look at the current bicyclerepair package, the addon is
> now installed to /usr/share/addons/vim/ftplugin/python_bike.vim. This
> means that the functionality will only be used when editing python files
> (once the user has enabled the plugin) instead of when editing any file,
> as was the case when it was installed to plugins/bike.vim.
Ok
> > I am also not really sure what is causing the automatic
> > import.
>
> Python, by default, has '' as the initial item in its sys.path list
>
> $ python
> Python 2.5.2 (r252:60911, Jun 25 2008, 17:58:32)
> [GCC 4.3.1] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys
> >>> sys.path
> ['', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', '/usr/lib/python2.5/lib-dynload', '/usr/local/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages/Numeric', '/usr/lib/python2.5/site-packages/gst-0.10', '/var/lib/python-support/python2.5', '/var/lib/python-support/python2.5/gtk-2.0']
>
> This means that anything in the current directory is first priority when
> trying to use/import a module.
This should only happen in an interactive session, not!?
nion@coredump:/tmp$] mkdir somedir
[nion@coredump:/tmp$] cd somedir
[nion@coredump:somedir$] cat > /tmp/test.py << EOF
heredoc> import sys
heredoc> print sys.path
heredoc> EOF
[nion@coredump:somedir$] python /tmp/test.py
['/tmp', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', '/usr/lib/python2.5/lib-dynload', '/usr/local/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages/Numeric', '/usr/lib/python2.5/site-packages/PIL', '/var/lib/python-support/python2.5', '/var/lib/python-support/python2.5/gtk-2.0', '/usr/lib/python2.5/site-packages/wx-2.6-gtk2-unicode']
The python docs also state "As initialized upon program startup, the first item
of this list, path[0], is the directory containing the script that was used to
invoke the Python interpreter."
So this should be no problem unless I missed your point.
> /usr/lib/pythonX.Y/compiler/transformer.py has the following lines that
> are run when the module is imported
>
> import token
> ...
> _cmp_types = {
> token.LESS : '<',
> token.GREATER : '>',
> token.EQEQUAL : '==',
> token.EQUAL : '==',
> token.LESSEQUAL : '<=',
> token.GREATEREQUAL : '>=',
> token.NOTEQUAL : '!=',
> }
>
> When the bike Vim plugin is loaded, it imports the bike python module,
> which imports compiler (and therefore compiler.transformer). Since
> Vim's current working directory is roundup-X.Y/roundup, the above lines
> from transformer.py combined with '' being the first item in sys.path
> cause python to load the token module in the current working directory
> (from roundup's source) instead of using /usr/lib/pythonX.Y/token.py.
I'd agree if that would be the case in a python script, I had this thought
as well first but this doesn't seem to work. That's why I said that I don't
see something like sys.path = [os.curdir] + sys.path in the code.
Did I misunderstand what you wrote?
Thanks for your help James!
Cheers
Nico
--
Nico Golde - http://www.ngolde.de - nion@jabber.ccc.de - GPG: 0x73647CFF
For security reasons, all text in this mail is double-rot13 encrypted.
Information forwarded to debian-bugs-dist@lists.debian.org, Robert Collins <robertc@robertcollins.net>: Bug#484305; Package bicyclerepair.
(full text, mbox, link).
Acknowledgement sent to James Vega <jamessan@debian.org>:
Extra info received and forwarded to list. Copy sent to Robert Collins <robertc@robertcollins.net>.
(full text, mbox, link).
On Mon, Jul 07, 2008 at 09:16:21PM +0200, Nico Golde wrote:
> Hi James,
> * James Vega <jamessan@debian.org> [2008-07-07 20:11]:
> > On Mon, Jul 07, 2008 at 07:11:10PM +0200, Nico Golde wrote:
> > > I am also not really sure what is causing the automatic
> > > import.
> >
> > Python, by default, has '' as the initial item in its sys.path list
> >
> > $ python
> > Python 2.5.2 (r252:60911, Jun 25 2008, 17:58:32)
> > [GCC 4.3.1] on linux2
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> import sys
> > >>> sys.path
> > ['', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', '/usr/lib/python2.5/lib-dynload', '/usr/local/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages/Numeric', '/usr/lib/python2.5/site-packages/gst-0.10', '/var/lib/python-support/python2.5', '/var/lib/python-support/python2.5/gtk-2.0']
> >
> > This means that anything in the current directory is first priority when
> > trying to use/import a module.
>
> This should only happen in an interactive session, not!?
> nion@coredump:/tmp$] mkdir somedir
> [nion@coredump:/tmp$] cd somedir
> [nion@coredump:somedir$] cat > /tmp/test.py << EOF
> heredoc> import sys
> heredoc> print sys.path
> heredoc> EOF
> [nion@coredump:somedir$] python /tmp/test.py
> ['/tmp', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', '/usr/lib/python2.5/lib-dynload', '/usr/local/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages/Numeric', '/usr/lib/python2.5/site-packages/PIL', '/var/lib/python-support/python2.5', '/var/lib/python-support/python2.5/gtk-2.0', '/usr/lib/python2.5/site-packages/wx-2.6-gtk2-unicode']
>
> The python docs also state "As initialized upon program startup, the first item
> of this list, path[0], is the directory containing the script that was used to
> invoke the Python interpreter."
In an interactive session, sys.path[0] is '' because the "script" is
simply the python interpreter. In your example, if you were in /tmp
when you ran test.py you would have seen '' as sys.path[0] as well.
In the case of Vim, sys.path[0] is always Vim's current working
directory.
/tmp $ mkdir somedir
/tmp $ cat test.vim
function! ShowPath()
python << EOF
import os
import sys
print os.getcwd()
print repr(sys.path[0])
EOF
endfunction
call ShowPath()
cd somedir
call ShowPath()
q
/tmp $ python -S test.vim
/tmp
''
/tmp/somedir
''
From what I can tell, every time the :python command (which is simply a
thin wrapper to the PyRun_SimpleString function from Python's library)
is run from Vim, it's like running a new script.
This would explain why why sys.path[0] and os.getcwd() are updated when
we change the current working directory of the Vim process.
--
James
GPG Key: 1024D/61326D40 2003-09-02 James Vega <jamessan@debian.org>
Information forwarded to debian-bugs-dist@lists.debian.org, Robert Collins <robertc@robertcollins.net>: Bug#484305; Package bicyclerepair.
(full text, mbox, link).
Acknowledgement sent to Nico Golde <nion@debian.org>:
Extra info received and forwarded to list. Copy sent to Robert Collins <robertc@robertcollins.net>.
(full text, mbox, link).
To: Thomas Arendsen Hein <thomas@intevation.de>, 484305@bugs.debian.org,
pkg-vim-maintainers@lists.alioth.debian.org,
Steffen Joeris <steffen.joeris@skolelinux.de>
Subject: Re: Bug#484305: PoC not working for bicyclerepair
Hi James,
* James Vega <jamessan@debian.org> [2008-07-07 22:04]:
> On Mon, Jul 07, 2008 at 09:16:21PM +0200, Nico Golde wrote:
> > * James Vega <jamessan@debian.org> [2008-07-07 20:11]:
> > > On Mon, Jul 07, 2008 at 07:11:10PM +0200, Nico Golde wrote:
[...]
> > [nion@coredump:somedir$] python /tmp/test.py
> > ['/tmp', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', '/usr/lib/python2.5/lib-dynload', '/usr/local/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages/Numeric', '/usr/lib/python2.5/site-packages/PIL', '/var/lib/python-support/python2.5', '/var/lib/python-support/python2.5/gtk-2.0', '/usr/lib/python2.5/site-packages/wx-2.6-gtk2-unicode']
> >
> > The python docs also state "As initialized upon program startup, the first item
> > of this list, path[0], is the directory containing the script that was used to
> > invoke the Python interpreter."
>
> In an interactive session, sys.path[0] is '' because the "script" is
> simply the python interpreter. In your example, if you were in /tmp
> when you ran test.py you would have seen '' as sys.path[0] as well.
Yes sure that conforms to the quote from the python docs.
> In the case of Vim, sys.path[0] is always Vim's current working
> directory.
[...]
> From what I can tell, every time the :python command (which is simply a
> thin wrapper to the PyRun_SimpleString function from Python's library)
> is run from Vim, it's like running a new script.
>
> This would explain why why sys.path[0] and os.getcwd() are updated when
> we change the current working directory of the Vim process.
Ok, that explains why this happens. This is somehow bad
because everyone who writes a python vim script needs to be
aware of that.
Can you think of a better solution than the following?
--- /usr/share/vim/addons/plugin/bike.vim 2008-07-07 22:14:28.000000000 +0200
+++ bike.vim.new 2008-07-07 22:14:26.000000000 +0200
@@ -100,6 +100,7 @@
try:
if sys.version_info < (2, 2):
raise ImportError, 'Bicycle Repair Man needs Python 2.2 or newer'
+ sys.path.remove('')
import bike
bikectx = bike.init()
bikectx.isLoaded # make sure bike package is recent enough
Kind regards
Nico
--
Nico Golde - http://www.ngolde.de - nion@jabber.ccc.de - GPG: 0x73647CFF
For security reasons, all text in this mail is double-rot13 encrypted.
Information forwarded to debian-bugs-dist@lists.debian.org, Robert Collins <robertc@robertcollins.net>: Bug#484305; Package bicyclerepair.
(full text, mbox, link).
Acknowledgement sent to James Vega <jamessan@jamessan.com>:
Extra info received and forwarded to list. Copy sent to Robert Collins <robertc@robertcollins.net>.
(full text, mbox, link).
On Mon, Jul 07, 2008 at 10:15:44PM +0200, Nico Golde wrote:
> Can you think of a better solution than the following?
> --- /usr/share/vim/addons/plugin/bike.vim 2008-07-07 22:14:28.000000000 +0200
> +++ bike.vim.new 2008-07-07 22:14:26.000000000 +0200
> @@ -100,6 +100,7 @@
> try:
> if sys.version_info < (2, 2):
> raise ImportError, 'Bicycle Repair Man needs Python 2.2 or newer'
> + sys.path.remove('')
> import bike
> bikectx = bike.init()
> bikectx.isLoaded # make sure bike package is recent enough
I think this is being addressed from the wrong perspective. This is a
Python problem. Performing "import compiler", where compiler is a
module in the std-lib and performs its own import of another std-lib
module should not cause the module in the user's working directory to be
imported instead.
As evidenced by PEP 328[0] and PEP 366[1], this is an acknowledged
problem upstream and they're working to address it. As for what that
means for the current bug, this is something that python users have to
deal with when they have a file whose name conflicts with one in the
std-lib. This also means that the *only* time users will run into
issues like this are specifically when they have a filename in Vim's
working directory that conflicts with the name of a Python module.
[0] - http://www.python.org/dev/peps/pep-0328/
[1] - http://www.python.org/dev/peps/pep-0366/
--
James
GPG Key: 1024D/61326D40 2003-09-02 James Vega <jamessan@debian.org>
Information forwarded to debian-bugs-dist@lists.debian.org, Robert Collins <robertc@robertcollins.net>: Bug#484305; Package bicyclerepair.
(full text, mbox, link).
Acknowledgement sent to Nico Golde <nion@debian.org>:
Extra info received and forwarded to list. Copy sent to Robert Collins <robertc@robertcollins.net>.
(full text, mbox, link).
To: Thomas Arendsen Hein <thomas@intevation.de>, 484305@bugs.debian.org,
pkg-vim-maintainers@lists.alioth.debian.org,
Steffen Joeris <steffen.joeris@skolelinux.de>
Subject: Re: Bug#484305: PoC not working for bicyclerepair
Hi James,
* James Vega <jamessan@jamessan.com> [2008-07-07 23:42]:
> On Mon, Jul 07, 2008 at 10:15:44PM +0200, Nico Golde wrote:
> > Can you think of a better solution than the following?
> > --- /usr/share/vim/addons/plugin/bike.vim 2008-07-07 22:14:28.000000000 +0200
> > +++ bike.vim.new 2008-07-07 22:14:26.000000000 +0200
> > @@ -100,6 +100,7 @@
> > try:
> > if sys.version_info < (2, 2):
> > raise ImportError, 'Bicycle Repair Man needs Python 2.2 or newer'
> > + sys.path.remove('')
> > import bike
> > bikectx = bike.init()
> > bikectx.isLoaded # make sure bike package is recent enough
>
> I think this is being addressed from the wrong perspective.
Yes I agree, I would also be not really happy about this
solution as all module writes would need to be aware of this
problem.
> This is a Python problem. Performing "import compiler", where compiler is a
> module in the std-lib and performs its own import of another std-lib
> module should not cause the module in the user's working directory to be
> imported instead.
>
> As evidenced by PEP 328[0] and PEP 366[1], this is an acknowledged
> problem upstream and they're working to address it. As for what that
> means for the current bug, this is something that python users have to
> deal with when they have a file whose name conflicts with one in the
> std-lib. This also means that the *only* time users will run into
> issues like this are specifically when they have a filename in Vim's
> working directory that conflicts with the name of a Python module.
Do you think reassigning this to python would be
appropriate?
Thanks for the PEP links, I wasn't aware of them.
Kind regards
Nico
--
Nico Golde - http://www.ngolde.de - nion@jabber.ccc.de - GPG: 0x73647CFF
For security reasons, all text in this mail is double-rot13 encrypted.
Information forwarded to debian-bugs-dist@lists.debian.org, Robert Collins <robertc@robertcollins.net>: Bug#484305; Package bicyclerepair.
(full text, mbox, link).
Acknowledgement sent to James Vega <jamessan@debian.org>:
Extra info received and forwarded to list. Copy sent to Robert Collins <robertc@robertcollins.net>.
(full text, mbox, link).
On Tue, Jul 08, 2008 at 10:18:46PM +0200, Nico Golde wrote:
> Hi James,
> * James Vega <jamessan@jamessan.com> [2008-07-07 23:42]:
> > On Mon, Jul 07, 2008 at 10:15:44PM +0200, Nico Golde wrote:
> > > Can you think of a better solution than the following?
> > > --- /usr/share/vim/addons/plugin/bike.vim 2008-07-07 22:14:28.000000000 +0200
> > > +++ bike.vim.new 2008-07-07 22:14:26.000000000 +0200
> > > @@ -100,6 +100,7 @@
> > > try:
> > > if sys.version_info < (2, 2):
> > > raise ImportError, 'Bicycle Repair Man needs Python 2.2 or newer'
> > > + sys.path.remove('')
> > > import bike
> > > bikectx = bike.init()
> > > bikectx.isLoaded # make sure bike package is recent enough
> >
> > I think this is being addressed from the wrong perspective.
>
> Yes I agree, I would also be not really happy about this
> solution as all module writes would need to be aware of this
> problem.
>
> > This is a Python problem. Performing "import compiler", where compiler is a
> > module in the std-lib and performs its own import of another std-lib
> > module should not cause the module in the user's working directory to be
> > imported instead.
> >
> > As evidenced by PEP 328[0] and PEP 366[1], this is an acknowledged
> > problem upstream and they're working to address it. As for what that
> > means for the current bug, this is something that python users have to
> > deal with when they have a file whose name conflicts with one in the
> > std-lib. This also means that the *only* time users will run into
> > issues like this are specifically when they have a filename in Vim's
> > working directory that conflicts with the name of a Python module.
>
> Do you think reassigning this to python would be
> appropriate?
That would probably be a better place for it, yes. I definitely don't
think it's a bug in Vim or the specific Vim script and I'm unsure
whether it's truly a bug in Python or just less-than-ideal behavior.
Even with the listed PEPs implemented, it sounds like this behavior is
still possible, just not as likely.
--
James
GPG Key: 1024D/61326D40 2003-09-02 James Vega <jamessan@debian.org>
Information forwarded to debian-bugs-dist@lists.debian.org, Robert Collins <robertc@robertcollins.net>: Bug#484305; Package bicyclerepair.
(full text, mbox, link).
Acknowledgement sent to Simon McVittie <smcv@debian.org>:
Extra info received and forwarded to list. Copy sent to Robert Collins <robertc@robertcollins.net>.
(full text, mbox, link).
clone 484305 -1
reassign -1 vim-python
thanks
Shouldn't Python builds of vim avoid this bug by stopping '' from being
prepended to sys.path in the first place?
After looking through Python initialization and vim's if_python.c it
seems that the way forward is to set Python's argv, via PySys_SetArgv(),
to have a non-empty and absolute first argument.
vim sets Python's argv to { "", NULL }, which according to a comment is to
avoid a crash when warn() is called. Changing that to { "/usr/bin/vim", NULL }
would seem to solve this problem - but for that matter, any safe value is fine.
A safe value for argv[0] is any value where there won't be files dir/*.py or
dir/*/__init__.py, where dir == dirname(argv[0]). So setting argv[0] to
"/", "/usr/lib/something" or "/usr/share/vim" would be safe too, for instance.
I'm afraid I haven't tested this in vim itself (the multiple builds take a
while...) but the attached program demonstrates it:
smcv@carbon% gcc -o484305 `python-config --cflags` `python-config --ldflags` 484305.c
smcv@carbon% ./484305
(I have no argv!)
['/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2',
... more output ...
'/var/lib/python-support/python2.5/pyinotify']
smcv@carbon% ./484305 ""
['']
['', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2',
... more output ...
'/var/lib/python-support/python2.5/pyinotify']
smcv@carbon% ./484305 "/usr/bin/vim"
['/usr/bin/vim']
['/usr/bin', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2',
... more output ...
'/var/lib/python-support/python2.5/pyinotify']
Hope this helps,
Simon
Bug 484305 cloned as bug 493937.
Request was from Simon McVittie <smcv@debian.org>
to control@bugs.debian.org.
(Wed, 06 Aug 2008 00:42:02 GMT) (full text, mbox, link).
Information forwarded to debian-bugs-dist@lists.debian.org: Bug#484305; Package bicyclerepair.
(full text, mbox, link).
Acknowledgement sent to Robert Collins <robertc@robertcollins.net>:
Extra info received and forwarded to list.
(full text, mbox, link).
Information forwarded to debian-bugs-dist@lists.debian.org, Robert Collins <robertc@robertcollins.net>: Bug#484305; Package bicyclerepair.
(full text, mbox, link).
Acknowledgement sent to James Vega <jamessan@debian.org>:
Extra info received and forwarded to list. Copy sent to Robert Collins <robertc@robertcollins.net>.
(full text, mbox, link).
On Wed, Aug 06, 2008 at 01:39:15AM +0100, Simon McVittie wrote:
> Shouldn't Python builds of vim avoid this bug by stopping '' from being
> prepended to sys.path in the first place?
As I mentioned earlier[0][1] in the bug log, I don't think removing ''
from sys.argv is the correct change to make in Vim.
> After looking through Python initialization and vim's if_python.c it
> seems that the way forward is to set Python's argv, via PySys_SetArgv(),
> to have a non-empty and absolute first argument.
>
> vim sets Python's argv to { "", NULL }, which according to a comment is to
> avoid a crash when warn() is called. Changing that to { "/usr/bin/vim", NULL }
> would seem to solve this problem - but for that matter, any safe value is fine.
The way Vim is using PySys_SetArgv (and therefore the resulting
behavior) is exactly following the recommended use of PySys_SetArgv
according to upstream's documentation[2].
> A safe value for argv[0] is any value where there won't be files dir/*.py or
> dir/*/__init__.py, where dir == dirname(argv[0]). So setting argv[0] to
> "/", "/usr/lib/something" or "/usr/share/vim" would be safe too, for instance.
>
> I'm afraid I haven't tested this in vim itself (the multiple builds take a
> while...) but the attached program demonstrates it:
>
> smcv@carbon% gcc -o484305 `python-config --cflags` `python-config --ldflags` 484305.c
> smcv@carbon% ./484305
> (I have no argv!)
> ['/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2',
> ... more output ...
> '/var/lib/python-support/python2.5/pyinotify']
> smcv@carbon% ./484305 ""
> ['']
> ['', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2',
> ... more output ...
> '/var/lib/python-support/python2.5/pyinotify']
> smcv@carbon% ./484305 "/usr/bin/vim"
> ['/usr/bin/vim']
> ['/usr/bin', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2',
> ... more output ...
> '/var/lib/python-support/python2.5/pyinotify']
While this does provide a workaround for the issue, this is behavior
inherent in the way Python is designed and should be fixed in Python.
If we choose to instead address every application that embeds Python,
we're just creating an endless stream of work for ourselves.
From a quick check via Google's codesearch, at least X-Chat[3],
Gnumeric[4], python-nautilus[5], and gedit[6] are likely to have this
same problem.
N.B., most of the above projects use a single-element argv of the
project name. This is no different than using a single-element argv of
"" since PySys_SetArgv attempts to resolve argv[0] to an absolute path
and uses "" when it is unable to do so.
[0] - http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=484305#51
[1] - http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=484305#61
[2] - "If there isn't a script that will be run, the first entry in argv
can be an empty string."
http://docs.python.org/api/initialization.html#l2h-881
[3] - http://ln-s.net/27az
[4] - http://ln-s.net/27ay
[5] - http://ln-s.net/27b1
[6] - http://ln-s.net/27b9
--
James
GPG Key: 1024D/61326D40 2003-09-02 James Vega <jamessan@debian.org>
On Tue, 2008-08-05 at 23:07 -0400, James Vega wrote:
> While this does provide a workaround for the issue, this is behavior
> inherent in the way Python is designed and should be fixed in Python.
> If we choose to instead address every application that embeds Python,
> we're just creating an endless stream of work for ourselves.
Possibly. I did file a bug [rejected] on reportbug itself just a few
days ago, because it also will load from . if '' is in the pythonpath.
OTOH perhaps having '' in sys.path is always wrong and we should start a
mass set of bugs to prevent it?
-Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.
Information forwarded
to debian-bugs-dist@lists.debian.org, Robert Collins <robertc@robertcollins.net>: Bug#484305; Package bicyclerepair.
(Tue, 07 Oct 2008 15:06:02 GMT) (full text, mbox, link).
Acknowledgement sent
to <marcos.marado@sonae.com>:
Extra info received and forwarded to list. Copy sent to Robert Collins <robertc@robertcollins.net>.
(Tue, 07 Oct 2008 15:06:02 GMT) (full text, mbox, link).
Hi there,
It seems to me that there are two choices here:
1) this is considered a python issue, that should be solved in python;
2) this is considered an issue in the python apps that act like this (and
as it was said, there's probably a handfull of them), and those should be
solved in each of those apps.
Shouldn't we ask it to the Python Apps Packaging Team [1]?
[1] http://wiki.debian.org/Teams/PythonAppsPackagingTeam
Best regards,
--
Marcos Marado
Information forwarded
to debian-bugs-dist@lists.debian.org, Robert Collins <robertc@robertcollins.net>: Bug#484305; Package bicyclerepair.
(Tue, 07 Oct 2008 15:27:02 GMT) (full text, mbox, link).
Acknowledgement sent
to <marcos.marado@sonae.com>:
Extra info received and forwarded to list. Copy sent to Robert Collins <robertc@robertcollins.net>.
(Tue, 07 Oct 2008 15:27:03 GMT) (full text, mbox, link).
Reply sent
to James Vega <jamessan@debian.org>:
You have taken responsibility.
(Tue, 21 Oct 2008 17:21:03 GMT) (full text, mbox, link).
Notification sent
to Thomas Arendsen Hein <thomas@intevation.de>:
Bug acknowledged by developer.
(Tue, 21 Oct 2008 17:21:03 GMT) (full text, mbox, link).
On Tue, Oct 07, 2008 at 04:00:49PM +0100, marcos.marado@sonae.com wrote:
> It seems to me that there are two choices here:
> 1) this is considered a python issue, that should be solved in python;
This problem should be solved in 2.6 since absolute imports are the
default.
> 2) this is considered an issue in the python apps that act like this (and
> as it was said, there's probably a handfull of them), and those should be
> solved in each of those apps.
I'm currently looking through the various source packages which provide
a binary that uses PySys_SetArgv and investigating whether they're
affected by this problem. If so, I'll be filing bug reports (with
patches).
> Shouldn't we ask it to the Python Apps Packaging Team [1]?
This isn't specifically related to PAPT as most of the affected
applications aren't actually Python applications, but applications
embedding Python.
As far as bicyclerepair is concerned, this bug can be closed (and I'm
doing so with this message) now that the Vim packages remove the empty
directory from sys.path.
--
James
GPG Key: 1024D/61326D40 2003-09-02 James Vega <jamessan@debian.org>
Debbugs is free software and licensed under the terms of the GNU General
Public License version 2. The current version can be obtained
from https://bugs.debian.org/debbugs-source/.