Debian Bug report logs - #786440
python-pip: pip is out-of-date, 'pip list' fails sometimes

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Brian Warner <warner@lothar.com>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: python-pip: pip is out-of-date, 'pip list' fails sometimes

Date: Thu, 21 May 2015 10:56:34 -0700

Package: python-pip
Version: 1.5.6-5
Severity: important

I noticed that 'pip list' on my sid system failed, with a rather obscure
error:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/list.py", line 80, in run
    self.run_listing(options)
  File "/usr/lib/python2.7/dist-packages/pip/commands/list.py", line 142, in run_listing
    self.output_package_listing(installed_packages)
  File "/usr/lib/python2.7/dist-packages/pip/commands/list.py", line 151, in output_package_listing
    if dist_is_editable(dist):
  File "/usr/lib/python2.7/dist-packages/pip/util.py", line 367, in dist_is_editable
    req = FrozenRequirement.from_dist(dist, [])
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 299, in from_dist
    assert len(specs) == 1 and specs[0][0] == '=='
AssertionError

After some digging, it turned out that the problem was that I have the
'python-pyhsm' package installed, which has a version string of "1.0.4k",
and the extra "k" causes pkg_resources to return a "spec" that looks like
"=== 1.0.4k" (with three equal signs, not two), triggering the assert. Most
packages have a "normalized" version string, for which pkg_resources emits
a spec like "== 1.0.0", which doesn't cause pip to complain.

This was fixed in pip-6.0, which tolerates both == and ===. Sid currently
has a pkg_resources that can emit ===, but not a pip that can tolerate it.
This only shows up if you happen to have at least one python package
installed with a non-normalized version string. For me, I was able to make
pip work again by uninstalling python-pyhsm.

I'm sure you're aware that sid's pip-1.5.6 is pretty out-of-date, and that
6.1.1 is the current version (https://pip.pypa.io/en/stable/news.html).
It's not a trivial upgrade, for sure, but consider this note as supporting
evidence in the campaign to get pip updated :).

cheers,
 -Brian


Steps To Reproduce:
 apt-get install python-pip python-pyhsm
 pip list
Expected:
 a list of packages
Got Instead:
 part of the list, then an exception with an AssertionError

cheers,
 -Brian


-- System Information:
Debian Release: stretch/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.16.0-4-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)

Versions of packages python-pip depends on:
ii  ca-certificates       20141019
ii  python                2.7.9-1
ii  python-colorama       0.3.3-1
ii  python-distlib        0.2.0-1
ii  python-html5lib       0.999-3
ii  python-pkg-resources  16.0-1
ii  python-requests       2.4.3-6
ii  python-setuptools     16.0-1
ii  python-six            1.9.0-3
pn  python:any            <none>

Versions of packages python-pip recommends:
ii  build-essential  11.7
pn  python-dev-all   <none>
ii  python-wheel     0.24.0-1

python-pip suggests no packages.

-- no debconf information

Message #10 received at submit@bugs.debian.org (full text, mbox, reply):

From: Donald Stufft <donald@stufft.io>

To: Brian Warner <warner@lothar.com>, 786440@bugs.debian.org

Cc: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: Re: Bug#786440: python-pip: pip is out-of-date, 'pip list' fails sometimes

Date: Thu, 21 May 2015 14:01:59 -0400

[Message part 1 (text/plain, inline)]

I know that Barry is working on this, pip introduced several new dependencies
in 6.x that he had to package and get added to the archive before he could get
pip updated.


> On May 21, 2015, at 1:56 PM, Brian Warner <warner@lothar.com> wrote:
> 
> Package: python-pip
> Version: 1.5.6-5
> Severity: important
> 
> I noticed that 'pip list' on my sid system failed, with a rather obscure
> error:
> 
> Traceback (most recent call last):
>  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
>    status = self.run(options, args)
>  File "/usr/lib/python2.7/dist-packages/pip/commands/list.py", line 80, in run
>    self.run_listing(options)
>  File "/usr/lib/python2.7/dist-packages/pip/commands/list.py", line 142, in run_listing
>    self.output_package_listing(installed_packages)
>  File "/usr/lib/python2.7/dist-packages/pip/commands/list.py", line 151, in output_package_listing
>    if dist_is_editable(dist):
>  File "/usr/lib/python2.7/dist-packages/pip/util.py", line 367, in dist_is_editable
>    req = FrozenRequirement.from_dist(dist, [])
>  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 299, in from_dist
>    assert len(specs) == 1 and specs[0][0] == '=='
> AssertionError
> 
> After some digging, it turned out that the problem was that I have the
> 'python-pyhsm' package installed, which has a version string of "1.0.4k",
> and the extra "k" causes pkg_resources to return a "spec" that looks like
> "=== 1.0.4k" (with three equal signs, not two), triggering the assert. Most
> packages have a "normalized" version string, for which pkg_resources emits
> a spec like "== 1.0.0", which doesn't cause pip to complain.
> 
> This was fixed in pip-6.0, which tolerates both == and ===. Sid currently
> has a pkg_resources that can emit ===, but not a pip that can tolerate it.
> This only shows up if you happen to have at least one python package
> installed with a non-normalized version string. For me, I was able to make
> pip work again by uninstalling python-pyhsm.
> 
> I'm sure you're aware that sid's pip-1.5.6 is pretty out-of-date, and that
> 6.1.1 is the current version (https://pip.pypa.io/en/stable/news.html).
> It's not a trivial upgrade, for sure, but consider this note as supporting
> evidence in the campaign to get pip updated :).
> 
> cheers,
> -Brian
> 
> 
> Steps To Reproduce:
> apt-get install python-pip python-pyhsm
> pip list
> Expected:
> a list of packages
> Got Instead:
> part of the list, then an exception with an AssertionError
> 
> cheers,
> -Brian
> 
> 
> -- System Information:
> Debian Release: stretch/sid
>  APT prefers unstable
>  APT policy: (500, 'unstable')
> Architecture: amd64 (x86_64)
> 
> Kernel: Linux 3.16.0-4-amd64 (SMP w/4 CPU cores)
> Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
> Shell: /bin/sh linked to /bin/dash
> Init: sysvinit (via /sbin/init)
> 
> Versions of packages python-pip depends on:
> ii  ca-certificates       20141019
> ii  python                2.7.9-1
> ii  python-colorama       0.3.3-1
> ii  python-distlib        0.2.0-1
> ii  python-html5lib       0.999-3
> ii  python-pkg-resources  16.0-1
> ii  python-requests       2.4.3-6
> ii  python-setuptools     16.0-1
> ii  python-six            1.9.0-3
> pn  python:any            <none>
> 
> Versions of packages python-pip recommends:
> ii  build-essential  11.7
> pn  python-dev-all   <none>
> ii  python-wheel     0.24.0-1
> 
> python-pip suggests no packages.
> 
> -- no debconf information
> 

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

[signature.asc (application/pgp-signature, attachment)]

Message #20 received at 786440@bugs.debian.org (full text, mbox, reply):

From: "Felix C. Stegerman" <flx@obfusk.net>

To: 786580@bugs.debian.org

Cc: 786440@bugs.debian.org, 785787@bugs.debian.org

Subject: pip and distlib mismatch breaks pip and virtualenv

Date: Sun, 24 May 2015 21:34:38 +0200

[Message part 1 (text/plain, inline)]

Hi,

I've looked at what's going on, and the problem seems to be caused by
the fact that distlib 0.1.9 uses PEP426 and 0.2.0 uses PEP440; the
latter has an extra "epoch" field.

Because pip.util (in the version included with Debian) expects the
output not to have this epoch field, it breaks.

Downgrading python-distlib and python-distlib-whl to the version in
testing (0.1.9) makes pip and virtualenv work again.

Alternatively, updating to the latest pip (which is not yet available
in Debian) would probably fix this as well.

The third option would be to temporarily patch the current pip to
support both versions of distlib.  I've attached a patch (for
pip/util.py) that seems to work.

- Felix

[pip_util.diff (text/x-diff, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #25 received at 786440@bugs.debian.org (full text, mbox, reply):

From: Barry Warsaw <barry@debian.org>

To: 786440@bugs.debian.org

Date: Tue, 26 May 2015 17:45:30 -0400

[Message part 1 (text/plain, inline)]

@Felix: Actually the pip_util.diff is only relevant for bug #758787.  I've
tested what will be pip 1.5.6-6 with that patch and it isn't relevant for bug
#786440 afaict.

I'm inclined not to fix this for the 1.5.6 series, and just concentrate on
getting (now) pip 7 into Debian.

[Message part 2 (application/pgp-signature, inline)]

Message #30 received at 786440@bugs.debian.org (full text, mbox, reply):

From: "Felix C. Stegerman" <flx@obfusk.net>

To: Barry Warsaw <barry@debian.org>, 786440@bugs.debian.org

Subject: Re: Bug#786440: (no subject)

Date: Wed, 27 May 2015 01:18:06 +0200

[Message part 1 (text/plain, inline)]

* Barry Warsaw <barry@debian.org> [2015-05-26 23:45]:
> @Felix: Actually the pip_util.diff is only relevant for bug #758787.  I've
> tested what will be pip 1.5.6-6 with that patch and it isn't relevant for bug
> #786440 afaict.

You're absolutely right.  I intentionally CC'd this bug because it's
another bug related to an outdated pip, but I should have been clear
about the problems being unrelated otherwise.

> I'm inclined not to fix this for the 1.5.6 series, and just concentrate on
> getting (now) pip 7 into Debian.

That would be great of course.  Thanks!

- Felix

[signature.asc (application/pgp-signature, inline)]

Message #35 received at 786440@bugs.debian.org (full text, mbox, reply):

From: Grant Wu <grantwwu@gmail.com>

To: Debian Bug Tracking System <786440@bugs.debian.org>

Subject: python-pip: Incompatibility with newer versions of Python Requests

Date: Thu, 18 Jun 2015 12:34:15 -0400

Source: python-pip
Followup-For: Bug #786440

Dear Maintainer,

It appears that the version of pip available through this
package is incompatible with the current version of Python Requests.

Installing pip from this package and running pip results in the
following traceback:

Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    load_entry_point('pip==1.5.6', 'console_scripts', 'pip')()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 356, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2476, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2190, in load
    ['__name__'])
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 74, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar  # noqa
  File "/usr/lib/python2.7/dist-packages/pip/vcs/mercurial.py", line 9, in <module>
    from pip.download import path_to_url
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 25, in <module>
    from requests.compat import IncompleteRead
ImportError: cannot import name IncompleteRead

According to http://stackoverflow.com/a/27341847 this is due to IncompleteRead
 being removed from requests.

-- System Information:
Debian Release: 8.1
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.16.0-4-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Message #40 received at 786440-done@bugs.debian.org (full text, mbox, reply):

From: Barry Warsaw <barry@debian.org>

To: 786440-done@bugs.debian.org

Date: Sat, 30 Jan 2016 16:58:16 -0500

[Message part 1 (text/plain, inline)]

I'm fairly certain the recent upload of 8.0.2 fixes this.

[Message part 2 (application/pgp-signature, inline)]

Send a report that this bug log contains spam.