Debian Bug report logs - #824347
fails in jessie with "django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet."

Display info messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Antoine Beaupré <anarcat@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: fails in jessie with "django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet."

Date: Sat, 14 May 2016 14:59:22 -0400

Package: planet-venus
Version: 0~git9de2109-3
Severity: grave
Tags: upstream

I get a fatal exception when following the bootstrap procedure for
Debian planet documented here:

https://wiki.debian.org/PlanetDebian#test

Here's the backtrace, which happens after all the feeds are parsed (so
it takes a *loong* time to get there on debian-planet!):

INFO:planet.runner:Loading cached data
Traceback (most recent call last):
  File "/usr/bin/planet", line 158, in <module>
    splice.apply(doc.toxml('utf-8'))
  File "/usr/lib/python2.7/dist-packages/planet/splice.py", line 142, in apply
    output_file = shell.run(template_file, doc)
  File "/usr/lib/python2.7/dist-packages/planet/shell/__init__.py", line 66, in run
    module.run(template_resolved, doc, output_file, options)
  File "/usr/lib/python2.7/dist-packages/planet/shell/dj.py", line 38, in run
    t = get_template(script)
  File "/usr/lib/python2.7/dist-packages/django/template/loader.py", line 144, in get_template
    template, origin = find_template(template_name, dirs)
  File "/usr/lib/python2.7/dist-packages/django/template/loader.py", line 126, in find_template
    loader = find_template_loader(loader_name)
  File "/usr/lib/python2.7/dist-packages/django/template/loader.py", line 98, in find_template_loader
    TemplateLoader = import_string(loader)
  File "/usr/lib/python2.7/dist-packages/django/utils/module_loading.py", line 26, in import_string
    module = import_module(module_path)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/lib/python2.7/dist-packages/django/template/loaders/app_directories.py", line 33, in <module>
    app_template_dirs = calculate_app_template_dirs()
  File "/usr/lib/python2.7/dist-packages/django/template/loaders/app_directories.py", line 21, in calculate_app_template_dirs
    for app_config in apps.get_app_configs():
  File "/usr/lib/python2.7/dist-packages/django/apps/registry.py", line 137, in get_app_configs
    self.check_apps_ready()
  File "/usr/lib/python2.7/dist-packages/django/apps/registry.py", line 124, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

In fact, it actually fails before that, because there are two
dependencies missing (python-django and python-genshi) but that's a
separate issue.

The fix I have found it to patch venus with this:

--- planet-venus-0~git9de2109/planet/shell/dj.py	2016-05-13 15:56:44.000000000 -0400
+++ /usr/lib/python2.7/dist-packages/planet/shell/dj.py	2016-05-14 08:43:55.587328754 -0400
@@ -27,6 +27,9 @@
             )
     except RuntimeError:
         pass
+    import django
+    django.setup()
+
     from django.template import Context
     from django.template.loader import get_template
 
I am not sure it is the correct way of doing this: i am only
marginally familiar with Django. I basically cargo-culted this from:

http://stackoverflow.com/a/27194927/1174784
http://stackoverflow.com/a/26215548/1174784
http://django.readthedocs.io/en/latest/releases/1.7.html#standalone-scripts
https://docs.djangoproject.com/en/1.8/topics/settings/#calling-django-setup-is-required-for-standalone-django-usage

So it looks okay-ish. I still get deprecation warnings, but at least
the feed is generated:

INFO:planet.runner:Loading cached data
/usr/lib/python2.7/dist-packages/django/template/base.py:290: RemovedInDjango18Warning: 'The `firstof` template tag is changing to escape its arguments; the non-autoescaping version is deprecated. Load it from the `future` tag library to start using the new behavior.
  compiled_result = compile_func(self, token)

WARNING:py.warnings:/usr/lib/python2.7/dist-packages/django/template/base.py:290: RemovedInDjango18Warning: 'The `firstof` template tag is changing to escape its arguments; the non-autoescaping version is deprecated. Load it from the `future` tag library to start using the new behavior.
  compiled_result = compile_func(self, token)

Proof it is working: http://paste.anarc.at/planet/www/

-- System Information:
Debian Release: 8.4
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'proposed-updates'), (500, 'stable'), (1, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 4.2.0-0.bpo.1-amd64 (SMP w/2 CPU cores)
Locale: LANG=fr_CA.UTF-8, LC_CTYPE=fr_CA.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages planet-venus depends on:
ii  python              2.7.9-1
ii  python-chardet      2.3.0-1
ii  python-feedparser   5.1.3-3
ii  python-html5lib     0.999-3
ii  python-htmltmpl     1.22-10
ii  python-httplib2     0.9+dfsg-2
ii  python-librdf       1.0.17.1+dfsg-1.1+b1
ii  python-libxml2      2.9.1+dfsg1-5+deb8u1
ii  python-portalocker  0.5.1-1
ii  python-utidylib     0.2-9

Versions of packages planet-venus recommends:
ii  python-beautifulsoup  3.2.1-1
ii  python-libxslt1       1.1.28-2+b2

Versions of packages planet-venus suggests:
ii  python-django  1.7.7-1+deb8u4
ii  python-genshi  0.7-3
ii  python-lxml    3.4.0-1

-- no debconf information

Message #12 received at 824347@bugs.debian.org (full text, mbox, reply):

From: Antoine Beaupré <anarcat@debian.org>

To: 824347@bugs.debian.org

Subject: patches from philp

Date: Sat, 14 May 2016 16:06:36 -0400

Actually, I was wondering how this could possibly be working at all, and
I looked at the deployed instance on debian.org. It turns out there's a
patch sitting there, uncommitted:

anarcat@philp:/srv/planet.debian.org/code/venus$ git diff
diff --git a/planet/shell/dj.py b/planet/shell/dj.py
index d2199fc..f26b8da 100644
--- a/planet/shell/dj.py
+++ b/planet/shell/dj.py
@@ -19,7 +19,7 @@ def run(script, doc, output_file=None, options={}):
     # I need to re-import the settings at every call because I have to 
     # set the TEMPLATE_DIRS variable programmatically
     from django.conf import settings
-    settings._wrapped=None
+    #settings._wrapped=None
     try:
         settings.configure(
             DEBUG=True, TEMPLATE_DEBUG=True, 
@@ -27,6 +27,11 @@ def run(script, doc, output_file=None, options={}):
             )
     except EnvironmentError:
         pass
+    except RuntimeError:
+        settings.TEMPLATE_DIRS=(os.path.dirname(script),)
+
+    import django
+    django.setup()
     from django.template import Context
     from django.template.loader import get_template
 
Maybe those 2 (3?) chunks are unrelated but they are certainly relevant
in the discussion here!

A.

-- 
Arguing for surveillance because you have nothing to hide is no
different than making the claim, "I don't care about freedom of speech
because I have nothing to say."
                        - Edward Snowden

Message #22 received at 824347-close@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <dr@jones.dk>

To: 824347-close@bugs.debian.org

Subject: Bug#824347: fixed in planet-venus 0~git9de2109-4.1

Date: Thu, 11 Oct 2018 17:09:06 +0000

Source: planet-venus
Source-Version: 0~git9de2109-4.1

We believe that the bug you reported is fixed in the latest version of
planet-venus, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 824347@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Jonas Smedegaard <dr@jones.dk> (supplier of updated planet-venus package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Mon, 01 Oct 2018 17:46:13 +0200
Source: planet-venus
Binary: planet-venus
Architecture: source all
Version: 0~git9de2109-4.1
Distribution: unstable
Urgency: medium
Maintainer: Python Applications Packaging Team <python-apps-team@lists.alioth.debian.org>
Changed-By: Jonas Smedegaard <dr@jones.dk>
Description:
 planet-venus - aggregate feed generator
Closes: 824347 845987
Changes:
 planet-venus (0~git9de2109-4.1) unstable; urgency=medium
 .
   * Non-maintainer upload.
   * Modernize patch html5lib-no_XHTMLSerializer.
     Closes: Bug#845987. Thanks to Jakob Haufe.
   * Add patch django-setup.patch to fix setup django before use.
     Closes: Bug#824347. Thanks to Antoine Beaupré.
Checksums-Sha1:
 46cfbd846f462d84bf99351b3755a5e7acaea967 2205 planet-venus_0~git9de2109-4.1.dsc
 e701d5e3577ab8ec138c7368792cfda1e471b050 146320 planet-venus_0~git9de2109-4.1.debian.tar.xz
 7dd5bf8cba69e020ff6e97cedc6b75c9195abc5b 188764 planet-venus_0~git9de2109-4.1_all.deb
 2f12af10c9c6f571d4334f97fd6dc71f54ff577c 6339 planet-venus_0~git9de2109-4.1_amd64.buildinfo
Checksums-Sha256:
 8482c32f71d256b8211b67dde7beb5c8433c34e1b6101039f3886993311367f0 2205 planet-venus_0~git9de2109-4.1.dsc
 7337a4bbab78ded5a17721f7e9db66610034c0e93ca4828ac62e163a0598467c 146320 planet-venus_0~git9de2109-4.1.debian.tar.xz
 09181453fdb731aa3b4ce237bd462f43c0adffdcec726ea35a6d9fc6ccc38ef4 188764 planet-venus_0~git9de2109-4.1_all.deb
 d8c3af657bffb22111914921ae64e0193fe01bfc1b93fcdb2e05429fe8436168 6339 planet-venus_0~git9de2109-4.1_amd64.buildinfo
Files:
 087614fa2522a014afc6ea5a944a652e 2205 python extra planet-venus_0~git9de2109-4.1.dsc
 72209db37cdaef225142cb9aac8eda0c 146320 python extra planet-venus_0~git9de2109-4.1.debian.tar.xz
 297900402144b093bc75015e89862504 188764 python extra planet-venus_0~git9de2109-4.1_all.deb
 f66826509cf58fa3b0e338a949996f4f 6339 python extra planet-venus_0~git9de2109-4.1_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEn+Ppw2aRpp/1PMaELHwxRsGgASEFAluyQ1sACgkQLHwxRsGg
ASFGPg/9EFDbLw9ETrYaF8PQSewW1y8qOcvUzAn4LNZSqJiBDeTj89mmKoq0cTJv
s4tK1CNdK3zNDlEn5g1L46kdASHuQgI/Sw0D79r5ksikMthuAQ+zZdJkLHaHdUkY
qXQse6q81hUOO6lQ6fOVjPrlU/2FaiLzuP1wM3CB2RrEZfj3iq/9B7FrvdOdSTvd
nvfNOiVpFFYtxUc9RJgIijZmIxg1hVABWGIX5zMlkSCm14iNvBfTPhopT+52UrSv
wNgzB+LNfc++tOlZFuqNBYs3Cv8v7UjNPYXVddb9JhoR+AXrWGGHk2EBMX4Dgr8+
8dN720piFzfJL68gDEm4NCUWi6nwb/mqO1jP0JpkqW7qW1YHMV74de6M0NAUMxUr
I2Q5+Cco7Rrg/uqyu78o45KXmTUzIDlwbKj4pHIvRaKKWgTyfDIE7GkZ/8euhMis
6L22agcX5zwFS7sq4cGl4owMhvh9libLPFuV2WY8F++yndR8AQ8K2laALPEFj1dW
FC6YJmAt7z84NURdS+w70LYKILOWv3HHxUP4c5Lnzj7TFJQPU/rcpYJz61CJBfHN
OXnFmMLf7cmrEOAH/l03wdSaKZ0JbdOhormWoknRdu0WXrOJwORALdWNbTXDGjj0
CadIWqAlmsamtYtCD++5Lh3fWLWGqV1bNUUROHQM/S5X0PMdj1k=
=ZISj
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.