Debian Bug report logs - #311328
python-pgsql: Parsing of float values is not locale agnostic

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Martin Pitt <mpitt@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: python-pgsql: Parsing of float values is not locale agnostic

Date: Tue, 31 May 2005 10:26:25 +0200

[Message part 1 (text/plain, inline)]

Package: python-pgsql
Version: 2.4.0-5
Severity: important
Tags: patch

Hi!

The parsing of floats in python-pgsql is not locale agnostic since it
uses strtod(). E. g. if you parse the value "1.5" in a locale that
uses a comma as a decimal separator (like de_DE), you wrongly get "1".
Please see

  https://bugzilla.ubuntu.com/show_bug.cgi?id=10191

for details.

I prepared a patch for this which you can get at

  http://patches.ubuntu.com/patches/python-pgsql.floatparsing.diff

According to PEP 331, the python 2.4 version now uses
PyOS_ascii_strtod() to parse the floats; I also added an #if-wrapped
verbatim copy of this function to pgresult.c to be able to compile the
code with python versions earlier than 2.4 (which introduced
PyOS_ascii_strtod()).

Since this is highly important and can lead to grossly wrong results
in e. g. financial applications, can you please try to push this
change into Sarge and also pass this to upstream?


-- System Information:
Debian Release: 3.1
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: i386 (i686)
Kernel: Linux 2.6.11.9
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8)

-- 
Martin Pitt              http://www.piware.de
Ubuntu Developer   http://www.ubuntulinux.org
Debian Developer        http://www.debian.org

[signature.asc (application/pgp-signature, inline)]

Message #10 received at 311328@bugs.debian.org (full text, mbox, reply):

From: Roger Leigh <rleigh@whinlatter.ukfsn.org>

To: 312847@bugs.debian.org, 311328@bugs.debian.org

Subject: Re: python-pgsql: Can't connect to UNIX socket with postgresql-7.4

Date: Fri, 10 Jun 2005 23:03:34 +0100

Hi,

The connection error is because the location of the UNIX socket has
changed between PostgreSQL 7.4 and 8.0.  The package needs to be
rebuilt against libpq4.  The following diff details the changes needed
to do this (and also fixes #311328 for good measure).

You can download the packages from here:
http://people.debian.org/~rleigh/python-pgsql/python-pgsql_2.4.0-5.1_powerpc.changes

Please could you confirm if these fix your problem (you might need to
rebuild for your architecture).

I haven't uploaded these, but I can NMU if needed.

Note to the maintainer: the setup.py change is a quick hack to make it
build.  Ideally upstream should switch to use "pg_config
--includedir", which will give the correct output on all platforms,
rather than guessing defaults which become outdated.


Regards,
Roger


diff -urN python-pgsql-2.4.0.old/debian/changelog python-pgsql-2.4.0/debian/changelog
--- python-pgsql-2.4.0.old/debian/changelog	2005-06-10 22:33:28.217955528 +0100
+++ python-pgsql-2.4.0/debian/changelog	2005-06-10 22:35:57.383278952 +0100
@@ -1,3 +1,21 @@
+python-pgsql (2.4.0-5.1) unstable; urgency=low
+
+  * Non-maintainer upload.
+  * PostgreSQL 8.0 transition.
+    - Build-Depend upon libpq-dev rather than postgresql-dev.
+    - setup.py: Change the include directory to
+      "/usr/include/postgresql/8.0".
+    - Building against libpq4 changes the UNIX socket location
+      (closes: #312847).
+  * Parse float values in a locale-independent manner.  Applied
+    patch to pgresult.c from Martin Pitt (closes: #311328):
+    - Use PyOS_ascii_strtod() instead of strtod() to use locale agnostic
+      parser.
+    - If compiling with a Python version < 2.4, add a local copy of
+      python 2.4's PyOS_ascii_strtod() function.
+
+ -- Roger Leigh <rleigh@debian.org>  Fri, 10 Jun 2005 22:34:58 +0100
+
 python-pgsql (2.4.0-5) unstable; urgency=low
 
   * In python-pgsql, use symlinks to examples and tests from python2.3-pgsql
diff -urN python-pgsql-2.4.0.old/debian/control python-pgsql-2.4.0/debian/control
--- python-pgsql-2.4.0.old/debian/control	2005-06-10 22:33:28.220955072 +0100
+++ python-pgsql-2.4.0/debian/control	2005-06-10 22:10:25.410174264 +0100
@@ -2,7 +2,7 @@
 Section: python
 Priority: optional
 Maintainer: Ben Burton <bab@debian.org>
-Build-Depends: debhelper (>> 3.0.48), ed, postgresql-dev (>= 7.2.1), python2.1-dev, python2.2-dev, python2.3-dev
+Build-Depends: debhelper (>> 3.0.48), ed, libpq-dev (>= 8.0.3), python2.1-dev, python2.2-dev, python2.3-dev
 Standards-Version: 3.6.1
 
 Package: python2.1-pgsql
diff -urN python-pgsql-2.4.0.old/pgresult.c python-pgsql-2.4.0/pgresult.c
--- python-pgsql-2.4.0.old/pgresult.c	2002-10-02 05:08:38.000000000 +0100
+++ python-pgsql-2.4.0/pgresult.c	2005-06-10 22:26:52.932048120 +0100
@@ -110,6 +110,175 @@
 
 /*--------------------------------------------------------------------------*/
 
+#if (PY_VERSION_HEX < 0x02040000)
+
+/* code stolen from Python 2.4 */
+
+#include <locale.h>
+
+#define ISSPACE(c)  ((c) == ' ' || (c) == '\f' || (c) == '\n' || \
+                     (c) == '\r' || (c) == '\t' || (c) == '\v')
+#define ISDIGIT(c)  ((c) >= '0' && (c) <= '9')
+#define ISXDIGIT(c) (ISDIGIT(c) || ((c) >= 'a' && (c) <= 'f') || ((c) >= 'A' && (c) <= 'F'))
+
+/**
+ * PyOS_ascii_strtod:
+ * @nptr:    the string to convert to a numeric value.
+ * @endptr:  if non-%NULL, it returns the character after
+ *           the last character used in the conversion.
+ * 
+ * Converts a string to a #gdouble value.
+ * This function behaves like the standard strtod() function
+ * does in the C locale. It does this without actually
+ * changing the current locale, since that would not be
+ * thread-safe.
+ *
+ * This function is typically used when reading configuration
+ * files or other non-user input that should be locale independent.
+ * To handle input from the user you should normally use the
+ * locale-sensitive system strtod() function.
+ *
+ * If the correct value would cause overflow, plus or minus %HUGE_VAL
+ * is returned (according to the sign of the value), and %ERANGE is
+ * stored in %errno. If the correct value would cause underflow,
+ * zero is returned and %ERANGE is stored in %errno.
+ * 
+ * This function resets %errno before calling strtod() so that
+ * you can reliably detect overflow and underflow.
+ *
+ * Return value: the #gdouble value.
+ **/
+double
+PyOS_ascii_strtod(const char  *nptr, 
+	    char       **endptr)
+{
+	char *fail_pos;
+	double val;
+	struct lconv *locale_data;
+	const char *decimal_point;
+	int decimal_point_len;
+	const char *p, *decimal_point_pos;
+	const char *end = NULL; /* Silence gcc */
+
+/* 	g_return_val_if_fail (nptr != NULL, 0); */
+	assert(nptr != NULL);
+
+	fail_pos = NULL;
+
+	locale_data = localeconv();
+	decimal_point = locale_data->decimal_point;
+	decimal_point_len = strlen(decimal_point);
+
+	assert(decimal_point_len != 0);
+
+	decimal_point_pos = NULL;
+	if (decimal_point[0] != '.' || 
+	    decimal_point[1] != 0)
+	{
+		p = nptr;
+		  /* Skip leading space */
+		while (ISSPACE(*p))
+			p++;
+
+		  /* Skip leading optional sign */
+		if (*p == '+' || *p == '-')
+			p++;
+
+		if (p[0] == '0' && 
+		    (p[1] == 'x' || p[1] == 'X'))
+		{
+			p += 2;
+			  /* HEX - find the (optional) decimal point */
+
+			while (ISXDIGIT(*p))
+				p++;
+
+			if (*p == '.')
+			{
+				decimal_point_pos = p++;
+
+				while (ISXDIGIT(*p))
+					p++;
+
+				if (*p == 'p' || *p == 'P')
+					p++;
+				if (*p == '+' || *p == '-')
+					p++;
+				while (ISDIGIT(*p))
+					p++;
+				end = p;
+			}
+		}
+		else
+		{
+			while (ISDIGIT(*p))
+				p++;
+
+			if (*p == '.')
+			{
+				decimal_point_pos = p++;
+
+				while (ISDIGIT(*p))
+					p++;
+
+				if (*p == 'e' || *p == 'E')
+					p++;
+				if (*p == '+' || *p == '-')
+					p++;
+				while (ISDIGIT(*p))
+					p++;
+				end = p;
+			}
+		}
+		  /* For the other cases, we need not convert the decimal point */
+	}
+
+	  /* Set errno to zero, so that we can distinguish zero results
+	     and underflows */
+	errno = 0;
+
+	if (decimal_point_pos)
+	{
+		char *copy, *c;
+
+		  /* We need to convert the '.' to the locale specific decimal point */
+		copy = malloc(end - nptr + 1 + decimal_point_len);
+
+		c = copy;
+		memcpy(c, nptr, decimal_point_pos - nptr);
+		c += decimal_point_pos - nptr;
+		memcpy(c, decimal_point, decimal_point_len);
+		c += decimal_point_len;
+		memcpy(c, decimal_point_pos + 1, end - (decimal_point_pos + 1));
+		c += end - (decimal_point_pos + 1);
+		*c = 0;
+
+		val = strtod(copy, &fail_pos);
+
+		if (fail_pos)
+		{
+			if (fail_pos > decimal_point_pos)
+				fail_pos = (char *)nptr + (fail_pos - copy) - (decimal_point_len - 1);
+			else
+				fail_pos = (char *)nptr + (fail_pos - copy);
+		}
+
+		free(copy);
+
+	}
+	else
+		val = strtod(nptr, &fail_pos);
+
+	if (endptr)
+		*endptr = fail_pos;
+
+	return val;
+}
+
+#endif
+
+/*--------------------------------------------------------------------------*/
+
 PyObject *PgResult_New(PGresult *res, PgConnection *conn, int type)
 {
     PgResult *self;
@@ -623,7 +792,7 @@
 		/*FALLTHRU*/
 
 	    case PG_FLOAT8:
-		valueObj = Py_BuildValue("d", strtod(value, NULL));
+		valueObj = Py_BuildValue("d", PyOS_ascii_strtod(value, NULL));
 		break;
 
 	    case PG_BYTEA:
diff -urN python-pgsql-2.4.0.old/setup.py python-pgsql-2.4.0/setup.py
--- python-pgsql-2.4.0.old/setup.py	2003-07-14 22:02:02.000000000 +0100
+++ python-pgsql-2.4.0/setup.py	2005-06-10 22:24:05.774459936 +0100
@@ -101,7 +101,7 @@
 	include_dirs = YOUR_LIST_HERE
 	library_dirs = YOUR_LIST_HERE
     elif sys.platform == "linux2":
-	include_dirs = ["/usr/include", "/usr/include/postgresql",
+	include_dirs = ["/usr/include", "/usr/include/postgresql/8.0",
 	    "/usr/include/pgsql"]
 	library_dirs = ["/usr/lib"]
 


-- 
Roger Leigh
                Printing on GNU/Linux?  http://gimp-print.sourceforge.net/
                Debian GNU/Linux        http://www.debian.org/
                GPG Public Key: 0x25BFB848.  Please sign and encrypt your mail.

Message #15 received at 311328@bugs.debian.org (full text, mbox, reply):

From: Ben Burton <bab@debian.org>

To: Roger Leigh <rleigh@whinlatter.ukfsn.org>, 312847@bugs.debian.org

Cc: 311328@bugs.debian.org

Subject: Re: Bug#312847: python-pgsql: Can't connect to UNIX socket with postgresql-7.4

Date: Sat, 11 Jun 2005 17:50:05 +1000

Hi,

Thanks for the report, and thanks Roger for verifying that the libpq4
rebuild fixes it.

> I haven't uploaded these, but I can NMU if needed.

Please don't -- the postgresql 8.0 / libpq4 transition has only just
happened and the bug report is but a day old, and there are other things
I'd like to look at as well for the postgresql 8.0 upload.

b.

Message #20 received at 311328@bugs.debian.org (full text, mbox, reply):

From: Roger Leigh <rleigh@whinlatter.ukfsn.org>

To: Ben Burton <bab@debian.org>

Cc: 312847@bugs.debian.org, 311328@bugs.debian.org

Subject: Re: Bug#312847: python-pgsql: Can't connect to UNIX socket with postgresql-7.4

Date: Sat, 11 Jun 2005 14:34:26 +0100

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ben Burton <bab@debian.org> writes:

> Thanks for the report, and thanks Roger for verifying that the libpq4
> rebuild fixes it.

I haven't actually tested the new packages, but when I rebuilt libpqxx
a couple of days ago, that also had the same problem, and libpq4 fixed
it.

>> I haven't uploaded these, but I can NMU if needed.
>
> Please don't -- the postgresql 8.0 / libpq4 transition has only just
> happened and the bug report is but a day old, and there are other things
> I'd like to look at as well for the postgresql 8.0 upload.

Sure.

[I just spotted it in the etch RC list; it's my goal to fix at least
one RC bug a week for etch]


Regards,
Roger

- -- 
Roger Leigh
                Printing on GNU/Linux?  http://gimp-print.sourceforge.net/
                Debian GNU/Linux        http://www.debian.org/
                GPG Public Key: 0x25BFB848.  Please sign and encrypt your mail.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>

iD8DBQFCquhhVcFcaSW/uEgRAuRHAJ4r7Jrc0sQy/rswjTscaoaG8iPiJQCdFnoc
6N64ydVqPb55kh+dYOnW4c4=
=chnE
-----END PGP SIGNATURE-----

Message #25 received at 311328@bugs.debian.org (full text, mbox, reply):

From: Ben Burton <bab@debian.org>

To: 312847@bugs.debian.org, 311328@bugs.debian.org

Cc: control@bugs.debian.org

Subject: python-pgsql upload pending

Date: Wed, 15 Jun 2005 09:10:42 +1000

tags 312847 + pending
tags 311328 + pending
thanks mate

Hi.  Just a note that fixes for these bugs are ready to go, and will be
uploaded once libpq4 has fixed its unix socket locations (#313602).

b.

Message #32 received at 311328-close@bugs.debian.org (full text, mbox, reply):

From: Ben Burton <bab@debian.org>

To: 311328-close@bugs.debian.org

Subject: Bug#311328: fixed in python-pgsql 2.4.0-6

Date: Mon, 20 Jun 2005 12:36:27 -0400

Source: python-pgsql
Source-Version: 2.4.0-6

We believe that the bug you reported is fixed in the latest version of
python-pgsql, which is due to be installed in the Debian FTP archive:

python-pgsql_2.4.0-6.diff.gz
  to pool/main/p/python-pgsql/python-pgsql_2.4.0-6.diff.gz
python-pgsql_2.4.0-6.dsc
  to pool/main/p/python-pgsql/python-pgsql_2.4.0-6.dsc
python-pgsql_2.4.0-6_all.deb
  to pool/main/p/python-pgsql/python-pgsql_2.4.0-6_all.deb
python2.1-pgsql_2.4.0-6_i386.deb
  to pool/main/p/python-pgsql/python2.1-pgsql_2.4.0-6_i386.deb
python2.2-pgsql_2.4.0-6_i386.deb
  to pool/main/p/python-pgsql/python2.2-pgsql_2.4.0-6_i386.deb
python2.3-pgsql_2.4.0-6_i386.deb
  to pool/main/p/python-pgsql/python2.3-pgsql_2.4.0-6_i386.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 311328@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Ben Burton <bab@debian.org> (supplier of updated python-pgsql package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Tue, 21 Jun 2005 02:06:27 +1000
Source: python-pgsql
Binary: python-pgsql python2.1-pgsql python2.3-pgsql python2.2-pgsql
Architecture: source all i386
Version: 2.4.0-6
Distribution: unstable
Urgency: low
Maintainer: Ben Burton <bab@debian.org>
Changed-By: Ben Burton <bab@debian.org>
Description: 
 python-pgsql - A Python DB-API 2.0 interface to PostgreSQL v7.x
 python2.1-pgsql - A Python DB-API 2.0 interface to PostgreSQL v7.x
 python2.2-pgsql - A Python DB-API 2.0 interface to PostgreSQL v7.x
 python2.3-pgsql - A Python DB-API 2.0 interface to PostgreSQL v7.x
Closes: 311328
Changes: 
 python-pgsql (2.4.0-6) unstable; urgency=low
 .
   * Rebuilt against libpq4 and libpq-dev for the PostgreSQL 8.0 transition.
   * Patched setup.py to use pg_config to determine the PostgreSQL client
     include directory.
   * Fixed parsing of floats under arbitrary locales (closes: #311328).
     This involves calling the locale-agnostic PyOS_ascii_strtod() instead of
     the locale-dependent strtod().  A local copy of PyOS_ascii_strtod() is
     also included since this function is new for python 2.4.  Thanks to
     Martin Pitt for this patch.
Files: 
 087cbc5def4ea9749968d32f3260417b 701 python optional python-pgsql_2.4.0-6.dsc
 234ff85c633d2556be5b2a8613e439ba 13153 python optional python-pgsql_2.4.0-6.diff.gz
 b23f1369e98e69d9ac286b341a11201a 17516 python optional python-pgsql_2.4.0-6_all.deb
 e9df7c2fe2dde052610ac9597d0e1a1d 143130 python optional python2.1-pgsql_2.4.0-6_i386.deb
 34eb320e962f96a0cc72b60fceed3107 145362 python optional python2.2-pgsql_2.4.0-6_i386.deb
 684caa72f39fe667d2e59cd8181e0d00 145402 python optional python2.3-pgsql_2.4.0-6_i386.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFCtundMQNuxza4YcERAs3oAJ9G2Ij386M1gceeeFLTNfpAghRDPQCeLqzh
ZeLZBV5ltX0pLnaYOx4bOZQ=
=zqeq
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.