Debian Bug report logs - #237675
[UTF-8] patch for dselect UTF-8 support

version graph

Package: dselect; Maintainer for dselect is Dpkg Developers <debian-dpkg@lists.debian.org>; Source for dselect is src:dpkg.

Reported by: Eugeniy Meshcheryakov <eugen@univ.kiev.ua>

Date: Fri, 12 Mar 2004 17:48:02 UTC

Severity: normal

Tags: l10n

Merged with 208992, 410855

Found in versions 1.10.10, 1.10.20, dpkg/1.13.25

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Dpkg Development <debian-dpkg@lists.debian.org>:
Bug#237675; Package dselect. Full text and rfc822 format available.

Acknowledgement sent to Eugeniy Meshcheryakov <eugen@univ.kiev.ua>:
New Bug report received and forwarded. Copy sent to Dpkg Development <debian-dpkg@lists.debian.org>. Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Eugeniy Meshcheryakov <eugen@univ.kiev.ua>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: patch for UTF-8 support
Date: Fri, 12 Mar 2004 19:37:32 +0200
[Message part 1 (text/plain, inline)]
Package: dselect
Version: 1.10.20
Severity: normal
Tags: patch

  dselect does not support UTF-8 so I have prepared a patch (attached) to fix it.

What this patch do:
  - compile dselect with libncursesw5 instead of libncurses
  - calculate and use string widths in colums instead of sizes in bytes
    when needed (use standart C99 functions for this).
  - text wrapping code is too complicated for me to rewrite it, so i
    have used libtextwrap for that.

Problems:
  - textwrap hangs if string contains illegal symbols (in UTF-8), this
    is bug #237630. So debconf hangs while displaying description for
    some packages (doc-linux-text-pt -- is it a bug in this package
    too?).
  - text wrapping is different for some packages (manpages - is its'
    dsecription (indentation) correct?)
  - it does not display horisontal lines around section descriptions in
    linux terminal with KOI8-U encoding (but displays them in jfbterm
    and konsole and with UTF-8 encoding -- may be bug in libncursesw?)
  - some format strings was changed, requires new translations.

I have tested this patch with ru_RU and ja_JP locales. I have not found
other problems. Text with both languages looks same with different
encodings (KOI8-R and UTF-8 for Russian, EUC-JP and UTF-8 for Japanese).

-- System Information:
Debian Release: testing/unstable
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: i386 (i686)
Kernel: Linux 2.4.25
Locale: LANG=uk_UA, LC_CTYPE=uk_UA

Versions of packages dselect depends on:
ii  libc6                       2.3.2.ds1-11 GNU C Library: Shared libraries an
ii  libgcc1                     1:3.3.3-2    GCC support library
ii  libncursesw5                5.4-2        Shared libraries for terminal hand
ii  libstdc++5                  1:3.3.3-2    The GNU Standard C++ Library v3
ii  libtextwrap1                0.1-1        text-wrapping library with i18n - 

-- no debconf information
[dselect-unicode.diff (text/plain, attachment)]

Changed Bug title. Request was from Scott James Remnant <scott@netsplit.com> to control@bugs.debian.org. Full text and rfc822 format available.

Changed Bug title. Request was from Scott James Remnant <scott@netsplit.com> to control@bugs.debian.org. Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Development <debian-dpkg@lists.debian.org>:
Bug#237675; Package dselect. Full text and rfc822 format available.

Acknowledgement sent to Changwoo Ryu <cwryu@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Development <debian-dpkg@lists.debian.org>. Full text and rfc822 format available.

Message #14 received at 237675@bugs.debian.org (full text, mbox):

From: Changwoo Ryu <cwryu@debian.org>
To: 237675@bugs.debian.org
Subject: Bug#237675: ok with ko_KR.utf-8
Date: Sun, 20 Jun 2004 19:47:07 +0900
I tested the patch in my ko_KR.UTF-8 locale and it worked very well.
All the UTF-8 breakages seemed to be fixed.






Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Development <debian-dpkg@lists.debian.org>:
Bug#237675; Package dselect. Full text and rfc822 format available.

Acknowledgement sent to Changwoo Ryu <cwryu@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Development <debian-dpkg@lists.debian.org>. Full text and rfc822 format available.

Message #19 received at 237675@bugs.debian.org (full text, mbox):

From: Changwoo Ryu <cwryu@debian.org>
To: 237675@bugs.debian.org
Subject: Re: Bug#237675: ok with ko_KR.utf-8
Date: Sun, 20 Jun 2004 22:08:55 +0900
[Message part 1 (text/plain, inline)]
> I tested the patch in my ko_KR.UTF-8 locale and it worked very well.
> All the UTF-8 breakages seemed to be fixed.
> 
> 

Oops, there were some truncated strings by the multibyte-unaware use of 
*addnstr() functions.

New patch attached: I wrote some replacement functions for those
functions to fix them.


[dselect-unicode.diff (text/x-patch, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Development <debian-dpkg@lists.debian.org>:
Bug#237675; Package dselect. Full text and rfc822 format available.

Acknowledgement sent to Changwoo Ryu <cwryu@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Development <debian-dpkg@lists.debian.org>. Full text and rfc822 format available.

Message #24 received at 237675@bugs.debian.org (full text, mbox):

From: Changwoo Ryu <cwryu@debian.org>
To: 237675@bugs.debian.org
Subject: Bug#237675: new multibyte patch
Date: Wed, 23 Jun 2004 04:25:31 +0900
[Message part 1 (text/plain, inline)]
> New patch attached: I wrote some replacement functions for those
> functions to fix them.

A bit more simplified patch attached.


[dselect-unicode.diff (text/x-patch, attachment)]

Tags added: l10n Request was from Christian Perrier <bubulle@debian.org> to control@bugs.debian.org. Full text and rfc822 format available.

Tags removed: l10n Request was from Changwoo Ryu <cwryu@debian.org> to control@bugs.debian.org. Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Development <debian-dpkg@lists.debian.org>:
Bug#237675; Package dselect. Full text and rfc822 format available.

Acknowledgement sent to Changwoo Ryu <cwryu@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Development <debian-dpkg@lists.debian.org>. Full text and rfc822 format available.

Message #33 received at 237675@bugs.debian.org (full text, mbox):

From: Changwoo Ryu <cwryu@debian.org>
To: 237675@bugs.debian.org
Subject: Re: Bug#237675: [UTF-8] patch for dselect UTF-8 support
Date: Thu, 3 Mar 2005 16:18:20 +0900
[Message part 1 (text/plain, inline)]
If the patch is too long to be accepted for now, how about starting
with replacing libncurses5 with libncursesw5?  Just replacing, without
touching other things, is also useful.

The only problem is that libncursesw5 is not in base...  but 
after sarge all programs can be replaced.

-- 
Changwoo Ryu
[dpkg-ncursesw.diff (text/plain, attachment)]

Forcibly Merged 208992 237675. Request was from Bruce Sass <bmsass@shaw.ca> to control@bugs.debian.org. Full text and rfc822 format available.

Merged 208992 237675 410855. Request was from Nicolas François <nicolas.francois@centraliens.net> to control@bugs.debian.org. Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#237675; Package dselect. Full text and rfc822 format available.

Acknowledgement sent to Raphael Hertzog <hertzog@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. Full text and rfc822 format available.

Message #42 received at 237675@bugs.debian.org (full text, mbox):

From: Raphael Hertzog <hertzog@debian.org>
To: Changwoo Ryu <cwryu@debian.org>
Cc: 237675@bugs.debian.org, Eugeniy Meshcheryakov <eugen@univ.kiev.ua>
Subject: Re: Bug#237675: [UTF-8] patch for dselect UTF-8 support
Date: Fri, 27 Jun 2008 23:14:45 +0200
[Message part 1 (text/plain, inline)]
On Thu, 03 Mar 2005, Changwoo Ryu wrote:
> If the patch is too long to be accepted for now, how about starting
> with replacing libncurses5 with libncursesw5?  Just replacing, without
> touching other things, is also useful.
> 
> The only problem is that libncursesw5 is not in base...  but 
> after sarge all programs can be replaced.

This has been done some time ago when it really broke badly. For the
rest of your patch, I just forward-ported it to the current git tree,
the fix for #342495 was in conflict with your patch and had to be redone
given that you use libtextwrap. Apart from that and from some build-system
adjustement, the patch applied mostly fine.

I built an udpated dselect and it seems to work as expected.

I'm a bit uneasy however with adding a libtextwrap dependency on dpkg for
dpkg. Given that dselect can be completely disabled in the build system it
doesn't complicate the bootstrap process of a new architecture but still
it would be nice if someone could just fix the code to not require it at
all.

Please test and keep me informed! We can then merge the patch if it
doesn't have any other side-effects.

Cheers,
-- 
Raphaël Hertzog

Le best-seller français mis à jour pour Debian Etch :
http://www.ouaza.com/livre/admin-debian/
[0001-Add-wide-character-support-to-dselect.patch (text/x-diff, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#237675; Package dselect. Full text and rfc822 format available.

Acknowledgement sent to Raphael Hertzog <hertzog@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. Full text and rfc822 format available.

Message #47 received at 237675@bugs.debian.org (full text, mbox):

From: Raphael Hertzog <hertzog@debian.org>
To: Changwoo Ryu <cwryu@debian.org>
Cc: 237675@bugs.debian.org, Eugeniy Meshcheryakov <eugen@univ.kiev.ua>
Subject: Re: Bug#237675: [UTF-8] patch for dselect UTF-8 support
Date: Sun, 6 Jul 2008 21:32:49 +0200
On Fri, 27 Jun 2008, Raphael Hertzog wrote:
> On Thu, 03 Mar 2005, Changwoo Ryu wrote:
> > If the patch is too long to be accepted for now, how about starting
> > with replacing libncurses5 with libncursesw5?  Just replacing, without
> > touching other things, is also useful.
> > 
> > The only problem is that libncursesw5 is not in base...  but 
> > after sarge all programs can be replaced.
> 
> This has been done some time ago when it really broke badly. For the
> rest of your patch, I just forward-ported it to the current git tree,
> the fix for #342495 was in conflict with your patch and had to be redone
> given that you use libtextwrap. Apart from that and from some build-system
> adjustement, the patch applied mostly fine.

Some other conflicting changes have been merged in the mean time. I
reupdated the patch and I'm keeping a branch here:
http://git.debian.org/?p=users/hertzog/dpkg.git;a=shortlog;h=refs/heads/pu/bug237675-dselect-wide-char-support

Cheers,
-- 
Raphaël Hertzog

Le best-seller français mis à jour pour Debian Etch :
http://www.ouaza.com/livre/admin-debian/




Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#237675; Package dselect. Full text and rfc822 format available.

Acknowledgement sent to "Yuri Kozlov" <kozlov.y@gmail.com>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. Full text and rfc822 format available.

Message #52 received at 237675@bugs.debian.org (full text, mbox):

From: "Yuri Kozlov" <kozlov.y@gmail.com>
To: 237675@bugs.debian.org
Subject: Re: Bug#237675: [UTF-8] patch for dselect UTF-8 support
Date: Wed, 23 Jul 2008 21:50:18 +0400
[Message part 1 (text/plain, inline)]
Hello.

1.14.20 version have truncated menu (locale is ru_RU.UTF-8).

-- 
Regards,
Yuri Kozlov
[dselect.png (image/png, attachment)]

Removed tag(s) patch. Request was from Raphaël Hertzog <hertzog@debian.org> to control@bugs.debian.org. (Thu, 06 May 2010 13:33:10 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#237675; Package dselect. (Thu, 16 Sep 2010 14:00:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Michal Suchanek <hramrach@centrum.cz>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Thu, 16 Sep 2010 14:00:03 GMT) Full text and rfc822 format available.

Message #59 received at 237675@bugs.debian.org (full text, mbox):

From: Michal Suchanek <hramrach@centrum.cz>
To: 237675@bugs.debian.org
Cc: Yuri Kozlov <kozlov.y@gmail.com>
Subject: UTF-8 string length calculated incorrectly in dselect
Date: Thu, 16 Sep 2010 15:56:14 +0200
[Message part 1 (text/plain, inline)]
Hello

I would say this is only cosmetic issue now. Or is there a place where
 dselect fails due to this issue?

Apparently dslelct calculates the string length incorrectly and the
strings are truncated more or less depending on the amount of
multibyte characters included but it errs on the side of safety and
displays shorter strings than would fit.

Thanks

Michal
[gkrellShoot_2010-09-16_154709.png (image/png, attachment)]
[gkrellShoot_2010-09-16_154635.png (image/png, attachment)]
[gkrellShoot_2010-09-16_154125.png (image/png, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#237675; Package dselect. (Thu, 16 Sep 2010 14:57:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to Yuri Kozlov <yuray@komyakino.ru>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Thu, 16 Sep 2010 14:57:06 GMT) Full text and rfc822 format available.

Message #64 received at 237675@bugs.debian.org (full text, mbox):

From: Yuri Kozlov <yuray@komyakino.ru>
To: 237675@bugs.debian.org
Subject: UTF-8 string length calculated incorrectly in dselect
Date: Thu, 16 Sep 2010 18:50:45 +0400
Hello.

Yes, it is only cosmetic issue (dselect 1.15.8.4).
But software with utf-8 problems seems questionably in 2010 year.

-- 
Best Regards,
Yuri Kozlov





Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#237675; Package dselect. (Thu, 16 Sep 2010 18:24:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Christian PERRIER <bubulle@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Thu, 16 Sep 2010 18:24:03 GMT) Full text and rfc822 format available.

Message #69 received at 237675@bugs.debian.org (full text, mbox):

From: Christian PERRIER <bubulle@debian.org>
To: Yuri Kozlov <yuray@komyakino.ru>, 237675@bugs.debian.org
Subject: Re: Bug#237675: UTF-8 string length calculated incorrectly in dselect
Date: Thu, 16 Sep 2010 20:19:45 +0200
[Message part 1 (text/plain, inline)]
Quoting Yuri Kozlov (yuray@komyakino.ru):
> Hello.
> 
> Yes, it is only cosmetic issue (dselect 1.15.8.4).
> But software with utf-8 problems seems questionably in 2010 year.

dselect is questionable in 2010..:-)


[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Wed Apr 16 23:42:43 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.