Debian Bug report logs - #174527
tin: broken multi-byte character set entries

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: "Chuan-kai Lin" <b86506063@ntu.edu.tw>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: tin: broken multi-byte character set entries

Date: Sat, 28 Dec 2002 16:12:28 +0800

Package: tin
Version: 1:1.5.14+20020917-1
Severity: normal

Well, this bug has been observed in tin for quite a long time.

The problem goes like this: whenever tin prompts you to enter some
information in the bottom of the terminal (like, when you want to
conduct a search), entering multi-byte characters (Big5 in my case)
always result in garbage, regardless of locale settings.

Why does the problem exist?  All input characters are filtered through
the my_isprint function in misc.c, which in turn invokes the isprint
function in the C library.  Normally this should not be a problem,
because the isprint function in the C library is now locale aware --
but the catch is that it works for single-byte character sets only.
You see, isprint process one byte at a time, and any single byte in a
multi-byte character set is naturally not printable.

There are, in my opinion, two ways to solve this problem: the easy
way and the right way.  The easy way is to make special cases for all
multi-byte characters so that my_isprint always returns true (as
currently done when NO_LOCALE is defined at compile time).  The right
way is to convert the entire program to make use of wide characters
(wchar_t and wint_t) and replace isprint to iswprint.

I think this one should be forwarded to upstream.

-- System Information
Debian Release: 3.0
Architecture: i386
Kernel: Linux debian 2.4.19 #3 SMP Fri Nov 22 01:13:13 CST 2002 i686
Locale: LANG=C, LC_CTYPE=zh_TW.Big5

Versions of packages tin depends on:
ii  debconf                  1.2.20          Debian configuration management sy
ii  libc6                    2.2.5-14.3      GNU C Library: Shared libraries an
ii  libncurses5              5.2.20020112a-8 Shared libraries for terminal hand
ii  libpcre3                 3.4-1.1         Philip Hazel's Perl Compatible Reg

Message #10 received at 174527@bugs.debian.org (full text, mbox, reply):

From: Andreas Metzler <ametzler@downhill.at.eu.org>

To: 174527@bugs.debian.org

Cc: Chuan-kai Lin <b86506063@ntu.edu.tw>

Subject: Re: Bug#174527: tin: broken multi-byte character set entries

Date: Sat, 28 Dec 2002 11:36:10 +0100

Chuan-kai Lin schrieb:
[...]
> The problem goes like this: whenever tin prompts you to enter some
> information in the bottom of the terminal (like, when you want to
> conduct a search), entering multi-byte characters (Big5 in my case)
> always result in garbage, regardless of locale settings.
[...]
> There are, in my opinion, two ways to solve this problem: the easy
> way and the right way.  The easy way is to make special cases for all
> multi-byte characters so that my_isprint always returns true (as
> currently done when NO_LOCALE is defined at compile time).  The right
> way is to convert the entire program to make use of wide characters
> (wchar_t and wint_t) and replace isprint to iswprint.

> I think this one should be forwarded to upstream.

Just FYI: This problem is known and conversion to wchar (and ncursesw) 
has aleady been started some months ago.

IIRC just two weeks ago somebody attacked the problem you discussed, 
that it is impossible to enter no multi-byte chatacters. (I am sorry I 
cannot give him him credit, I haven't got my mails with me and can't 
look it up.
         thanks, cu andreas

Message #19 received at 174527@bugs.debian.org (full text, mbox, reply):

From: Marco d'Itri <md@Linux.IT>

To: Chuan-kai Lin <b86506063@ntu.edu.tw>, 174527@bugs.debian.org

Subject: Re: Bug#174527: tin: broken multi-byte character set entries

Date: Sun, 15 Jun 2003 23:19:39 +0200

I uploaded a new tin package which should fix this bug.
Please let me know how it works.

-- 
ciao, |
Marco | [1536 unyrq0dSfrvwQ]

Message #24 received at 174527@bugs.debian.org (full text, mbox, reply):

From: Chuan-kai Lin <b86506063@ntu.edu.tw>

To: Bug #174527 <174527@bugs.debian.org>

Subject: Re: Bug#174527: tin: broken multi-byte character set entries

Date: Mon, 16 Jun 2003 17:18:01 +0800

On Sun, Jun 15, 2003 at 11:19:39PM +0200, Marco d'Itri wrote:
> I uploaded a new tin package which should fix this bug.
> Please let me know how it works.

I have installed the 1:1.5.19+20030609-1 package.  The getline function
now indeed makes good use of wide character support in the C library as
well as ncurses, but there are still some problems:

 1. When entering wide characters (Big5 Chinese with LC_CTYPE=zh_TW.Big5
    in my case), the cursor always advances one character too many.
    Since the gl_fixup function in getline.c keeps track of cursor
    positions through its own internal model and without actually
    checking through ncurses to see if the cursor is indeed where it is
    supposed to be, the whole text entry area tend to turn into garbage
    after a few text manipulation or cursor movement commands.  I
    believe that this is caused by a bug in the ncurses library itself,
    and I have written to the ncurses developers for clarifications.

 2. It appears that tin_getline and friends still does not clearly
    differentiate between the length of string measured as the number of
    (wide) characters in it, and the length of string measured as the
    number of ASCII character cells it would occupy when displayed on
    the terminal.  In the ASCII-only and multibyte character cases, both
    of these would be the same, but when wide characters are used, they
    need not have any trivial relationships: a wide character can take
    up one or two ASCII character cells on the terminal.  This should be
    looked into and fixed as well.

 3. When fixing problem 2, please make sure that cursor movement are
    wide-character aware as well, eg. the left arrow key should move the
    cursor across exactly one character regardless of the displayed
    width of the character.

Perhaps it is a good idea to make use of GNU readline for text entry.
AFAIK it supports all the current features offered by tin_getline, works
with wide characters quite gracefully, and is free from the bugs
described above.  Having one less module to maintain should also be
beneficial from the developer's point of view. (I guess you will help me
forward this message upstream?)

Please contact me if you need any more information.

Regards,

-- 
Chuan-kai Lin
http://www.csie.ntu.edu.tw/~b6506063/

Message #29 received at 174527@bugs.debian.org (full text, mbox, reply):

From: Urs Janßen <urs@tin.org>

To: Bug #174527 <174527@bugs.debian.org>

Subject: Re: Bug#174527: tin: broken multi-byte character set entries

Date: Mon, 1 May 2006 04:36:37 +0200

the multibyte problems in getline.c should have been fixed in
tin >= 1.7.6

urs
-- 
"Only whimps use tape backup: _real_ men just upload their important stuff
 on ftp, and let the rest of the world mirror it ;)" - Linus

Send a report that this bug log contains spam.