Debian Bug report logs - #257540
bash: Prompt ignores UTF-8 character width

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Dmitry Azhichakov <admitry@rol.ru>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: bash: Prompt ignores UTF-8 character width

Date: Sun, 04 Jul 2004 09:05:03 +0400

Package: bash
Version: 2.05b-15
Severity: normal
Tags: l10n

Prompt uses byte width not character width. I encountered this behavior
with Russian directory names and '\w' in PS1. This produce many annoying
glitches when browsing history.

-- System Information:
Debian Release: testing/unstable
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: i386 (i686)
Kernel: Linux 2.4.25-p4
Locale: LANG=ru_RU.UTF-8, LC_CTYPE=ru_RU.UTF-8

Versions of packages bash depends on:
ii  base-files                  3.0.16       Debian base system miscellaneous f
ii  libc6                       2.3.2.ds1-13 GNU C Library: Shared libraries an
ii  libncurses5                 5.4-4        Shared libraries for terminal hand

-- no debconf information

Message #10 received at 257540@bugs.debian.org (full text, mbox, reply):

From: Tomohiro KUBOTA <debian@tmail.plala.or.jp>

To: 257540@bugs.debian.org

Subject: Re: bash: Prompt ignores UTF-8 character width

Date: Sat, 31 Jul 2004 18:05:21 +0900 (LDT)

Hi,

This bug is a bit important than people would think, because
prompt of bash (or readline library) is used for debconf's
"Readline" interface.  In this case, bash's prompt is set to
debconf's question message (translatable).

This bug may be same as #195678.

---
Tomohiro KUBOTA <kubota@debian.org>
http://www.debian.or.jp/~kubota/

Message #15 received at 257540@bugs.debian.org (full text, mbox, reply):

From: Matthias Klose <doko@cs.tu-berlin.de>

To: Tomohiro KUBOTA <debian@tmail.plala.or.jp>, 257540@bugs.debian.org

Subject: Re: Bug#257540: bash: Prompt ignores UTF-8 character width

Date: Sat, 31 Jul 2004 14:00:30 +0200

Tomohiro KUBOTA writes:
> Hi,
> 
> This bug is a bit important than people would think, because
> prompt of bash (or readline library) is used for debconf's
> "Readline" interface.  In this case, bash's prompt is set to
> debconf's question message (translatable).
> 
> This bug may be same as #195678.

please could you check out the readline5 packages at
http://people.debian.org/~doko/bash/ ? you must install them using
dpkg - --force-overwrite, they are not perfect yet. Anyway, then
you'll need a debconf rebuild as well.

Thanks, Matthias

Message #20 received at 257540@bugs.debian.org (full text, mbox, reply):

From: Tomohiro KUBOTA <debian@tmail.plala.or.jp>

To: doko@cs.tu-berlin.de

Cc: 257540@bugs.debian.org

Subject: Re: Bug#257540: bash: Prompt ignores UTF-8 character width

Date: Mon, 02 Aug 2004 14:23:07 +0900 (LDT)

Hi,

From: Matthias Klose <doko@cs.tu-berlin.de>
Subject: Re: Bug#257540: bash: Prompt ignores UTF-8 character width
Date: Sat, 31 Jul 2004 14:00:30 +0200

> please could you check out the readline5 packages at
> http://people.debian.org/~doko/bash/ ? you must install them using
> dpkg - --force-overwrite, they are not perfect yet. Anyway, then
> you'll need a debconf rebuild as well.

I tested it but the problem was not solved.

How to test (on UTF-8 terminal with 80-column width):

$ LANG=en_US.UTF-8  (or other UTF-8 locale)
$ PS1=`perl -e 'print "\xe7\x94\xb0" x 35'`

\xe7\x94\xb0 is a UTF-8 character.  It is CJK Ideogram "rice field"
which looks like a window.  It is a double-width character and therefore
the prompt occupies 70 columns.

Then type more than 45 characters after the prompt.  The line will be
folded at 45-character point because the length of the prompt is
wrongly calculated as 35.

Next, push Backspace key many times to erase your input.  When you erase
10th - 45th characters, the whole line (beginning from the prompt) will
be rewritten at next line.  I cannot explain the situation very well
in English.  Please try yourself.


I found a patch for this problem at Slashdot Japan.
    http://slashdot.jp/comments.pl?sid=199321&cid=599293
I tested this and found that the above problem is fixed but it still
fails when the prompt itself exceeds 80 columns.

Please inform this upstream.

---
Tomohiro KUBOTA <kubota@debian.org>
http://www.debian.or.jp/~kubota/

Message #25 received at 257540@bugs.debian.org (full text, mbox, reply):

From: Tomohiro KUBOTA <debian@tmail.plala.or.jp>

To: doko@cs.tu-berlin.de

Cc: 257540@bugs.debian.org

Subject: Re: Bug#257540: bash: Prompt ignores UTF-8 character width

Date: Wed, 18 Aug 2004 12:17:17 +0900 (LDT)

Hi,

From: Tomohiro KUBOTA <debian@tmail.plala.or.jp>
Subject: Re: Bug#257540: bash: Prompt ignores UTF-8 character width
Date: Mon, 02 Aug 2004 14:23:07 +0900 (LDT)

> I found a patch for this problem at Slashdot Japan.
>     http://slashdot.jp/comments.pl?sid=199321&cid=599293
> I tested this and found that the above problem is fixed but it still
> fails when the prompt itself exceeds 80 columns.

A renewed version of the patch is available:

   http://slashdot.jp/comments.pl?sid=199321&cid=601375

---
Tomohiro KUBOTA <kubota@debian.org>
http://www.debian.or.jp/~kubota/

Message #30 received at 257540@bugs.debian.org (full text, mbox, reply):

From: Matthias Klose <doko@cs.tu-berlin.de>

To: chet@po.cwru.edu, 257540@bugs.debian.org

Subject: Bug#257540: bash: Prompt ignores UTF-8 character width (forwarded from Tomohiro KUBOTA)

Date: Sat, 28 Aug 2004 17:16:58 +0200

here is the patch from the website.

--- bash-3.0/lib/readline/display.c.org	2004-05-28 11:57:51.000000000 +0900
+++ bash-3.0/lib/readline/display.c	2004-08-04 23:08:56.493454504 +0900
@@ -249,7 +249,10 @@
 	      while (l--)
 	        *r++ = *p++;
 	      if (!ignoring)
-		rl += ind - pind;
+		{
+		  rl += ind - pind;
+		  physchars += _rl_col_width(pmt, pind, ind);
+		}
 	      else
 		ninvis += ind - pind;
 	      p--;			/* compensate for later increment */
@@ -259,16 +262,16 @@
 	    {
 	      *r++ = *p;
 	      if (!ignoring)
-		rl++;			/* visible length byte counter */
+		{
+		  rl++;			/* visible length byte counter */
+		  physchars++;
+		}
 	      else
 		ninvis++;		/* invisible chars byte counter */
 	    }
 
 	  if (rl >= _rl_screenwidth)
 	    invfl = ninvis;
-
-	  if (ignoring == 0)
-	    physchars++;
 	}
     }
 
@@ -573,6 +576,7 @@
 
 #if defined (HANDLE_MULTIBYTE)
   memset (_rl_wrapped_line, 0, vis_lbsize);
+  int num = 0;
 #endif
 
   /* prompt_invis_chars_first_line is the number of invisible characters in
@@ -591,13 +595,32 @@
          probably too much work for the benefit gained.  How many people have
          prompts that exceed two physical lines?
          Additional logic fix from Edward Catmur <ed@catmur.co.uk> */
+#if defined (HANDLE_MULTIBYTE)
+      int num0 = num;
+      num += _rl_screenwidth;
+      while (num < strlen(local_prompt))
+        {
+          if (_rl_col_width(local_prompt, num0, num) > _rl_screenwidth)
+            {
+              num = _rl_find_prev_mbchar(local_prompt, num, MB_FIND_ANY);
+              break;
+            }
+          num++;
+        }
+      temp = num +
+#else
       temp = ((newlines + 1) * _rl_screenwidth) +
+#endif
              ((local_prompt_prefix == 0) ? ((newlines == 0) ? prompt_invis_chars_first_line
 							    : ((newlines == 1) ? wrap_offset : 0))
 					 : ((newlines == 0) ? wrap_offset :0));
              
       inv_lbreaks[++newlines] = temp;
+#if defined (HANDLE_MULTIBYTE)
+      lpos -= _rl_col_width(local_prompt, num0, num);
+#else
       lpos -= _rl_screenwidth;
+#endif
     }
 
   prompt_last_screen_line = newlines;

Message #35 received at 257540@bugs.debian.org (full text, mbox, reply):

From: Matthias Klose <doko@cs.tu-berlin.de>

To: Tomohiro KUBOTA <debian@tmail.plala.or.jp>, 257540@bugs.debian.org

Subject: Re: Bug#257540: bash: Prompt ignores UTF-8 character width

Date: Sat, 28 Aug 2004 17:17:54 +0200

Tomohiro KUBOTA writes:
> Hi,
> 
> From: Tomohiro KUBOTA <debian@tmail.plala.or.jp>
> Subject: Re: Bug#257540: bash: Prompt ignores UTF-8 character width
> Date: Mon, 02 Aug 2004 14:23:07 +0900 (LDT)
> 
> > I found a patch for this problem at Slashdot Japan.
> >     http://slashdot.jp/comments.pl?sid=199321&cid=599293
> > I tested this and found that the above problem is fixed but it still
> > fails when the prompt itself exceeds 80 columns.
> 
> A renewed version of the patch is available:
> 
>    http://slashdot.jp/comments.pl?sid=199321&cid=601375

please could you check (or translate ;) if thats's needed/available
for bash-3.0 as well?

Thanks, Matthias

Message #40 received at 257540@bugs.debian.org (full text, mbox, reply):

From: Tomohiro KUBOTA <debian@tmail.plala.or.jp>

To: doko@cs.tu-berlin.de

Cc: 257540@bugs.debian.org

Subject: Re: Bug#257540: bash: Prompt ignores UTF-8 character width

Date: Sun, 29 Aug 2004 01:09:20 +0900 (LDT)

Hi,

From: Matthias Klose <doko@cs.tu-berlin.de>
Subject: Re: Bug#257540: bash: Prompt ignores UTF-8 character width
Date: Sat, 28 Aug 2004 17:17:54 +0200


> >    http://slashdot.jp/comments.pl?sid=199321&cid=601375
> 
> please could you check (or translate ;) if thats's needed/available
> for bash-3.0 as well?

The message reads:

    The previous patch had a bug (possibility of infinite loop).
    Here is a fixed version.

The whole story of Slashdot Japan is for bash-3.0 and the patch
was written for bash-3.0 (not bash-2.x).

---
Tomohiro KUBOTA <kubota@debian.org>
http://www.debian.or.jp/~kubota/

Message #49 received at 257540-close@bugs.debian.org (full text, mbox, reply):

From: Matthias Klose <doko@debian.org>

To: 257540-close@bugs.debian.org

Subject: Bug#257540: fixed in bash 3.0-6

Date: Sat, 04 Sep 2004 05:47:05 -0400

Source: bash
Source-Version: 3.0-6

We believe that the bug you reported is fixed in the latest version of
bash, which is due to be installed in the Debian FTP archive:

bash-builtins_3.0-6_i386.deb
  to pool/main/b/bash/bash-builtins_3.0-6_i386.deb
bash-doc_3.0-6_all.deb
  to pool/main/b/bash/bash-doc_3.0-6_all.deb
bash-minimal_3.0-6_i386.deb
  to pool/main/b/bash/bash-minimal_3.0-6_i386.deb
bash-static_3.0-6_i386.deb
  to pool/main/b/bash/bash-static_3.0-6_i386.deb
bash_3.0-6.diff.gz
  to pool/main/b/bash/bash_3.0-6.diff.gz
bash_3.0-6.dsc
  to pool/main/b/bash/bash_3.0-6.dsc
bash_3.0-6_i386.deb
  to pool/main/b/bash/bash_3.0-6_i386.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 257540@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Matthias Klose <doko@debian.org> (supplier of updated bash package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Sat,  4 Sep 2004 10:21:23 +0200
Source: bash
Binary: bashdb bash-builtins bash bash-minimal bash-static bash-doc
Architecture: source i386 all
Version: 3.0-6
Distribution: unstable
Urgency: low
Maintainer: Matthias Klose <doko@debian.org>
Changed-By: Matthias Klose <doko@debian.org>
Description: 
 bash       - The GNU Bourne Again SHell
 bash-builtins - Bash loadable builtins - headers & examples
 bash-doc   - Documentation and examples for the The GNU Bourne Again SHell
 bash-minimal - The GNU Bourne Again SHell (minimal version)
 bash-static - The GNU Bourne Again SHell (static version)
Closes: 7047 10494 141292 161057 187983 192831 203767 224916 243015 253766 257540 261142 262105 265259 265982 268922
Changes: 
 bash (3.0-6) unstable; urgency=low
 .
   * Fixed in bash-3.0:
     - New option pipefail. If set, the return value of a pipeline is the
       value of the last (rightmost) command to exit with a non-zero status,
       or zero if all commands in the pipeline exit successfully.  This option
       is disabled by default. The two oldest outstanding bash reports are
       gone (closes: #7047, #10494).
     - "$( substitution strangeness (closes: #187983).
     - timestamp support in history (closes: #161057).
   * Various updates from the net:
     - Fix a bug in array expansion.
     - Fix a bug in brace expansion.
     - Handle multibyte characters in IFS values.
     - Fix a bug, when pipefail option is set.
     - History saved-line handling (closes: #253766, #268922).
     - Fix parameter expansion with UTF-8 and ${#var} or ${var: -1}.
   * Apply patch to fix non POSIX function name and avoid use of global
     variables (Stephen Gildea). Closes: #262105.
   * Add command_not_found_handle in non POSIX interactive shells.
     Closes: #243015.
   * Check for /usr/sbin/remove-shell before using it (closes: #265982).
   * bash-builtins: Install missing header files (closes: #265259).
   * Clarify documentation about substring expansion (closes: #192831, #203767).
   * Fix completion on ~/../``/<tab> (closes: #261142).
   * Clarify documentation about case-insensitive pathname expansion.
     Closes: #141292.
   * Apply patch to fix the display of UTF-8 characters.
     Closes: #224916, #257540.
Files: 
 8c00f265c7a5ddeda7f2ea0489f9e433 723 base required bash_3.0-6.dsc
 072502889a21e9d6d9067f8cf4ffc45e 165836 base required bash_3.0-6.diff.gz
 d59c67c2503e505c746ab7cf14022422 714274 doc optional bash-doc_3.0-6_all.deb
 74b051f137b7d3f8288258ce4437c9d0 782020 base required bash_3.0-6_i386.deb
 18943c6dbf48f974376cd42c303353ba 97864 utils optional bash-builtins_3.0-6_i386.deb
 40a482fa65d8103135f6e6091ae32bc3 625378 shells optional bash-static_3.0-6_i386.deb
 d524037e925afd8b2c4032654723d276 181000 shells optional bash-minimal_3.0-6_i386.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)

iD8DBQFBOYl+StlRaw+TLJwRAhrQAJ9x1pVM5jKATOpje7I+6SlqonZVtgCePK5p
2PZ/sP7ecEOTD4wGdrrfBck=
=CZX8
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.