Debian Bug report logs - #240989
procps: watch doesn't handle UTF-8 or VT100 line-drawing characters properly

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Branden Robinson <branden@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: procps: watch doesn't handle UTF-8 or VT100 line-drawing characters properly

Date: Tue, 30 Mar 2004 03:01:55 -0500

Package: procps
Version: 1:3.2.0-1
Severity: normal
File: /usr/bin/watch

"watch pstree -G" and "watch pstree -U" don't look very good.

-- System Information:
Debian Release: testing/unstable
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'testing')
Architecture: powerpc (ppc)
Kernel: Linux 2.4.25-powerpc
Locale: LANG=C, LC_CTYPE=en_US.UTF-8

Versions of packages procps depends on:
ii  libc6                       2.3.2.ds1-11 GNU C Library: Shared libraries an
ii  libncurses5                 5.4-3        Shared libraries for terminal hand

-- no debconf information

Message #14 received at 240989@bugs.debian.org (full text, mbox, reply):

From: nutzteil@web.de

To: 240989@bugs.debian.org

Subject: is not yet fixed

Date: Thu, 08 May 2008 13:31:51 +0200

Package: procps
Version: 1:3.2.7-8

debian testing/lenny

watch is not able to show german umlauts ä,ö,ü and degree-sign "°", too.

Thanks
christian

_____________________________________________________________________
Unbegrenzter Speicherplatz für Ihr E-Mail Postfach? Jetzt aktivieren!
http://freemail.web.de/club/landingpage.htm/?mc=025555

Message #19 received at 240989@bugs.debian.org (full text, mbox, reply):

From: Γιώργος Πάλλας <gpall@ccf.auth.gr>

To: Debian Bug Tracking System <240989@bugs.debian.org>

Subject: procps: watch does not show UTF-8 encoded command line

Date: Fri, 03 Jul 2009 13:01:06 +0300

Package: procps
Version: 1:3.2.8-1
Severity: normal


I execute:
watch 'ls -la Πάλλας*'

And watch shows at the header:
Every 2.0s: ls -la M-NM- M-NM-,M-NM-;M-NM-;M-NM-1M-O~B* Fri Jul  3 
12:57:16 2009

Could this be fixed?


-- System Information:
Debian Release: squeeze/sid
 APT prefers testing
 APT policy: (500, 'testing')
Architecture: i386 (i686)

Kernel: Linux 2.6.26-2-686 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages procps depends on:
ii  libc6                     2.9-12         GNU C Library: Shared libraries
ii  libncurses5               5.7+20090523-1 shared libraries for 
terminal hand
ii  lsb-base                  3.2-22         Linux Standard Base 3.2 
init scrip

Versions of packages procps recommends:
ii  psmisc                        22.7-1     utilities that use the proc 
file s

procps suggests no packages.

-- no debconf information

Message #24 received at 240989@bugs.debian.org (full text, mbox, reply):

From: Jarrod Lowe <ubuntu@rrod.net>

To: 240989@bugs.debian.org

Subject: procps/watch Unicode patch

Date: Tue, 3 Nov 2009 19:24:27 +0000

[Message part 1 (text/plain, inline)]

I have written a patch for this issue.

Logged in Ubuntu: https://bugs.launchpad.net/ubuntu/+source/procps/+bug/318221

I have tried to contact the upstream, but have failed to do so - the
email addresses in the man page do not seem to be read, and the
mailing list contains only spam.



-- 
Jarrod Lowe

[watch-unicode-3.2.7.patch (text/x-patch, attachment)]

[watch-unicode-3.2.8.patch (text/x-patch, attachment)]

Message #31 received at 240989-quiet@bugs.debian.org (full text, mbox, reply):

From: Torsten Landschoff <t.landschoff@gmx.net>

To: control@bugs.debian.org, 240989-quiet@bugs.debian.org

Subject: Re: Bug#240989: is not yet fixed

Date: Thu, 7 Jan 2010 16:52:49 +0100

found 240989 1:3.2.8-2
thanks

Despite the upstream changelog mentioning that umlauts are now tolerated
by watch, I can still reproduce this problem just by running 

  LC_ALL=de_DE.UTF-8 watch df

Greetings, Torsten 

Message #36 received at 240989@bugs.debian.org (full text, mbox, reply):

From: Uli Martens <uli@youam.net>

To: Jarrod Lowe <ubuntu@rrod.net>, 240989@bugs.debian.org

Cc: control@bugs.debian.org

Subject: Re: Bug#240989: procps/watch Unicode patch

Date: Mon, 11 Jan 2010 10:00:48 +0100

tags 240989 + patch
thanks

Hi Jarrod, hi bts.

On Tue, Nov 03, 2009 at 07:24:27PM +0000, Jarrod Lowe wrote:
> I have written a patch for this issue.

I have mangled your patch into a dpatch file which applies after all other
dpatch patches included in the current Debian package. Seems to work for me,
but no guaranties...

greetings,
	Uli


diff -urN procps-3.2.8/debian/changelog procps-3.2.8~utf8/debian/changelog
--- procps-3.2.8/debian/changelog	2010-01-08 12:49:03.000000000 +0100
+++ procps-3.2.8~utf8/debian/changelog	2010-01-08 12:46:53.000000000 +0100
@@ -1,3 +1,10 @@
+procps (1:3.2.8-2.1) UNRELEASED; urgency=low
+
+  * No upload at all, not even an NMU...
+  * added watch-unicode patch by Jarrod Lowe Closes: #240989
+
+ -- Uli Martens <uli@youam.net>  Fri, 08 Jan 2010 10:12:07 +0100
+
 procps (1:3.2.8-2) unstable; urgency=low
 
   * ps displays supplementary groups Closes: #506303
diff -urN procps-3.2.8/debian/control procps-3.2.8~utf8/debian/control
--- procps-3.2.8/debian/control	2010-01-08 12:49:03.000000000 +0100
+++ procps-3.2.8~utf8/debian/control	2010-01-11 09:51:30.000000000 +0100
@@ -2,7 +2,7 @@
 Section: admin
 Priority: required
 Maintainer: Craig Small <csmall@debian.org>
-Build-Depends: debhelper (>= 7), libncurses5-dev, make (>= 3.78.1-8), dpatch
+Build-Depends: debhelper (>= 7), libncurses5-dev, libncursesw5-dev, make (>= 3.78.1-8), dpatch
 Standards-Version: 3.8.3
 Homepage: http://procps.sf.net/
 
diff -urN procps-3.2.8/debian/patches/00list procps-3.2.8~utf8/debian/patches/00list
--- procps-3.2.8/debian/patches/00list	2010-01-08 12:49:03.000000000 +0100
+++ procps-3.2.8~utf8/debian/patches/00list	2010-01-08 12:46:53.000000000 +0100
@@ -51,3 +51,4 @@
 55_top_highlight
 60_top_nohz
 65_fix_partition_format
+70_watch_unicode
diff -urN procps-3.2.8/debian/patches/70_watch_unicode.dpatch procps-3.2.8~utf8/debian/patches/70_watch_unicode.dpatch
--- procps-3.2.8/debian/patches/70_watch_unicode.dpatch	1970-01-01 01:00:00.000000000 +0100
+++ procps-3.2.8~utf8/debian/patches/70_watch_unicode.dpatch	2010-01-11 09:50:29.000000000 +0100
@@ -0,0 +1,301 @@
+#! /bin/sh /usr/share/dpatch/dpatch-run
+## 70_watch-unicode.dpatch by Jarrod Lowe <procps@rrod.net>
+##
+## All lines beginning with `## DP:' are a description of the patch.
+## DP: add support for unicode characters
+
+@DPATCH@
+diff -urNad procps-3.2.8~/AUTHORS procps-3.2.8/AUTHORS
+--- procps-3.2.8~/AUTHORS	2010-01-08 09:55:26.000000000 +0100
++++ procps-3.2.8/AUTHORS	2010-01-08 09:59:12.000000000 +0100
+@@ -47,4 +47,5 @@
+ watch:
+ Tony Rems <rembo@unisoft.com>
+ Mike Coleman <mkc@acm.org>
++Jarrod Lowe <procps@rrod.net>
+ 
+diff -urNad procps-3.2.8~/Makefile procps-3.2.8/Makefile
+--- procps-3.2.8~/Makefile	2010-01-08 09:58:52.000000000 +0100
++++ procps-3.2.8/Makefile	2010-01-08 10:00:27.000000000 +0100
+@@ -68,6 +68,7 @@
+ _TARFILES := Makefile
+ 
+ CURSES := -lncurses
++CURSESW := -lncursesw
+ 
+ # This seems about right for the dynamic library stuff.
+ # Something like this is probably needed to make the SE Linux
+@@ -119,7 +120,7 @@
+ # Unlike the kernel one, this check_gcc goes all the way to
+ # producing an executable. There might be a -m64 that works
+ # until you go looking for a 64-bit curses library.
+-check_gcc = $(shell if $(CC) $(ALL_CPPFLAGS) $(ALL_CFLAGS) dummy.c $(ALL_LDFLAGS) $(1) -o will_this_file_really_exist.tmp $(CURSES) > /dev/null 2>&1; then echo "$(1)"; else echo "$(2)"; fi ; rm -f will_this_file_really_exist.tmp)
++check_gcc = $(shell if $(CC) $(ALL_CPPFLAGS) $(ALL_CFLAGS) dummy.c $(ALL_LDFLAGS) $(1) -o will_this_file_really_exist.tmp $(CURSES) $(CURSESW) > /dev/null 2>&1; then echo "$(1)"; else echo "$(2)"; fi ; rm -f will_this_file_really_exist.tmp)
+ 
+ # Be 64-bit if at all possible. In a cross-compiling situation, one may
+ # do "make m64=-m32 lib64=lib" to produce 32-bit executables. DO NOT
+@@ -250,7 +251,7 @@
+ 	$(CC) $(ALL_CFLAGS) $^ $(ALL_LDFLAGS) -o $@ $(CURSES)
+ 
+ watch: % : %.o
+-	$(CC) $(ALL_CFLAGS) $^ $(ALL_LDFLAGS) -o $@ $(CURSES)
++	$(CC) $(ALL_CFLAGS) $^ $(ALL_LDFLAGS) -o $@ $(CURSESW)
+ 
+ ############ progX --> progY
+ 
+diff -urNad procps-3.2.8~/watch.1 procps-3.2.8/watch.1
+--- procps-3.2.8~/watch.1	2010-01-08 09:58:52.000000000 +0100
++++ procps-3.2.8/watch.1	2010-01-08 10:04:37.000000000 +0100
+@@ -139,6 +139,16 @@
+ Non-printing characters are stripped from program output.  Use "cat -v" as
+ part of the command pipeline if you want to see them.
+ .PP
++Combining Characters that are supposed to display on the character at the
++last column on the screen may display one column early, or they may not
++display at all.
++.PP
++Combining Characters never count as different in
++.I --differences
++mode. Only the base character counts.
++.PP
++Blank lines directly after a line which ends in the last column do not
++display.
+ .I \-\-precise
+ mode doesn't yet have advanced temporal distortion technology to
+ compensate for a
+@@ -161,6 +171,7 @@
+ corrections by Francois Pinard.  It was reworked and new features added by
+ Mike Coleman <mkc@acm.org> in 1999.  The beep, exec, and error handling 
+ features were added by Morty Abzug <morty@frakir.org> in 2008.
++Unicode support was added in 2009 by Jarrod Lowe <procps@rrod.net>.
+ On a not so dark and stormy morning
+ in March of 2003, Anthony DeRobertis <asd@suespammers.org> got sick of
+ his watches that should update every minute eventually updating many
+diff -urNad procps-3.2.8~/watch.c procps-3.2.8/watch.c
+--- procps-3.2.8~/watch.c	2010-01-08 09:58:52.000000000 +0100
++++ procps-3.2.8/watch.c	2010-01-08 10:09:51.000000000 +0100
+@@ -9,15 +9,17 @@
+  *
+  * Changes by Albert Cahalan, 2002-2003.
+  * stderr handling, exec, and beep option added by Morty Abzug, 2008
++ * Unicode Support added by Jarrod Lowe <procps@rrod.net> in 2009.
+  */
+ 
+-#define VERSION "0.2.0"
++#define VERSION "0.3.0"
+ 
++#include <wchar.h>
+ #include <ctype.h>
+ #include <getopt.h>
+ #include <signal.h>
+-#include <ncurses.h>
+ #include <stdio.h>
++#include <ncursesw/ncurses.h>
+ #include <stdlib.h>
+ #include <string.h>
+ #include <sys/ioctl.h>
+@@ -27,6 +29,7 @@
+ #include <termios.h>
+ #include <locale.h>
+ #include "proc/procps.h"
++#include <errno.h>
+ 
+ #ifdef FORCE_8BIT
+ #undef isprint
+@@ -150,6 +153,32 @@
+ 	return USECS_PER_SEC*now.tv_sec + now.tv_usec;
+ }
+ 
++// read a wide character from a popen'd stream
++#define MAX_ENC_BYTES 16
++wint_t my_getwc(FILE *s);
++wint_t my_getwc(FILE *s) {
++	char i[MAX_ENC_BYTES]; //assuming no encoding ever consumes more than 16 bytes
++	int byte = 0;
++	int convert;
++	int x;
++	wchar_t rval;
++	while(1) {
++		i[byte] = getc(s);
++		if (i[byte]==EOF) { return WEOF; }
++		byte++;
++		errno = 0;
++		mbtowc(NULL, NULL, 0);
++		convert = mbtowc(&rval, i, byte);
++		x = errno;
++		if(convert > 0) { return rval; } //legal conversion
++		if(byte == MAX_ENC_BYTES) {
++		while(byte > 1) { ungetc(i[--byte], s); } //at least *try* to fix up
++		errno = -EILSEQ;
++		return WEOF;
++		}
++	}
++}
++
+ int
+ main(int argc, char *argv[])
+ {
+@@ -162,7 +191,10 @@
+ 	    option_help = 0, option_version = 0;
+ 	double interval = 2;
+ 	char *command;
++	wchar_t *wcommand = NULL;
+ 	char **command_argv;
++	int wcommand_columns = 0;	/* not including final \0 */
++	int wcommand_characters = 0; /* not including final \0 */
+ 	int command_length = 0;	/* not including final \0 */
+     watch_usec_t next_loop; /* next loop time in us, used for precise time
+                                keeping only */
+@@ -259,6 +291,23 @@
+ 		command[command_length] = '\0';
+ 	}
+ 
++	// convert to wide for printing purposes
++	//mbstowcs(NULL, NULL, 0);
++	wcommand_characters = mbstowcs(NULL, command, 0);
++	if(wcommand_characters < 0) {
++		fprintf(stderr, "Unicode Handling Error\n");
++		exit(1);
++	}
++	wcommand = (wchar_t*)malloc((wcommand_characters+1) * sizeof(wcommand));
++	if(wcommand == NULL) {
++		fprintf(stderr, "Unicode Handling Error (malloc)\n");
++		exit(1);
++	}
++	mbstowcs(wcommand, command, wcommand_characters+1);
++	wcommand_columns = wcswidth(wcommand, -1);
++
++
++	
+ 	get_terminal_size();
+ 
+ 	/* Catch keyboard interrupts so we can put tty back in a sane state.  */
+@@ -298,12 +347,44 @@
+ 		if (show_title) {
+ 			// left justify interval and command,
+ 			// right justify time, clipping all to fit window width
+-			asprintf(&header, "Every %.1fs: %.*s",
+-				interval, min(width - 1, command_length), command);
+-			mvaddstr(0, 0, header);
+-			if (strlen(header) > (size_t) (width - tsl - 1))
+-				mvaddstr(0, width - tsl - 4, "...  ");
+-			mvaddstr(0, width - tsl + 1, ts);
++
++			int hlen = asprintf(&header, "Every %.1fs: ", interval);
++
++			// the rules:
++			//   width < tsl : print nothing
++			//   width < tsl + hlen + 1: print ts
++			//   width = tsl + hlen + 1: print header, ts
++			//   width < tsl + hlen + 4: print header, ..., ts
++			//   width < tsl + hlen + wcommand_columns: print header, truncated wcommand, ..., ts
++			//   width > "": print header, wcomand, ts
++			// this is slightly different from how it used to be
++			if(width >= tsl) {
++				if(width >= tsl + hlen + 1) {
++					mvaddstr(0, 0, header);
++					if(width >= tsl + hlen + 2) {
++						if(width < tsl + hlen + 4) {
++							mvaddstr(0, width - tsl - 4, "...  ");
++						}else{
++							if(width < tsl + hlen + wcommand_columns) {
++								// print truncated
++								int avail_columns = width - tsl - hlen;
++								int using_columns = wcommand_columns;
++								int using_characters = wcommand_characters;
++								while(using_columns > avail_columns - 4) {
++									using_characters--;
++								using_columns = wcswidth(wcommand, using_characters);
++								}
++								mvaddnwstr(0, hlen, wcommand, using_characters);
++								mvaddstr(0, width - tsl - 4, "... ");
++							}else{
++								mvaddwstr(0, hlen, wcommand);
++							}
++						}
++					}
++				}
++				mvaddstr(0, width - tsl + 1, ts);
++			}
++			
+ 			free(header);
+ 		}
+ 
+@@ -360,47 +441,62 @@
+ 
+ 		for (y = show_title; y < height; y++) {
+ 			int eolseen = 0, tabpending = 0;
++			wint_t carry = WEOF;
+ 			for (x = 0; x < width; x++) {
+-				int c = ' ';
++				wint_t c = L' ';
+ 				int attr = 0;
+ 
+ 				if (!eolseen) {
+ 					/* if there is a tab pending, just spit spaces until the
+ 					   next stop instead of reading characters */
+ 					if (!tabpending)
+-						do
+-							c = getc(p);
+-						while (c != EOF && !isprint(c)
+-						       && c != '\n'
+-						       && c != '\t');
+-					if (c == '\n')
++						do {
++							if(carry == WEOF) {
++								c = my_getwc(p);
++							}else{
++								c = carry;
++								carry = WEOF;
++							}
++						}while (c != WEOF && !isprint(c) && c<128
++						       && wcwidth(c) == 0
++						       && c != L'\n'
++						       && c != L'\t');
++					if (c == L'\n')
+ 						if (!oldeolseen && x == 0) {
+ 							x = -1;
+ 							continue;
+ 						} else
+ 							eolseen = 1;
+-					else if (c == '\t')
++					else if (c == L'\t')
+ 						tabpending = 1;
+-					if (c == EOF || c == '\n' || c == '\t')
+-						c = ' ';
++					if (x==width-1 && wcwidth(c)==2) {
++						y++;
++						x = -1; //process this double-width
++						carry = c; //character on the next line
++						continue; //because it won't fit here
++					}
++					if (c == WEOF || c == L'\n' || c == L'\t')
++						c = L' ';
+ 					if (tabpending && (((x + 1) % 8) == 0))
+ 						tabpending = 0;
+ 				}
+ 				move(y, x);
+ 				if (option_differences) {
+-					chtype oldch = inch();
+-					unsigned char oldc = oldch & A_CHARTEXT;
++					cchar_t oldc;
++					in_wch(&oldc);
+ 					attr = !first_screen
+-					    && ((char)c != oldc
++					    && ((wchar_t)c != oldc.chars[0]
+ 						||
+ 						(option_differences_cumulative
+-						 && (oldch & A_ATTRIBUTES)));
++						 && (oldc.attr & A_ATTRIBUTES)));
+ 				}
+ 				if (attr)
+ 					standout();
+-				addch(c);
++				addnwstr((wchar_t*)&c,1);
+ 				if (attr)
+ 					standend();
++				if(wcwidth(c) == 0) { x--; }
++				if(wcwidth(c) == 2) { x++; }
+ 			}
+ 			oldeolseen = eolseen;
+ 		}

Message #43 received at 240989-done@bugs.debian.org (full text, mbox, reply):

From: Craig Small <csmall@debian.org>

To: 240989-done@bugs.debian.org

Subject: Fixed in procps 3.2.8-3

Date: Fri, 5 Feb 2010 21:46:09 +1100

Package: procps
Version: 1:3.2.8-3

procps 3.2.8-3 has the extra fixes so that its UTF-8 clean. escape
characters cannot be passed through as it uses ncurses.

 - Craig
-- 
Craig Small      GnuPG:1C1B D893 1418 2AF4 45EE  95CB C76C E5AC 12CA DFA5
http://www.enc.com.au/                             csmall at : enc.com.au
http://www.debian.org/          Debian GNU/Linux, software should be Free 

Send a report that this bug log contains spam.