Debian Bug report logs - #387704
grep: -i breaks \W in some locales (perhaps UTF-8 locales only)

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Christoph Biedl <cbiedl@gmx.de>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: grep: -i breaks \W in some locales (perhaps UTF-8 locales only)

Date: Sat, 16 Sep 2006 10:35:26 +0200

[Message part 1 (text/plain, inline)]

Package: grep
Version: 2.5.1.ds2-5
Severity: normal

I noticed that enabling --ignore-case suddenly caused certain patterns
not to match any longer although they should:

$ echo 'foo bar' | grep    '^foo\W'
foo bar
$ echo 'foo bar' | grep -i '^foo\W'
$

Digging further reveals that there's an locales influence since
$ echo 'foo bar' | LANG=C grep -i '^foo\W'
foo bar
$

matches again. After a check using all my generated locales:

MATCH:
- de_DE
- de_DE@euro
- en_US

FAIL:
- de_DE.UTF-8
- de_DE.UTF-8@euro
- en_US.UTF-8

there's a strong impression that UTF-8 locales somehow disturb \W when
using -i.

Even more confusing, using the bracket expression instead of the synonym
matches again:
$ echo 'foo bar' | LANG=de_DE.UTF-8 grep -i '^foo[^[:alnum:]]'
foo bar
$

For the records, this sounds somewhat similar to #209194 and #218873 but
these bugs are fixed in this version (2.5.1.ds2-5), I've checked.

By the way, there's a typo in the manpage

  and
  .B \eW
  is a synonym for
- .BR [^[:alnum]] .
+ .BR [^[:alnum:]] .
  .PP

-- System Information:
Debian Release: testing/unstable
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: i386 (i686)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.6.17.13
Locale: LANG=de_DE.UTF-8@euro, LC_CTYPE=de_DE.UTF-8@euro (charmap=UTF-8)

Versions of packages grep depends on:
ii  libc6                        2.3.6.ds1-4 GNU C Library: Shared libraries

grep recommends no packages.

-- no debconf information

[signature.asc (application/pgp-signature, inline)]

Message #10 received at 387704@bugs.debian.org (full text, mbox, reply):

From: Ruben Molina <rmolina@udea.edu.co>

To: control@bugs.debian.org

Cc: 387704@bugs.debian.org

Subject: found 387704 in 2.5.3~dfsg-6

Date: Sun, 11 Jan 2009 11:14:07 -0500

[Message part 1 (text/plain, inline)]

tags  387704  + confirmed
found 387704 2.5.3~dfsg-6
thanks

$ locale
LANG=es_CO.UTF-8
LC_CTYPE="es_CO.UTF-8"
LC_NUMERIC="es_CO.UTF-8"
LC_TIME="es_CO.UTF-8"
LC_COLLATE="es_CO.UTF-8"
LC_MONETARY="es_CO.UTF-8"
LC_MESSAGES="es_CO.UTF-8"
LC_PAPER="es_CO.UTF-8"
LC_NAME="es_CO.UTF-8"
LC_ADDRESS="es_CO.UTF-8"
LC_TELEPHONE="es_CO.UTF-8"
LC_MEASUREMENT="es_CO.UTF-8"
LC_IDENTIFICATION="es_CO.UTF-8"
LC_ALL=

$ echo 'foo bar' | grep    '^foo\W'
foo bar
$

$ echo 'foo bar' | grep -i '^foo\W'
$

$ echo 'foo bar' | LANG=C grep -i '^foo\W'
foo bar
$

[signature.asc (application/pgp-signature, inline)]

Message #19 received at 387704@bugs.debian.org (full text, mbox, reply):

From: Aníbal Monsalve Salazar <anibal@debian.org>

To: bug-grep@gnu.org

Cc: Christoph Biedl <cbiedl@gmx.de>, Ruben Molina <rmolina@udea.edu.co>, 387704@bugs.debian.org

Subject: Re: Bug#387704: grep: -i breaks \W in some locales (perhaps UTF-8 locales only)

Date: Sun, 29 Mar 2009 16:46:22 +1100

forwarded 387704 bug-grep@gnu.org
thanks

On Sun, Jan 11, 2009 at 11:14:07AM -0500, Ruben Molina wrote:
>On Sat, Sep 16, 2006 at 10:35:26AM +0200, Christoph Biedl wrote:
>>Package: grep
>>Version: 2.5.1.ds2-5
>>Severity: normal
>>
>>I noticed that enabling --ignore-case suddenly caused certain patterns
>>not to match any longer although they should:
>>
>>$ echo 'foo bar' | grep    '^foo\W'
>>foo bar
>>$ echo 'foo bar' | grep -i '^foo\W'
>>$
>>
>>Digging further reveals that there's an locales influence since
>>$ echo 'foo bar' | LANG=C grep -i '^foo\W'
>>foo bar
>>$
>>
>>matches again. After a check using all my generated locales:
>>
>>MATCH:
>>- de_DE
>>- de_DE@euro
>>- en_US
>>
>>FAIL:
>>- de_DE.UTF-8
>>- de_DE.UTF-8@euro
>>- en_US.UTF-8
>>
>>there's a strong impression that UTF-8 locales somehow disturb \W when
>>using -i.
>>
>>Even more confusing, using the bracket expression instead of the
>>synonym matches again:
>>$ echo 'foo bar' | LANG=de_DE.UTF-8 grep -i '^foo[^[:alnum:]]'
>>foo bar
>>$
>>
>>For the records, this sounds somewhat similar to #209194 and #218873
>>but these bugs are fixed in this version (2.5.1.ds2-5), I've checked.
>>
>>By the way, there's a typo in the manpage
>>
>>  and
>>  .B \eW
>>  is a synonym for
>>- .BR [^[:alnum]] .
>>+ .BR [^[:alnum:]] .
>>  .PP
>>
>>-- System Information:
>>Debian Release: testing/unstable
>>  APT prefers testing
>>  APT policy: (500, 'testing')
>>Architecture: i386 (i686)
>>Shell:  /bin/sh linked to /bin/bash
>>Kernel: Linux 2.6.17.13
>>Locale: LANG=de_DE.UTF-8@euro, LC_CTYPE=de_DE.UTF-8@euro
>>(charmap=UTF-8)
>>
>>Versions of packages grep depends on:
>>ii  libc6                        2.3.6.ds1-4 GNU C Library: Shared
>>libraries
>>
>>grep recommends no packages.
>>
>>-- no debconf information
>
>tags  387704  + confirmed
>found 387704 2.5.3~dfsg-6
>thanks
>
>$ locale
>LANG=es_CO.UTF-8
>LC_CTYPE="es_CO.UTF-8"
>LC_NUMERIC="es_CO.UTF-8"
>LC_TIME="es_CO.UTF-8"
>LC_COLLATE="es_CO.UTF-8"
>LC_MONETARY="es_CO.UTF-8"
>LC_MESSAGES="es_CO.UTF-8"
>LC_PAPER="es_CO.UTF-8"
>LC_NAME="es_CO.UTF-8"
>LC_ADDRESS="es_CO.UTF-8"
>LC_TELEPHONE="es_CO.UTF-8"
>LC_MEASUREMENT="es_CO.UTF-8"
>LC_IDENTIFICATION="es_CO.UTF-8"
>LC_ALL=
>
>$ echo 'foo bar' | grep    '^foo\W'
>foo bar
>$
>
>$ echo 'foo bar' | grep -i '^foo\W'
>$
>
>$ echo 'foo bar' | LANG=C grep -i '^foo\W'
>foo bar
>$

I can reproduce this bug with 2.5.4

grep -V
GNU grep 2.5.4

echo 'foo bar' | grep '^foo\W'; echo $?
foo bar
0

echo 'foo bar' | grep -i '^foo\W'; echo $?
foo bar
0

echo 'foo bar' | LANG=C grep -i '^foo\W'; echo $?
foo bar
0

echo 'foo bar' | LANG=en_AU grep -i '^foo\W'; echo $?
foo bar
0

echo 'foo bar' | LANG=en_AU.UTF-8 grep -i '^foo\W'; echo $?
1

Message #26 received at 387704-done@bugs.debian.org (full text, mbox, reply):

From: santiago@debian.org

To: 387704-done@bugs.debian.org

Subject: Closing

Date: Tue, 24 Jun 2014 19:49:33 +0200

Version: 2.6.3-1

Hi,

I'm closing this bug since the issues with character classes and cases
ignored in multi-byte locales was fixed in grep 2.6.

$ echo 'foo bar' | LANG=C grep '^foo\W'; echo $?
foo bar

$ echo 'foo bar' | LANG=es_CO.UTF-8 grep '^foo\W'; echo $?
foo bar
0

Regards,

Santiago

Send a report that this bug log contains spam.