Debian Bug report logs - #389762
man-db: cannot select manpage correctly with LANG=ja_JP.UTF-8

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: KISE Hiroshi <kise@fuyuneko.jp>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: man-db: cannot select manpage correctly with LANG=ja_JP.UTF-8

Date: Thu, 28 Sep 2006 00:08:38 +0900 (JST)

[Message part 1 (text/plain, inline)]

Package: man-db
Version: 2.4.3-3
Severity: important

case 1: installed manpages-dev and manpages-ja-dev, not installed
 manpages-posix-dev

$ LANG=ja_JP.UTF-8 man -a -w select
/usr/share/man/man2/select.2.gz
/usr/share/man/ja/man2/select.2.gz
/usr/share/man/man2/select.2.gz
$ LANG=ja_JP.UTF-8 man -w select
/usr/share/man/man2/select.2.gz

It's incorrect.  I expected result "/usr/share/man/ja/man2/select.2.gz".

case 2: installed manpages-dev, manpages-ja-dev and manpages-posix-dev
$ LANG=ja_JP.UTF-8 man -a -w select
/usr/share/man/man2/select.2.gz
/usr/share/man/ja/man2/select.2.gz
/usr/share/man/man2/select.2.gz
/usr/share/man/man3/select.3posix.gz
$ LANG=ja_JP.UTF-8 man -w select
/usr/share/man/ja/man2/select.2.gz

It's correct.

$ egrep ^SECTION /etc/manpath.config
SECTION         1 n l 8 3 2 3posix 3pm 3perl 5 4 9 6 7

I attached debug outputs of "man -d -w select" for each case.
man-debug-1.txt for case 1, man-debug-2.txt for case 2.
(file encoding is UTF-8)

-- System Information:
Debian Release: testing/unstable
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: i386 (i686)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.6.17-2-686
Locale: LANG=ja_JP.eucJP, LC_CTYPE=ja_JP.eucJP (charmap=EUC-JP)

Versions of packages man-db depends on:
ii  bsdmainutils                 6.1.4       collection of more utilities from 
ii  debconf [debconf-2.0]        1.5.4       Debian configuration management sy
ii  dpkg                         1.13.21     package maintenance system for Deb
ii  groff-base                   1.18.1.1-12 GNU troff text-formatting system (
ii  libc6                        2.3.6.ds1-4 GNU C Library: Shared libraries
ii  libgdbm3                     1.8.3-3     GNU dbm database routines (runtime

man-db recommends no packages.

-- debconf information:
  man-db/build-database: true
  man-db/rebuild-database: true
* man-db/install-setuid: false

[man-debug.tar.gz (application/octet-stream, attachment)]

Message #10 received at 389762@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: KISE Hiroshi <kise@fuyuneko.jp>, 389762@bugs.debian.org

Subject: Re: Bug#389762: man-db: cannot select manpage correctly with LANG=ja_JP.UTF-8

Date: Wed, 3 Dec 2008 12:35:58 +0000

On Thu, Sep 28, 2006 at 12:08:38AM +0900, KISE Hiroshi wrote:
> case 1: installed manpages-dev and manpages-ja-dev, not installed
>  manpages-posix-dev
> 
> $ LANG=ja_JP.UTF-8 man -a -w select
> /usr/share/man/man2/select.2.gz
> /usr/share/man/ja/man2/select.2.gz
> /usr/share/man/man2/select.2.gz
> $ LANG=ja_JP.UTF-8 man -w select
> /usr/share/man/man2/select.2.gz
> 
> It's incorrect.  I expected result "/usr/share/man/ja/man2/select.2.gz".

This particular instance of the problem is not reproducible any more,
but I think the underlying bug is still present.

man processes the manpath twice (at least if you use -a). In the first
phase, it looks in the filesystem; in the second phase, it looks in its
database. In each phase it looks through the possible sections and tries
to determine which pages are preferable.

The case of select is somewhat confusing. There's an ordinary page in
section 2 (both for English and Japanese), which is straightforward. If
you install manpages-posix-dev then there's also an ordinary page in
section 3 with extension posix. However, manpages-dev installs links to
select(2) as both FD_ZERO(2) and FD_ZERO(3) (I don't know why), and the
section 3 version is spotted when doing a database lookup. This is why
you get the duplication of /usr/share/man/man2/select.2.gz - at the
level where man is checking for duplicate candidates, it sees that
they're in different sections and therefore distinct, but it hasn't
realised that the section 3 page is actually a symlink into section 2.

As of man-db 2.5.0, there are databases for non-English hierarchies as
well, and I think this has papered over the problem because the bizarre
section 3 page shows up for Japanese as well. However, the fact that the
duplication isn't noticed and the fact that the whole list isn't
properly sorted are still bugs.

For my own reference: it would probably be best to fix the sorting
first, since I have a good reproduction case here.

Thanks,

-- 
Colin Watson                                       [cjwatson@debian.org]

Message #15 received at 389762@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: KISE Hiroshi <kise@fuyuneko.jp>, 389762@bugs.debian.org

Subject: Re: Bug#389762: man-db: cannot select manpage correctly with LANG=ja_JP.UTF-8

Date: Mon, 8 Dec 2008 10:14:40 +0000

tags 389762 fixed-upstream
thanks

On Wed, Dec 03, 2008 at 12:35:58PM +0000, Colin Watson wrote:
> The case of select is somewhat confusing. There's an ordinary page in
> section 2 (both for English and Japanese), which is straightforward. If
> you install manpages-posix-dev then there's also an ordinary page in
> section 3 with extension posix. However, manpages-dev installs links to
> select(2) as both FD_ZERO(2) and FD_ZERO(3) (I don't know why), and the
> section 3 version is spotted when doing a database lookup. This is why
> you get the duplication of /usr/share/man/man2/select.2.gz - at the
> level where man is checking for duplicate candidates, it sees that
> they're in different sections and therefore distinct, but it hasn't
> realised that the section 3 page is actually a symlink into section 2.
> 
> As of man-db 2.5.0, there are databases for non-English hierarchies as
> well, and I think this has papered over the problem because the bizarre
> section 3 page shows up for Japanese as well. However, the fact that the
> duplication isn't noticed and the fact that the whole list isn't
> properly sorted are still bugs.
> 
> For my own reference: it would probably be best to fix the sorting
> first, since I have a good reproduction case here.

I've now fixed this for the next upstream release (man-db 2.5.4). On my
system, the bug showed up as follows:

  $ man -aw select
  /usr/share/man/man2/select.2.gz
  /usr/share/man/man2/select.2.gz
  /usr/share/man/man3/select.3posix.gz
  $ LC_ALL=ja_JP.UTF-8 man -aw select
  /usr/share/man/ja/man2/select.2.gz
  /usr/share/man/man2/select.2.gz
  /usr/share/man/ja/man2/select.2.gz
  /usr/share/man/man2/select.2.gz
  /usr/share/man/man3/select.3posix.gz

The corrected version behaves as follows instead:

  $ src/man -aw select
  /usr/share/man/man2/select.2.gz
  /usr/share/man/man3/select.3posix.gz
  $ LC_ALL=ja_JP.UTF-8 src/man -aw select
  /usr/share/man/ja/man2/select.2.gz
  /usr/share/man/man2/select.2.gz
  /usr/share/man/man3/select.3posix.gz

Mon Dec  8 10:08:45 GMT 2008  Colin Watson  <cjwatson@debian.org>

        Improve sorting and de-duplication of manual page candidates (Debian
        bug #389762).

        * src/man.c (struct candidate): Add ult member.
          (duplicate_candidates): Pare down to only check for duplication
          rather than trying to compare as well; the latter function is now
          handled entirely by compare_candidates. Compare the ultimate
          source file as the first test.
          (compare_candidates): Compare locale elements in candidate paths
          before comparing sections.
          (add_candidates): Look up the ultimate source file if necessary,
          and save it in the candidate structure. Use duplicate_candidates
          for duplicate testing only and compare_candidates for comparison.
          Insert elements at the latest possible position rather than the
          earliest possible.
          (try_section): Pass the ultimate source file that was already
          looked up here.
          (try_db): Adjust for new add_candidate signature.

Thanks for helping to improve man-db!

-- 
Colin Watson                                       [cjwatson@debian.org]

Message #22 received at 389762-close@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: 389762-close@bugs.debian.org

Subject: Bug#389762: fixed in man-db 2.5.4-1

Date: Tue, 24 Feb 2009 03:17:21 +0000

Source: man-db
Source-Version: 2.5.4-1

We believe that the bug you reported is fixed in the latest version of
man-db, which is due to be installed in the Debian FTP archive:

man-db_2.5.4-1.diff.gz
  to pool/main/m/man-db/man-db_2.5.4-1.diff.gz
man-db_2.5.4-1.dsc
  to pool/main/m/man-db/man-db_2.5.4-1.dsc
man-db_2.5.4-1_i386.deb
  to pool/main/m/man-db/man-db_2.5.4-1_i386.deb
man-db_2.5.4.orig.tar.gz
  to pool/main/m/man-db/man-db_2.5.4.orig.tar.gz



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 389762@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Colin Watson <cjwatson@debian.org> (supplier of updated man-db package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.8
Date: Tue, 24 Feb 2009 02:41:53 +0000
Source: man-db
Binary: man-db
Architecture: source i386
Version: 2.5.4-1
Distribution: unstable
Urgency: low
Maintainer: Colin Watson <cjwatson@debian.org>
Changed-By: Colin Watson <cjwatson@debian.org>
Description: 
 man-db     - on-line manual pager
Closes: 121997 166701 167159 271402 384301 389762 461319 477305 480678 480996 481226 482424 482791 482792 482810 483589 483862 483951 494287 494989 496172 512233 516133
Changes: 
 man-db (2.5.4-1) unstable; urgency=low
 .
   * New upstream release.
     - Exit as soon as possible if database writes return ENOSPC (closes:
       #167159).
     - Make it possible to override man's default of discarding stderr when
       stdout is a terminal (closes: #480996).
     - Reorganise lexgrog to stop on any unrecognised roff request, rather
       than continuing and often littering the database with garbage (closes:
       #271402).
     - Make handling of terminal widths for cat pages configurable (closes:
       #121997).
     - Improve sorting and de-duplication of manual page candidates (closes:
       #389762, #496172).
     - Consider SO_MAN equivalent to ULT_MAN for the purposes of sorting
       candidate pages for display (closes: #384301).
     - Add regular expression and shell wildcard search facilities to man
       (closes: #461319).
     - Add option to disable hyphenation (closes: #166701).
     - Line length is a property of output, not input, so only check whether
       standard output is a terminal, not also standard input (closes:
       #512233).
     - Partially rewrite building of manpath according to locale. The
       previous code was completely wrong: as well as handling duplicates
       rather oddly, it effectively handled LANGUAGE in reverse order
       (closes: #516133).
 .
 man-db (2.5.3-3) experimental; urgency=low
 .
   * Backport from trunk:
     - Update manual page search order to permit FHS-compliant installation
       of packages in /opt. Reported by Matt Domsch.
 .
 man-db (2.5.3-2) experimental; urgency=low
 .
   * Explicitly configure --with-lzma=lzma.
   * Increase versioned dependency on groff-base to (>= 1.18.1.1-15) for a
     working -w option in nroff (closes: #494287).
 .
 man-db (2.5.3-1) experimental; urgency=low
 .
   * New upstream release.
     - Cleaned up a number of possible crashes, memory leaks, and missing
       error checks found by the Coverity Scan project.
     - The LANGUAGE environment variable is now tokenised properly, rather
       than only taking the first two characters of each element.
     - man now correctly propagates the exit code of whatis or apropos when
       called with the -f or -k option respectively (closes: #477305).
     - Fix several manual page synopsis problems (thanks, Yuri Kozlov;
       closes: #480678, #481226, #482424, #482791, #482792, #482810, #483589,
       #483862, #483951).
     - Reduce the number of warnings emitted when using an unrecognised
       locale (closes: #494989).
     - manconv and zsoelim are now called internally rather than by executing
       external programs, to improve performance.
     - Manual pages may now be compressed with LZMA (although this is
       probably only worth it for very large pages).
     - Duplicate manual page hierarchies due to symlinks (e.g. /usr/man ->
       /usr/share/man) are detected and removed from the search order.
     - A locale modifier (e.g. @latin) in a directory name must now match the
       locale if the former is set, in addition to the language and
       territory.
     - Bare .so includes (e.g. ".so foo.1" rather than ".so man1/foo.1") now
       work, although only within the same manual page hierarchy for now
       (partially fixes #503472).
   * Add Homepage field.
   * Policy version 3.8.0: no changes required.
   * Pass CFLAGS and LDFLAGS to configure rather than make so that we don't
     have to keep up with the set of warnings that configure decides to
     enable.
Checksums-Sha1: 
 d8f225c48afcc0427fbe87c325ed07853f1b7ec4 1084 man-db_2.5.4-1.dsc
 f7730087396d30171c35d500209253795f2d05f9 1934745 man-db_2.5.4.orig.tar.gz
 e217283934d826858f5f4b54d88475e7b6d80a83 64410 man-db_2.5.4-1.diff.gz
 ddd3e5b6c4a810c5c146f2ec19a1b884f9859022 1185800 man-db_2.5.4-1_i386.deb
Checksums-Sha256: 
 624be742690134c7911623b8932db9ed70327edd6e7c5cd46ec9731711c2ac43 1084 man-db_2.5.4-1.dsc
 4de6cf413f44d971d304b174b35a75c7ec35b022fe04e265ff07951bbf6676d8 1934745 man-db_2.5.4.orig.tar.gz
 7e7330eba8c8d7cdfea9d018cc9e8b795df09afd147f82770518077d41233ec2 64410 man-db_2.5.4-1.diff.gz
 3f6293391182438f54c7450a8591c60ad9660c690d1028ce10020786a1922261 1185800 man-db_2.5.4-1_i386.deb
Files: 
 f53e6ee6d722c4398760492692395a81 1084 doc important man-db_2.5.4-1.dsc
 591752c2bd003f7f2f5ca3387da3ddc9 1934745 doc important man-db_2.5.4.orig.tar.gz
 d86d257bba9214e52348e91640091369 64410 doc important man-db_2.5.4-1.diff.gz
 2229dcde38011eb99c1566c4b30e07b8 1185800 doc important man-db_2.5.4-1_i386.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Colin Watson <cjwatson@debian.org> -- Debian developer

iD8DBQFJo2Mo9t0zAhD6TNERAgQ6AJ0RKJ6oXgD8Gj0U4oLNWTWNoL7nBACfVhrr
CGdkpg7Skvgej0quUBNRKPY=
=JyR/
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.