Debian Bug report logs - #18452
man-db: save cat pages in background?

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: <tbittih@pal.xgw.fi>

To: submit@bugs.debian.org

Subject: man-db: evil prog known as 'man' said this: "Still saving the page, please wait..." - why not fork it to bg?

Date: Sat, 21 Feb 1998 14:23:49 +0200 (EET)

Package: man-db
Version: 2.3.10-57

Why not fork 'man' to background when it's "Still saving the page" ? ;)

-- System Information
Debian Release: 2.0 (unstable)
Kernel Version: Linux bx1 2.1.85 #95-bx1-exp Fri Feb 6 10:24:36 EET 1998 i586 unknown

Versions of the packages man-db depends on:
groff	Version: 1.11a-1
libc6	Version: 2.0.6-3

Message #10 received at 18452@bugs.debian.org (full text, mbox, reply):

From: fpolacco@icenet.fi

To: tbittih@pal.xgw.fi, 18452@bugs.debian.org

Subject: Re: Bug#18452: man-db: evil prog known as 'man' said this: "Still saving the page, please wait..." - why not fork it to bg?

Date: Fri, 27 Feb 1998 13:45:21 +0200

On Sat, Feb 21, 1998 at 02:23:49PM +0200, tbittih@pal.xgw.fi wrote:
> Package: man-db
> Version: 2.3.10-57
> 
> Why not fork 'man' to background when it's "Still saving the page" ? ;)
> 

Sorry for the late reply, but that subject didn't invite me to read it
earlier :)


Could be a suggestion, I'll put it in the wishlist ...
... but anyway, what manpage is taking more than a few tenths of second
to be saved?
(don't tell me bash(1) ... that's an "howto-book", not a "manual-page"
:-)


fab
(ole kiltti, meille ei ole palkkaa)
-- 
| fpolacco@icenet.fi    fpolacco@debian.org    fpolacco@pluto.linux.it
| Líder Minimo del Pluto    -     Debian Developer & Happy Debian User
| 6F7267F5 fingerprint 57 16 C4 ED C9 86 40 7B 1A 69 A1 66 EC FB D2 5E
> more than 34 months are needed to get rid of the millennium. [me]

Message #15 received at 18452@bugs.debian.org (full text, mbox, reply):

From: Tuomas Heino <tbittih@xgw.fi>

To: fpolacco@icenet.fi

Cc: 18452@bugs.debian.org

Subject: Re: Bug#18452: man-db: evil prog known as 'man' said this: "Still saving the page, please wait..." - why not fork it to bg?

Date: Fri, 27 Feb 1998 21:02:47 +0200 (EET)

On Fri, 27 Feb 1998 fpolacco@icenet.fi wrote:

> On Sat, Feb 21, 1998 at 02:23:49PM +0200, tbittih@pal.xgw.fi wrote:
> > Package: man-db
> > Version: 2.3.10-57
> > 
> > Why not fork 'man' to background when it's "Still saving the page" ? ;)
> > 
> 
> Sorry for the late reply, but that subject didn't invite me to read it
> earlier :)
> 
> 
> Could be a suggestion, I'll put it in the wishlist ...
> ... but anyway, what manpage is taking more than a few tenths of second
> to be saved?
> (don't tell me bash(1) ... that's an "howto-book", not a "manual-page"
> :-)
> 
Well there are other big ones too... don't remember which one bothered me 
by wasting some 5 seconds - and yes the loadavg was >5 at the time...
Anyway... think about using a 386sx16 and typing "man bash" or 
something equivalent when in single user mode - sure control-z "works" 
but you'd need to bg it and then fg it a bit later or something...
The point is: gzip -9:ing those cat pages _does_ take some time on slower 
puters like those old 386s - forking to background _shouldn't_ be hard ;)

Message #24 received at 18452@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: Tuomas Heino <tbittih@xgw.fi>

Cc: 18452@bugs.debian.org

Subject: Re: Bug#18452: man-db: evil prog known as 'man' said this: "Still saving the page, please wait..." - why not fork it to bg?

Date: Thu, 27 Jun 2002 11:50:43 +0100

On Fri, Feb 27, 1998 at 09:02:47PM +0200, Tuomas Heino wrote:
> On Fri, 27 Feb 1998 fpolacco@icenet.fi wrote:
> > On Sat, Feb 21, 1998 at 02:23:49PM +0200, tbittih@pal.xgw.fi wrote:
> > > Why not fork 'man' to background when it's "Still saving the page" ? ;)
> > 
> > Sorry for the late reply, but that subject didn't invite me to read it
> > earlier :)
> > 
> > Could be a suggestion, I'll put it in the wishlist ...
> > ... but anyway, what manpage is taking more than a few tenths of second
> > to be saved?
> > (don't tell me bash(1) ... that's an "howto-book", not a "manual-page"
> > :-)
> 
> Well there are other big ones too... don't remember which one bothered me 
> by wasting some 5 seconds - and yes the loadavg was >5 at the time...
> Anyway... think about using a 386sx16 and typing "man bash" or 
> something equivalent when in single user mode - sure control-z "works" 
> but you'd need to bg it and then fg it a bit later or something...
> The point is: gzip -9:ing those cat pages _does_ take some time on slower 
> puters like those old 386s - forking to background _shouldn't_ be hard ;)

I'm trying to set a record here for slowest follow-up to a bug report.
:) Actually I took over the man-db package last year and am just now
getting round to some of the older wishlist bugs.

I don't think it would be a good idea to fork to the background while
"Still saving the page", because then correct error handling becomes
very difficult. (For instance, you might run out of disk space in
/var/cache/man, and that error should be reported.) However, at the
moment man doesn't write to the pager and the cat page independently, so
if you don't scroll down much in the pager then not much of the cat page
gets saved until you exit. That suggests an opportunity for
optimization.

Would you be happier if I fixed man to save the cat page asynchronously
while the user is in the pager? I expect that people will usually spend
more time in the pager than it takes to save the cat page.

Regards,

-- 
Colin Watson                                  [cjwatson@flatline.org.uk]

Message #29 received at 18452-quiet@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: 18452-quiet@bugs.debian.org

Subject: Re: Mail delivery failed: returning message to sender

Date: Thu, 27 Jun 2002 12:02:05 +0100

On Thu, Jun 27, 2002 at 11:50:51AM +0100, Mail Delivery System wrote:
> This message was created automatically by mail delivery software (Exim).
> 
> A message that you sent could not be delivered to one or more of its
> recipients. This is a permanent error. The following address(es) failed:
> 
>   tbittih@xgw.fi
>     unrouteable mail domain "xgw.fi"

Submitter unreachable. I'll just do the save-while-in-pager thing then,
I guess ...

-- 
Colin Watson                                  [cjwatson@flatline.org.uk]

Message #34 received at 18452@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: Tuomas Heino <tbittih@xgw.fi>

Cc: 18452@bugs.debian.org

Subject: Re: Bug#18452: man-db: evil prog known as 'man' said this: "Still saving the page, please wait..." - why not fork it to bg?

Date: Tue, 28 Aug 2007 17:24:41 +0100

tags 18452 fixed-upstream
thanks

On Thu, Jun 27, 2002 at 11:50:43AM +0100, Colin Watson wrote:
> I don't think it would be a good idea to fork to the background while
> "Still saving the page", because then correct error handling becomes
> very difficult. (For instance, you might run out of disk space in
> /var/cache/man, and that error should be reported.) However, at the
> moment man doesn't write to the pager and the cat page independently, so
> if you don't scroll down much in the pager then not much of the cat page
> gets saved until you exit. That suggests an opportunity for
> optimization.
> 
> Would you be happier if I fixed man to save the cat page asynchronously
> while the user is in the pager? I expect that people will usually spend
> more time in the pager than it takes to save the cat page.

I have now implemented this upstream by means of this rather large
change. It will be in man-db 2.5.0.

Tue Aug 28 17:19:16 BST 2007  Colin Watson  <cjwatson@debian.org>

        Implement and use a decompression library. This allows cat pages to
        be saved in the background (Debian bug #18452) and operation with a
        read-only /tmp (Debian bug #165499).

        * lib/pipeline.h (struct command): Add support for commands that
          consist of calling a function rather than executing a process.
          (struct pipeline): Add want_infile and want_outfile members. Note
          that infile and outfile default to NULL. Add source, buffer,
          buflen, bufmax, line_cache, and peek_offset members.
          (command_new_function, command_dump, command_tostring,
          pipeline_connect, pipeline_pump, pipeline_read, pipeline_peek,
          pipeline_peek_size, pipeline_peek_skip, pipeline_readline,
          pipeline_peekline): New prototypes.
          (pipeline_join): Update description for want_infile and
          want_outfile.
        * lib/pipeline.c (command_new, command_dup, command_arg,
          command_argv, command_args, command_argstr, command_free): Update
          for 'struct command' changes.
          (command_new_function, command_dump, command_tostring): New
          functions.
          (pipeline_new, pipeline_join, pipeline_dump, pipeline_tostring):
          Update for 'struct pipeline' changes.
          (pipeline_dump): Use command_dump.
          (pipeline_tostring): Use command_tostring.
          (pipeline_start): Implement want_infile, want_outfile, and
          function commands. Make zero-command case work properly (read
          directly from input file). Flush all pending stdio output so that
          subprocesses don't inherit it.
          (pipeline_connect, pipeline_pump, get_block, pipeline_read,
          pipeline_peek, pipeline_peek_size, pipeline_peek_skip, get_line,
          pipeline_readline, pipeline_peekline): New functions.

        * lib/decompress.c, lib/decompress.h: New files, implementing a
          decompression abstraction layer.
        * lib/Makefile.in: Always compile decompress.c.

        * src/lexgrog.l (YY_INPUT): Define to read from a 'struct pipeline'.
          (find_name): Use decompress_open/decompress_fdopen rather than
          older decompression methods. Split handling once we have a
          decompressor pipeline out to ...
          (find_name_decompressed): ... here.
        * src/check_mandirs.c (test_manfile): Remove decompression code;
          find_name will handle this itself now.
        * src/compression.c (create_ztemp, decompress, remove_ztemp,
          get_ztemp): Remove.
        * src/man.c (checked_popen): Remove.
          (local_man_loop): Remove decompression code.
          (get_preprocessors_from_file): Convert to peeking the first line
          from a pipeline.
          (get_preprocessors): Adjust arguments and
          get_preprocessors_from_file call.
          (remove_stdintmp, create_stdintmp): Remove.
          (make_roff_command): Remove special handling of stdin, now handled
          by peeking a pipeline.
          (open_cat_stream): Don't start the returned pipeline.
          (close_cat_stream): Remove unnecessary COMP_CAT special case.
          (format_display_and_save): Take a decompressor pipeline argument.
          Simplify by using pipeline_connect and pipeline_pump.
          (format_display): Take a decompressor pipeline argument. Use
          pipeline_connect and pipeline_pump.
          (display): Use decompress_open/decompress_fdopen rather than older
          decompression methods.
          (display_filesystem, display_database): Drop remove_ztemp calls.
        * src/straycats.c (check_for_stray): Use decompression layer.
          Rearrange control flow a bit to reduce duplication. Remove use of
          temporary file; use find_name_decompressed instead. Only increment
          strays if find_name_decompressed succeeds.
          (straycats): Remove temporary file creation.
        * src/ult_src.c (ult_src): Use pipeline and decompression layers
          rather than older decompression methods.
        * src/zsoelim.l (YY_INPUT): Define to read from a 'struct pipeline'.
          (<so>\"?[^ \t\n\"]+\"?): Call open_file only once; let it worry
          about compression extensions. Decrement so_stack_ptr here if
          open_file fails rather than having it do it.
          (<lfname>\"?[^ \t\n\"]+\"?): Free the old value of NAME, if set.
          (<<EOF>>): Clean up used pipeline and free NAME. Remove so_delete
          handling.
          (main): Let open_file handle stdin.
          (open_file): Use decompress_open/decompress_fdopen rather than
          older decompression methods. Don't decrement so_stack_ptr here on
          failure.

        * configure.ac: Remove decompressor variable. Check for gzopen in
          libz.
        * include/manconfig.h.in (DECOMPRESSOR): Remove.
          (comp_list): Declare extern here.
          (create_tempfile): Remove.
          (decompress, remove_ztemp, get_ztemp): Remove.
          (find_name_decompressed): Add prototype.
        * lib/tempfile.c (create_tempfile): Remove. We now only need
          temporary directory creation.
        * src/man_db.conf.in (decompressor): Remove.

        * docs/TODO: Update.
        * docs/NEWS: Document this.

Cheers,

-- 
Colin Watson                                       [cjwatson@debian.org]

Message #41 received at 18452-close@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: 18452-close@bugs.debian.org

Subject: Bug#18452: fixed in man-db 2.5.0-1

Date: Sun, 07 Oct 2007 20:47:18 +0000

Source: man-db
Source-Version: 2.5.0-1

We believe that the bug you reported is fixed in the latest version of
man-db, which is due to be installed in the Debian FTP archive:

man-db_2.5.0-1.diff.gz
  to pool/main/m/man-db/man-db_2.5.0-1.diff.gz
man-db_2.5.0-1.dsc
  to pool/main/m/man-db/man-db_2.5.0-1.dsc
man-db_2.5.0-1_i386.deb
  to pool/main/m/man-db/man-db_2.5.0-1_i386.deb
man-db_2.5.0.orig.tar.gz
  to pool/main/m/man-db/man-db_2.5.0.orig.tar.gz



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 18452@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Colin Watson <cjwatson@debian.org> (supplier of updated man-db package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Sun, 07 Oct 2007 20:38:25 +0100
Source: man-db
Binary: man-db
Architecture: source i386
Version: 2.5.0-1
Distribution: unstable
Urgency: low
Maintainer: Colin Watson <cjwatson@debian.org>
Changed-By: Colin Watson <cjwatson@debian.org>
Description: 
 man-db     - on-line manual pager
Closes: 18452 29448 165499 196642 259338 259340 281811 320108 334280 340673 348008 372939 378428 385651 391977 411220 411721 413349 421481 421481 430635 444187
Changes: 
 man-db (2.5.0-1) unstable; urgency=low
 .
   * New upstream release.
     - Remove /usr/X11R6/man from MANDATORY_MANPATH (closes: #413349).
     - man now supports the MANPAGER environment variable, overriding PAGER.
     - Resolve ambiguous error message when asking for a nonexistent manual
       page that might also be a section name (closes: #421481).
     - MANSEC -> MANSECT in German man(1) and catman(8) (closes: #430635).
     - Truncate apropos/whatis output to the terminal width by default
       (closes: #411721).
     - lexgrog now ignores alleged manual page names containing spaces, as
       these usually indicate parsing errors or ill-formed NAME sections and
       they clutter up apropos output badly.
     - Discard stderr from formatting processes when outputting to a pager,
       to avoid visual corruption from any error messages (closes: #372939).
     - Disallow sectional extensions of alphabetic sections, e.g. n and l
       from the default configuration (closes: #391977), and sectional
       extensions beginning with a digit (closes: #421481).
     - Suppress warnings in --quiet mode (closes: #348008, #378428, #411220;
       also closes: #334280 since that suggestion is no longer relevant).
     - If MAN_KEEP_FORMATTING is set in the environment, don't strip
       formatting characters with col (closes: #340673).
     - Implement and use a decompression library. This allows cat pages to be
       saved in the background while the pager is active (closes: #18452) and
       operation with a read-only /tmp (closes: #165499).
     - Add a configuration file flag (NOCACHE) to disable cat page caching
       (closes: #196642).
     - Add output keys to a hashtable, and skip any that have already been
       seen (closes: #259338).
     - Execute system utilities by command name rather than by full path
       (closes: #385651).
     - Add apropos -a/--and option to display only items matching all
       keywords (closes: #259340).
     - Adjust Finnish translation of "No manual entry for %s"
       (closes: #320108).
     - Create and use databases for non-English manual hierarchies
       (closes: #29448, #281811).
     - Improve per-locale directory handling. Directories such as "fr.UTF-8"
       may be used for occasions when it is appropriate to specify the
       character set but not the country, and so a full locale name is
       inconvenient.
     - There is a new "manconv" program which can try multiple possible
       encodings for a file, thus allowing UTF-8 manual pages to be installed
       in any directory even without an explicit encoding declaration.
     - Don't escape the hierarchy path while globbing; it isn't necessary and
       it causes problems for paths containing metacharacters
       (closes: #444187).
   * Use new configure options for external programs rather than hardcoding
     them in configure.ac.
   * Remove code in debian/rules to fiddle manual page paths for Debian,
     since the default of $LL.UTF-8 is now reasonable. (An
     announcement/discussion about what other Debian packages should do will
     be forthcoming shortly.)
   * Build-depend on zlib1g-dev so that we use zlib rather than gzip to
     decompress gzipped manual pages.
Files: 
 7d7e3a536a0c2432c10329b2054a5131 670 doc important man-db_2.5.0-1.dsc
 c23b59726cebe0e0820ced2becf5f52d 977883 doc important man-db_2.5.0.orig.tar.gz
 a5daa70f9dec5e800ee227c333230b18 68665 doc important man-db_2.5.0-1.diff.gz
 fcf95b057b07b54ee7d69ac83a88c2d8 791636 doc important man-db_2.5.0-1_i386.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Colin Watson <cjwatson@debian.org> -- Debian developer

iD8DBQFHCTgQ9t0zAhD6TNERAhXHAJoDy7LB/lRaTyIb9ZvVGiRIj1emSgCfXH1h
+vTb7zj4TIeWw3qxQZ5xMwg=
=t5YZ
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.