Debian Bug report logs - #373888
aptitude: error in description rendering

version graph

Package: aptitude; Maintainer for aptitude is Aptitude Development Team <aptitude-devel@lists.alioth.debian.org>; Source for aptitude is src:aptitude.

Reported by: Gerfried Fuchs <alfie@debian.org>

Date: Fri, 16 Jun 2006 06:19:12 UTC

Severity: normal

Found in version aptitude/0.4.1-1

Fixed in version aptitude/0.4.2-1

Done: Daniel Burrows <dburrows@debian.org>

Bug is archived. No further changes may be made.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Daniel Burrows <dburrows@debian.org>:
Bug#373888; Package aptitude. Full text and rfc822 format available.

Acknowledgement sent to Gerfried Fuchs <alfie@debian.org>:
New Bug report received and forwarded. Copy sent to Daniel Burrows <dburrows@debian.org>. Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Gerfried Fuchs <alfie@debian.org>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: aptitude: error in description rendering
Date: Fri, 16 Jun 2006 01:13:24 -0500
Package: aptitude
Version: 0.4.1-1
Severity: normal

        Hi!

 This is the description of simba:

#v+
Description: next generation mirroring tool
 Simba was created to be _the_ mirroring tool, to get more control
 over the mirrored content and (most importantly) more control over
 the reports you can generate using the mirrored content data.
 Using Simba, you can:
    +   Create web pages with mirrors status
    +   Create web pages with mirror details
    +   Generate RSS feeds
    +   Generate Google sitemaps
    +   Generate rsync configuration files
    +   ... and more
 Simba is extensible and has a dynamic plugin system. If you have some knowledge
 of perl, you can write your own plugins and extend Simba as you wish.
 .
 Homepage: http://simba.packages.ro/
#v-

 And this is how it is rendered:

#v+
Simba was created to be _the_ mirroring tool, to get more control over the mirrored content and (most importantly) more
control over the reports you can generate using the mirrored content data. Using Simba, you can:
* Create web pages with mirrors status
* Create web pages with mirror details
* Generate RSS feeds
* Generate Google sitemaps
* Generate rsync configuration files
*
Simba is extensible and has a dynamic plugin system. If you have some knowledge
of perl, you can write your own plugins and extend Simba as you wish.

Homepage: http://simba.packages.ro/
#v-

 a.) there is missing the "... and more"
 b.) The squeezing of the multiple spaces after the + is unpredicted and
can cause additional problem with formating.

 I beg you to remove the "interpretation" of the preformated
descriptions once again...

 So long,
Rhonda
-- 
    - Water effect intro screen (cool!).
    - Multiple views in the screen (cooler!).
    - Fixed a lot of bugs (coolest!).
 -- David Martínez Moreno, changelog.Debian for freecraft (1:1.17-1)



Reply sent to Daniel Burrows <dburrows@debian.org>:
You have taken responsibility. Full text and rfc822 format available.

Notification sent to Gerfried Fuchs <alfie@debian.org>:
Bug acknowledged by developer. Full text and rfc822 format available.

Message #10 received at 373888-close@bugs.debian.org (full text, mbox):

From: Daniel Burrows <dburrows@debian.org>
To: 373888-close@bugs.debian.org
Subject: Bug#373888: fixed in aptitude 0.4.2-1
Date: Wed, 02 Aug 2006 03:47:46 -0700
Source: aptitude
Source-Version: 0.4.2-1

We believe that the bug you reported is fixed in the latest version of
aptitude, which is due to be installed in the Debian FTP archive:

aptitude-doc-cs_0.4.2-1_all.deb
  to pool/main/a/aptitude/aptitude-doc-cs_0.4.2-1_all.deb
aptitude-doc-en_0.4.2-1_all.deb
  to pool/main/a/aptitude/aptitude-doc-en_0.4.2-1_all.deb
aptitude-doc-fi_0.4.2-1_all.deb
  to pool/main/a/aptitude/aptitude-doc-fi_0.4.2-1_all.deb
aptitude-doc-fr_0.4.2-1_all.deb
  to pool/main/a/aptitude/aptitude-doc-fr_0.4.2-1_all.deb
aptitude_0.4.2-1.diff.gz
  to pool/main/a/aptitude/aptitude_0.4.2-1.diff.gz
aptitude_0.4.2-1.dsc
  to pool/main/a/aptitude/aptitude_0.4.2-1.dsc
aptitude_0.4.2-1_i386.deb
  to pool/main/a/aptitude/aptitude_0.4.2-1_i386.deb
aptitude_0.4.2.orig.tar.gz
  to pool/main/a/aptitude/aptitude_0.4.2.orig.tar.gz



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 373888@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Daniel Burrows <dburrows@debian.org> (supplier of updated aptitude package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Tue,  1 Aug 2006 17:23:40 -0700
Source: aptitude
Binary: aptitude-doc-cs aptitude-doc-fr aptitude-doc-fi aptitude-doc-en aptitude
Architecture: source all i386
Version: 0.4.2-1
Distribution: unstable
Urgency: low
Maintainer: Daniel Burrows <dburrows@debian.org>
Changed-By: Daniel Burrows <dburrows@debian.org>
Description: 
 aptitude   - terminal-based apt frontend
 aptitude-doc-cs - Czech manual for aptitude, a terminal-based apt frontend
 aptitude-doc-en - English manual for aptitude, a terminal-based apt frontend
 aptitude-doc-fi - Finnish manual for aptitude, a terminal-based apt frontend
 aptitude-doc-fr - French manual for aptitude, a terminal-based apt frontend
Closes: 160418 275704 299127 330226 338989 341924 343748 343850 344078 344380 344569 344630 345226 345345 345372 346380 347311 347445 348481 348541 349161 349427 349500 349500 349869 350118 351323 351557 351643 351839 353449 354844 355586 355689 356396 357557 361439 362927 363648 363660 363905 364186 364306 364471 364621 366523 366884 369382 372796 372861 373888 373893 374207 375089 375327 375383 376340 376573
Changes: 
 aptitude (0.4.2-1) unstable; urgency=low
 .
   * New upstream release.
 .
     - Added an option "Aptitude::UI::ViewTabs" that controls
       whether the "tabs" at the top of the screen are displayed.
       (Closes: #344569)
 .
     - Make the lockfile default to /var/lock/aptitude as per FHS.
       (Closes: #160418)
 .
     - Add support for searching in the on-line user's manual.
       (Closes: #364306)
 .
     - Acknowledge NMU (Closes: #357557)
 .
     - Fix compilation errors with gcc 4.2. (Closes: #369382)
       Note that the compiled software appears to crash when run.
 .
     - Fix sporadic crashes on update that were due to interactions
       with the problem resolver. (Closes: #346380, #348541, #348481,
       #376573)
 .
     - Handle NULL returns from get_changelog_from_source; should
       fix some crashes in the command-line changelog fetcher.
       (Closes: #349869)
 .
     - Fix a crash that occurred when moving the selection up while
       the package list was empty.
 .
     - Disable bullet-detection by default, since it's incompatible
       in a bad way that cannot be easily fixed.  (Closes: #373888)
       Set Aptitude::UI::Parse-Description-Bullets to true to
       re-enable it.
 .
     - Make the selection and display of versions in "aptitude show"
       more 'intuitive' (Closes: #375383, #372796).
 .
     - Stick to portable pthreads features se we compile on the Hurd.
       (Closes: #361439)
 .
     - Don't read the package lists on startup if they're just going to
       be discarded immediately.  (Closes: #366884)
 .
     - Correctly handle scrollbar updates after resizing text layouts.
       (Closes: #347445)
 .
     - Fix how tab characters are rendered in the internal pager.
       (Closes: #351323)
 .
     - Use the apt library's file downloader again, as the extra hacks
       I was using have been implemented better there.  This should allow
       command-line downloads to work with file: sources.  (Closes: #299127)
 .
     - Translation updates:
 .
       + Use po4a for the manual. (Closes: #351643)
       + Basque (Closes: #349500, #275704, #349500)
       + Brazillian (Closes: #363905)
       + Catalan (Closes: #345226, #363648)
       + Chinese (Simplified) (Closes: #347311, 355689)
       + Czech (Closes: #345345, #349427)
       + Danish
       + Dutch (Closes: #350118, #364471)
       + Dzongkha
       + Finnish
       + French (Closes: #344078, 343748)
       + Galacian (Closes: #344380, #355586, #373893)
       + German (Closes: #330226, #351557)
       + Greek (Closes: #344630)
       + Hungarian (Closes: #354844)
       + Italian
       + Japanese (Closes: #364621)
       + Nepali (Closes: #372861)
       + Norwegian
       + Polish (Closes: #338989)
       + Portuguese (Closes: #364186)
       + Romanian (Closes: #362927, #375327)
       + Russian (Closes: #349161, #366523, #376340)
       + Slovak (Closes: #351839, #353449, #356396)
       + Spanish
       + Swedish (Closes: #345372, #363660, #374207)
       + Vietnamese (Closes: #341924, #343850, #375089)
Files: 
 c5a65c0babfb20992698ea47e686036b 802 admin - aptitude_0.4.2-1.dsc
 221f57a773cbce3bbf79da6637721826 5012352 admin - aptitude_0.4.2.orig.tar.gz
 b739de139ce4964d595c8b3d138323bc 23663 admin - aptitude_0.4.2-1.diff.gz
 5d3b9ffc8655d0a5f5ff48c906b012ff 272926 doc optional aptitude-doc-cs_0.4.2-1_all.deb
 c9c260af73dbad56f25795408405dd55 322648 doc optional aptitude-doc-en_0.4.2-1_all.deb
 b8f41372b5dd0b58c7ed630d2a1d379b 255436 doc optional aptitude-doc-fi_0.4.2-1_all.deb
 f70d625cc2da14d0f80c010194d757cc 265054 doc optional aptitude-doc-fr_0.4.2-1_all.deb
 37449337b45599a110a7bd61ab311085 2757136 admin important aptitude_0.4.2-1_i386.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)

iD8DBQFE0FY2ch6xsM7kSXgRAqPcAKD/CQzuSNj+mjgRb5vSvEVEKTiohACgun59
QetHcFg4vF3PkIc56e38YQg=
=Qc/Y
-----END PGP SIGNATURE-----




Information forwarded to debian-bugs-dist@lists.debian.org, Daniel Burrows <dburrows@debian.org>:
Bug#373888; Package aptitude. Full text and rfc822 format available.

Acknowledgement sent to Tomas Pospisek <tpo_deb@sourcepole.ch>:
Extra info received and forwarded to list. Copy sent to Daniel Burrows <dburrows@debian.org>. Full text and rfc822 format available.

Message #15 received at 373888@bugs.debian.org (full text, mbox):

From: Tomas Pospisek <tpo_deb@sourcepole.ch>
To: 388594@bugs.debian.org
Cc: 373888@bugs.debian.org, 373888-submitter@bugs.debian.org
Subject: Re: Bug#388594: take into account lists in descriptions when wrapping
Date: Sat, 23 Sep 2006 01:20:42 +0200 (CEST)
On Thu, 21 Sep 2006, Daniel Burrows wrote:

>  Does the behavior improve if you start aptitude with the option
> "-o Aptitude::Parse-Description-Bullets=true"?

Beautiful!

>  PS: see bug #373888.

Ummm ... I can understand that removing the last line ("+   ... and more") 
is a bug and should be avoided, however:

"b.) The squeezing of the multiple spaces after the + is unpredicted and
 can cause additional problem with formating."

I can understand that this *could* pose problems, however is there 
any *actual* example(s) where it *does* cause formatting problems?

In the example in case here the resulting rendering fits perfectly since 
what we have there is a list, and aptitude does render it as a list.

So is b) actually a real problem?

In cases where aptitude gets the rendering of the lists wrong, we might as 
well submit patches to those descriptions, so that they *do* get rendered 
correctly in the future. See [1] where I started such an effort.
*t

[1] http://wiki.debian.org/Aptitude::Parse-Description-Bullets=true

-- 
--------------------------------------------------------
  Tomas Pospisek
  http://sourcepole.com -  Linux & Open Source Solutions
--------------------------------------------------------



Message sent on to Gerfried Fuchs <alfie@debian.org>:
Bug#373888. Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#373888; Package aptitude. Full text and rfc822 format available.

Acknowledgement sent to Daniel Burrows <dburrows@debian.org>:
Extra info received and forwarded to list. Full text and rfc822 format available.

Message #23 received at 373888@bugs.debian.org (full text, mbox):

From: Daniel Burrows <dburrows@debian.org>
To: Tomas Pospisek <tpo_deb@sourcepole.ch>, 373888@bugs.debian.org
Subject: Re: Bug#373888: Bug#388594: take into account lists in descriptions when wrapping
Date: Fri, 22 Sep 2006 20:18:11 -0700
[Message part 1 (text/plain, inline)]
On Sat, Sep 23, 2006 at 01:20:42AM +0200, Tomas Pospisek <tpo_deb@sourcepole.ch> was heard to say:
> On Thu, 21 Sep 2006, Daniel Burrows wrote:
> 
> > Does the behavior improve if you start aptitude with the option
> >"-o Aptitude::Parse-Description-Bullets=true"?
> 
> Beautiful!
> 
> > PS: see bug #373888.
> 
> Ummm ... I can understand that removing the last line ("+   ... and more") 
> is a bug and should be avoided, however:
> 
> "b.) The squeezing of the multiple spaces after the + is unpredicted and
>  can cause additional problem with formating."
> 
> I can understand that this *could* pose problems, however is there 
> any *actual* example(s) where it *does* cause formatting problems?

  There are two real problems:

  (1) aptitude was collapsing all the spaces in the first line, preventing
      you from using any sort of explicit formatting in lists.  This has
      been fixed (lists won't be recognized unless there's exactly one
      space following the bullet, but that's usually the case).

  (2) The bulletting scheme cannot distinguish between a full stop at the
      beginning of a legitimate line of text and a syntactically
      significant full stop.  Since these periods may occur after other
      text or after two spaces, the standard description algorithm will
      not strip this text -- meaning that aptitude will massively mangle
      a description that displays just fine in the standard formatting.
      
      Worse, you need to know about aptitude's formatting to get this right;
      full-stops after indentation are just fine everywhere else, but not
      if you've written a bulletted list.  I presume that most Debian
      developers aren't even aware aptitude exists, and even if they do, they
      aren't going to be intimately familiar with its description parsing
      algorithm.  I can't really defend a feature that's likely to lead to
      surprising and undesired results.

      This actually occurred in the example that led to bug #373888, and
      at a quick check there's one other package (flac) that has this
      problem.

      I wasn't able to find any satisfactory solution to this problem.

  Daniel
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Daniel Burrows <dburrows@debian.org>:
Bug#373888; Package aptitude. Full text and rfc822 format available.

Acknowledgement sent to Tomas Pospisek <tpo_deb@sourcepole.ch>:
Extra info received and forwarded to list. Copy sent to Daniel Burrows <dburrows@debian.org>. Full text and rfc822 format available.

Message #28 received at 373888@bugs.debian.org (full text, mbox):

From: Tomas Pospisek <tpo_deb@sourcepole.ch>
To: 373888@bugs.debian.org
Subject: Re: Bug#373888: Bug#388594: take into account lists in descriptions when wrapping
Date: Sat, 23 Sep 2006 17:38:52 +0200 (CEST)
On Fri, 22 Sep 2006, Daniel Burrows wrote:

[...]
>  There are two real problems [with parsing and rendering lists in
>  package descriptions - tpo]:
[...]
>  (2) The bulletting scheme cannot distinguish between a full stop at the
>      beginning of a legitimate line of text and a syntactically
>      significant full stop.  Since these periods may occur after other
>      text or after two spaces, the standard description algorithm will
>      not strip this text -- meaning that aptitude will massively mangle
>      a description that displays just fine in the standard formatting.
>
>      Worse, you need to know about aptitude's formatting to get this right;
>      full-stops after indentation are just fine everywhere else, but not
>      if you've written a bulletted list.  I presume that most Debian
>      developers aren't even aware aptitude exists, and even if they do, they
>      aren't going to be intimately familiar with its description parsing
>      algorithm.  I can't really defend a feature that's likely to lead to
>      surprising and undesired results.
>
>      This actually occurred in the example that led to bug #373888, and
>      at a quick check there's one other package (flac) that has this
>      problem.
>
>      I wasn't able to find any satisfactory solution to this problem.

What I do not understand here, is how come that a '.' at the beginning of 
a line has syntactic influence on aptitude.

The Debian policy [1] distinguishes the following line starts:

* 1 space + whatever       -> paragraphs
* 1 space + dot            -> blank line
* 1 space + dot + whatever -> reserved / to be defined
* 2 spaces                 -> verbatim

Thus a line starting with:

* 1 dot            -> is a syntax error in the description
                   -> thus of no business to aptitude
* 1 space + dot    -> blank line -> of no business to aptitude
* 2 spaces + 1 dot -> per aptitudes list definition not part of a list
                      since the text of a list entry must be at least as
                      far indented as the first bullet and thus start
                      with at least 4 spaces off to th
                   -> thus render verbatim
* 3 spaces + 1 dot -> same as 2 spaces + 1 dot
* 4 spaces + 1 dot -> in case we're in a list entry (which is
                      determined by the previous line) then render the
                      dot as part of the list entry. Otherwise verbatim
* >4spaces + 1 dot -> dito, except that you'll probably want to ident it
                      with the additional spaces

* 1 space + bullet + whatever -> standard rendering - no list entry
* 2 spaces + bullet + whatever -> list entry

The crutial point being, that aptitude must not interpret a dot aka a 
period as a bullet otherwise confusion ensues - which AFAIK it doesn't.

So again. I unfortunately fail to see the problem. In case you could give 
me some examples that I succeed to grasp then I'd be grateful.

And to come back to my suggested part of the fix of the problem(-range): 
one (among other) ways to go would be to check *all* the descriptions and 
to make sure that they render well and if not to "fix" them [2]. This does 
not seem to be a infeasible thing. Once this is done, then there doesn't 
seem to be a technical hurdle to put the list syntax into the Debian 
policy?

*t

[1] http://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Description
[2] http://wiki.debian.org/Aptitude%3a%3aParse-Description-Bullets%3dtrue

--
-----------------------------------------------------------------------
     Tomas Pospisek
     http://sourcepole.com -   Linux & Open Source Solutions
------------------------------------------------------------------------



Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#373888; Package aptitude. Full text and rfc822 format available.

Acknowledgement sent to Daniel Burrows <dburrows@debian.org>:
Extra info received and forwarded to list. Full text and rfc822 format available.

Message #33 received at 373888@bugs.debian.org (full text, mbox):

From: Daniel Burrows <dburrows@debian.org>
To: Tomas Pospisek <tpo_deb@sourcepole.ch>, 373888@bugs.debian.org
Subject: Re: Bug#373888: Bug#388594: take into account lists in descriptions when wrapping
Date: Sun, 24 Sep 2006 10:40:54 -0700
[Message part 1 (text/plain, inline)]
On Sat, Sep 23, 2006 at 05:38:52PM +0200, Tomas Pospisek <tpo_deb@sourcepole.ch> was heard to say:
> On Fri, 22 Sep 2006, Daniel Burrows wrote:
> 
> [...]
> > There are two real problems [with parsing and rendering lists in
> > package descriptions - tpo]:
> [...]
> > (2) The bulletting scheme cannot distinguish between a full stop at the
> >     beginning of a legitimate line of text and a syntactically
> >     significant full stop.  Since these periods may occur after other
> >     text or after two spaces, the standard description algorithm will
> >     not strip this text -- meaning that aptitude will massively mangle
> >     a description that displays just fine in the standard formatting.
> >
> >     Worse, you need to know about aptitude's formatting to get this right;
> >     full-stops after indentation are just fine everywhere else, but not
> >     if you've written a bulletted list.  I presume that most Debian
> >     developers aren't even aware aptitude exists, and even if they do, 
> >     they
> >     aren't going to be intimately familiar with its description parsing
> >     algorithm.  I can't really defend a feature that's likely to lead to
> >     surprising and undesired results.
> >
> >     This actually occurred in the example that led to bug #373888, and
> >     at a quick check there's one other package (flac) that has this
> >     problem.
> >
> >     I wasn't able to find any satisfactory solution to this problem.
> 
> What I do not understand here, is how come that a '.' at the beginning of 
> a line has syntactic influence on aptitude.
> 
> The Debian policy [1] distinguishes the following line starts:
> 
> * 1 space + whatever       -> paragraphs
> * 1 space + dot            -> blank line
> * 1 space + dot + whatever -> reserved / to be defined
> * 2 spaces                 -> verbatim
> 
> Thus a line starting with:
> 
> * 1 dot            -> is a syntax error in the description
>                    -> thus of no business to aptitude
> * 1 space + dot    -> blank line -> of no business to aptitude
> * 2 spaces + 1 dot -> per aptitudes list definition not part of a list
>                       since the text of a list entry must be at least as
>                       far indented as the first bullet and thus start
>                       with at least 4 spaces off to th
>                    -> thus render verbatim
> * 3 spaces + 1 dot -> same as 2 spaces + 1 dot
> * 4 spaces + 1 dot -> in case we're in a list entry (which is
>                       determined by the previous line) then render the
>                       dot as part of the list entry. Otherwise verbatim
> * >4spaces + 1 dot -> dito, except that you'll probably want to ident it
>                       with the additional spaces

  All correct.

> * 1 space + bullet + whatever -> standard rendering - no list entry
> * 2 spaces + bullet + whatever -> list entry

  Ah.  The problem is that aptitude parses indented regions using the
standard grammar, but with N spaces stripped from each line before
applying the grammar.  This allows multi-paragraph list entries, like:

  * One list entry
  * Another list entry.
    .
    Here is some more text describing the list entry.

  The result of treating the last line as part of the list entry is that
it will be indented to exactly the same level as the rest of the entry,
and it will be word-wrapped instead of being treated as verbatim text.

  Using the standard grammar also allows verbatim text in lists.  This
was another reason for always stripping exactly one space after the
bullet; it allows you to write

  *  Homepage: http://some.url.example.com

  and have it not get word-wrapped. (previously this would be considered
a normal bullet point with 2 spaces of indentation beyond the bullet)

  Removing the ability to write multi-paragraph list entries is not
appealing to me.  Now that I look at this from a fresh perspective,
though, there is one other choice.  I could consider text following
a bullet that's at the same indentation level and separated only by
blank lines to be part of the bullet, as in:

  * One list entry
  * Another list entry.
 .
    Here is some more text describing the list entry.

  This syntax is also more compatible with older readers like dselect.

> The crutial point being, that aptitude must not interpret a dot aka a 
> period as a bullet otherwise confusion ensues - which AFAIK it doesn't.
>
> So again. I unfortunately fail to see the problem. In case you could give 
> me some examples that I succeed to grasp then I'd be grateful.
> 
> And to come back to my suggested part of the fix of the problem(-range): 
> one (among other) ways to go would be to check *all* the descriptions and 
> to make sure that they render well and if not to "fix" them [2]. This does 
> not seem to be a infeasible thing. Once this is done, then there doesn't 
> seem to be a technical hurdle to put the list syntax into the Debian 
> policy?

  I think the path of least resistance, especially for getting this into
policy, is to have a backwards-compatible syntax.  aptitude's list
handling was supposed to be backwards-compatible, but I made a mistake
in handling full stops that made it not backwards-compatible.  I'm not
aware of any other technical problems, so I'll re-enable it in the next
release (once I implement the change above).

  Daniel
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Daniel Burrows <dburrows@debian.org>:
Bug#373888; Package aptitude. Full text and rfc822 format available.

Acknowledgement sent to Tomas Pospisek <tpo_deb@sourcepole.ch>:
Extra info received and forwarded to list. Copy sent to Daniel Burrows <dburrows@debian.org>. Full text and rfc822 format available.

Message #38 received at 373888@bugs.debian.org (full text, mbox):

From: Tomas Pospisek <tpo_deb@sourcepole.ch>
To: 373888@bugs.debian.org
Subject: Re: Bug#373888: Bug#388594: take into account lists in descriptions when wrapping
Date: Mon, 25 Sep 2006 22:37:48 +0200 (CEST)
On Sun, 24 Sep 2006, Daniel Burrows wrote:

> On Sat, Sep 23, 2006 at 05:38:52PM +0200, Tomas Pospisek <tpo_deb@sourcepole.ch> was heard to say:
>> On Fri, 22 Sep 2006, Daniel Burrows wrote:
>>
>> [...]
>>> There are two real problems [with parsing and rendering lists in
>>> package descriptions - tpo]:
>> [...]
>>> (2) The bulletting scheme cannot distinguish between a full stop at the
>>>     beginning of a legitimate line of text and a syntactically
>>>     significant full stop.  Since these periods may occur after other
>>>     text or after two spaces, the standard description algorithm will
>>>     not strip this text -- meaning that aptitude will massively mangle
>>>     a description that displays just fine in the standard formatting.
>>>
>>>     Worse, you need to know about aptitude's formatting to get this right;
>>>     full-stops after indentation are just fine everywhere else, but not
>>>     if you've written a bulletted list.  I presume that most Debian
>>>     developers aren't even aware aptitude exists, and even if they do,
>>>     they
>>>     aren't going to be intimately familiar with its description parsing
>>>     algorithm.  I can't really defend a feature that's likely to lead to
>>>     surprising and undesired results.
>>>
>>>     This actually occurred in the example that led to bug #373888, and
>>>     at a quick check there's one other package (flac) that has this
>>>     problem.
>>>
>>>     I wasn't able to find any satisfactory solution to this problem.
>>
>> What I do not understand here, is how come that a '.' at the beginning of
>> a line has syntactic influence on aptitude.
>>
>> The Debian policy [1] distinguishes the following line starts:
>>
>> * 1 space + whatever       -> paragraphs
>> * 1 space + dot            -> blank line
>> * 1 space + dot + whatever -> reserved / to be defined
>> * 2 spaces                 -> verbatim
>>
>> Thus a line starting with:
>>
>> * 1 dot            -> is a syntax error in the description
>>                    -> thus of no business to aptitude
>> * 1 space + dot    -> blank line -> of no business to aptitude
>> * 2 spaces + 1 dot -> per aptitudes list definition not part of a list
>>                       since the text of a list entry must be at least as
>>                       far indented as the first bullet and thus start
>>                       with at least 4 spaces off to th
>>                    -> thus render verbatim
>> * 3 spaces + 1 dot -> same as 2 spaces + 1 dot
>> * 4 spaces + 1 dot -> in case we're in a list entry (which is
>>                       determined by the previous line) then render the
>>                       dot as part of the list entry. Otherwise verbatim
>> * >4spaces + 1 dot -> dito, except that you'll probably want to ident it
>>                       with the additional spaces
>
>  All correct.
>
>> * 1 space + bullet + whatever -> standard rendering - no list entry
>> * 2 spaces + bullet + whatever -> list entry
>
>  Ah.  The problem is that aptitude parses indented regions using the
> standard grammar, but with N spaces stripped from each line before
> applying the grammar.  This allows multi-paragraph list entries, like:
>
>  * One list entry
>  * Another list entry.
>    .
>    Here is some more text describing the list entry.
>
>  The result of treating the last line as part of the list entry is that
> it will be indented to exactly the same level as the rest of the entry,
> and it will be word-wrapped instead of being treated as verbatim text.
>
>  Using the standard grammar also allows verbatim text in lists.  This
> was another reason for always stripping exactly one space after the
> bullet; it allows you to write
>
>  *  Homepage: http://some.url.example.com
>
>  and have it not get word-wrapped. (previously this would be considered
> a normal bullet point with 2 spaces of indentation beyond the bullet)
>
>  Removing the ability to write multi-paragraph list entries is not
> appealing to me.

Mh. Well I understand a bit better now, thanks a lot. So the conclusion 
is that the grammar/parser needs a change. However:

>  Now that I look at this from a fresh perspective,
> though, there is one other choice.  I could consider text following
> a bullet that's at the same indentation level and separated only by
> blank lines to be part of the bullet, as in:
>
>  * One list entry
>  * Another list entry.
> .
>    Here is some more text describing the list entry.

This is very ugly. Why not allow the grammar as it was and instead say 
that:

   * One list entry
     .
     Some more text

only this form will be accepted as a blank line and none other. That means 
all other cases such as:

   * ...
   * bla
     ...

etc. will be part of a paragraph?

>  I think the path of least resistance, especially for getting this into
> policy, is to have a backwards-compatible syntax.  aptitude's list
> handling was supposed to be backwards-compatible, but I made a mistake
> in handling full stops that made it not backwards-compatible.  I'm not
> aware of any other technical problems, so I'll re-enable it in the next
> release (once I implement the change above).

In case you'd like my suggested behaveour above, then the *only* change 
wrt older description listers would be that the old listers would render 
the description:

  * bla bla
    .

as is. Whereas aptitude would render it as:

  * bla bla
[newline]

and that would, besides of the word wraping and bullet highlighting 
feature you've allready implemented, be the only change of behaveour.

If we can prove that ...

> And to come back to my suggested part of the fix of the problem(-range):
> one (among other) ways to go would be to check *all* the descriptions 
> and to make sure that they render well and if not to "fix" them [2]. 
> This does not seem to be a infeasible thing. Once this is done, then 
> there doesn't seem to be a technical hurdle to put the list syntax into 
> the Debian policy?

all existing packages are rendered correctly, even with the change of 
syntax above then the need for backward compatiblity will be remain for an 
empty set of packages. I can see your point about backward compatibilty, 
but IMHO it's not really sensible to require it even if no existing 
package will be affected any more...

Currently there are 1729 packages that contain bullets that will be 
interpreted as such by aptitude and thus would need to be checked.

Using a bit of scripting it should be possible to do the later in a day or 
so.
*t

--
--------------------------------------------------------
  Tomas Pospisek
  http://sourcepole.com -  Linux & Open Source Solutions
--------------------------------------------------------



Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#373888; Package aptitude. Full text and rfc822 format available.

Acknowledgement sent to Daniel Burrows <dburrows@debian.org>:
Extra info received and forwarded to list. Full text and rfc822 format available.

Message #43 received at 373888@bugs.debian.org (full text, mbox):

From: Daniel Burrows <dburrows@debian.org>
To: Tomas Pospisek <tpo_deb@sourcepole.ch>, 373888@bugs.debian.org
Subject: Re: Bug#373888: Bug#388594: take into account lists in descriptions when wrapping
Date: Mon, 25 Sep 2006 20:41:19 -0700
[Message part 1 (text/plain, inline)]
On Mon, Sep 25, 2006 at 10:37:48PM +0200, Tomas Pospisek <tpo_deb@sourcepole.ch> was heard to say:
> On Sun, 24 Sep 2006, Daniel Burrows wrote:
> > Ah.  The problem is that aptitude parses indented regions using the
> >standard grammar, but with N spaces stripped from each line before
> >applying the grammar.  This allows multi-paragraph list entries, like:
> >
> > * One list entry
> > * Another list entry.
> >   .
> >   Here is some more text describing the list entry.
> >
> > The result of treating the last line as part of the list entry is that
> >it will be indented to exactly the same level as the rest of the entry,
> >and it will be word-wrapped instead of being treated as verbatim text.
> >
> > Using the standard grammar also allows verbatim text in lists.  This
> >was another reason for always stripping exactly one space after the
> >bullet; it allows you to write
> >
> > *  Homepage: http://some.url.example.com
> >
> > and have it not get word-wrapped. (previously this would be considered
> >a normal bullet point with 2 spaces of indentation beyond the bullet)
> >
> > Removing the ability to write multi-paragraph list entries is not
> >appealing to me.
> 
> Mh. Well I understand a bit better now, thanks a lot. So the conclusion 
> is that the grammar/parser needs a change. However:
> 
> > Now that I look at this from a fresh perspective,
> >though, there is one other choice.  I could consider text following
> >a bullet that's at the same indentation level and separated only by
> >blank lines to be part of the bullet, as in:
> >
> > * One list entry
> > * Another list entry.
> >.
> >   Here is some more text describing the list entry.
> 
> This is very ugly. Why not allow the grammar as it was and instead say 
> that:
> 
>    * One list entry
>      .
>      Some more text
> 
> only this form will be accepted as a blank line and none other. That means 
> all other cases such as:

  This might be reasonable as an alternative.  It does mean, though,
that if policy starts to define a meaning for " .asdf\n", there isn't
a clean way of adding support for that within lists.

  I'm not sure that just using the same paragraph separator everywhere
is all that ugly, though, although it wasn't my first instinct.  Look
at it this way: in many typesetting languages, completely blank lines
separate paragraphs.  This isn't done in descriptions because blank
lines separate control file fields -- so we have the convention that
a full stop and nothing else stands in for a blank line.  It's not
part of the text structure and I don't think there's any need to indent
it.

> >And to come back to my suggested part of the fix of the problem(-range):
> >one (among other) ways to go would be to check *all* the descriptions 
> >and to make sure that they render well and if not to "fix" them [2]. 
> >This does not seem to be a infeasible thing. Once this is done, then 
> >there doesn't seem to be a technical hurdle to put the list syntax into 
> >the Debian policy?
> 
> all existing packages are rendered correctly, even with the change of 
> syntax above then the need for backward compatiblity will be remain for an 
> empty set of packages. I can see your point about backward compatibilty, 
> but IMHO it's not really sensible to require it even if no existing 
> package will be affected any more...
> 
> Currently there are 1729 packages that contain bullets that will be 
> interpreted as such by aptitude and thus would need to be checked.
> 
> Using a bit of scripting it should be possible to do the later in a day or 
> so.

  I did run a quick check the other day to find out what was (potentially)
affected by this particular problem, and there were only a handful of
packages using full stops at the beginning of non-blank lines.  I haven't
thoroughly checked the rest of the archive, but I also haven't heard
complaints about any other rendering problems with bullets.

  Daniel
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Daniel Burrows <dburrows@debian.org>:
Bug#373888; Package aptitude. Full text and rfc822 format available.

Acknowledgement sent to Tomas Pospisek <tpo_deb@sourcepole.ch>:
Extra info received and forwarded to list. Copy sent to Daniel Burrows <dburrows@debian.org>. Full text and rfc822 format available.

Message #48 received at 373888@bugs.debian.org (full text, mbox):

From: Tomas Pospisek <tpo_deb@sourcepole.ch>
To: 373888@bugs.debian.org
Subject: Re: Bug#373888: Bug#388594: take into account lists in descriptions when wrapping
Date: Thu, 28 Sep 2006 17:44:04 +0200 (CEST)
On Mon, 25 Sep 2006, Daniel Burrows wrote:

> On Mon, Sep 25, 2006 at 10:37:48PM +0200, Tomas Pospisek <tpo_deb@sourcepole.ch> was heard to say:
>> On Sun, 24 Sep 2006, Daniel Burrows wrote:
>>> Ah.  The problem is that aptitude parses indented regions using the
>>> standard grammar, but with N spaces stripped from each line before
>>> applying the grammar.  This allows multi-paragraph list entries, like:
>>>
>>> * One list entry
>>> * Another list entry.
>>>   .
>>>   Here is some more text describing the list entry.
>>>
>>> The result of treating the last line as part of the list entry is that
>>> it will be indented to exactly the same level as the rest of the entry,
>>> and it will be word-wrapped instead of being treated as verbatim text.
>>>
>>> Using the standard grammar also allows verbatim text in lists.  This
>>> was another reason for always stripping exactly one space after the
>>> bullet; it allows you to write
>>>
>>> *  Homepage: http://some.url.example.com
>>>
>>> and have it not get word-wrapped. (previously this would be considered
>>> a normal bullet point with 2 spaces of indentation beyond the bullet)
>>>
>>> Removing the ability to write multi-paragraph list entries is not
>>> appealing to me.
>>
>> Mh. Well I understand a bit better now, thanks a lot. So the conclusion
>> is that the grammar/parser needs a change. However:
>>
>>> Now that I look at this from a fresh perspective,
>>> though, there is one other choice.  I could consider text following
>>> a bullet that's at the same indentation level and separated only by
>>> blank lines to be part of the bullet, as in:
>>>
>>> * One list entry
>>> * Another list entry.
>>> .
>>>   Here is some more text describing the list entry.
>>
>> This is very ugly. Why not allow the grammar as it was and instead say
>> that:
>>
>>    * One list entry
>>      .
>>      Some more text
>>
>> only this form will be accepted as a blank line and none other. That means
>> all other cases such as:
>
>  This might be reasonable as an alternative.  It does mean, though,
> that if policy starts to define a meaning for " .asdf\n", there isn't
> a clean way of adding support for that within lists.
>
>  I'm not sure that just using the same paragraph separator everywhere
> is all that ugly, though, although it wasn't my first instinct.  Look
> at it this way: in many typesetting languages, completely blank lines
> separate paragraphs.  This isn't done in descriptions because blank
> lines separate control file fields -- so we have the convention that
> a full stop and nothing else stands in for a blank line.  It's not
> part of the text structure and I don't think there's any need to indent
> it.

At the end of the day your solution sounds reasonable to me...

>>> And to come back to my suggested part of the fix of the problem(-range):
>>> one (among other) ways to go would be to check *all* the descriptions
>>> and to make sure that they render well and if not to "fix" them [2].
>>> This does not seem to be a infeasible thing. Once this is done, then
>>> there doesn't seem to be a technical hurdle to put the list syntax into
>>> the Debian policy?
>>
>> all existing packages are rendered correctly, even with the change of
>> syntax above then the need for backward compatiblity will be remain for an
>> empty set of packages. I can see your point about backward compatibilty,
>> but IMHO it's not really sensible to require it even if no existing
>> package will be affected any more...
>>
>> Currently there are 1729 packages that contain bullets that will be
>> interpreted as such by aptitude and thus would need to be checked.
>>
>> Using a bit of scripting it should be possible to do the later in a day or
>> so.
>
>  I did run a quick check the other day to find out what was (potentially)
> affected by this particular problem, and there were only a handful of
> packages using full stops at the beginning of non-blank lines.  I haven't
> thoroughly checked the rest of the archive, but I also haven't heard
> complaints about any other rendering problems with bullets.

I might go through them and check them.
*t

--
-----------------------------------------------------------------------
     Tomas Pospisek
     http://sourcepole.com -   Linux & Open Source Solutions
------------------------------------------------------------------------



Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Wed, 27 Jun 2007 00:33:03 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Thu Apr 24 05:27:52 2014; Machine Name: buxtehude.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.