Debian Bug report logs - #333231
flex generates a C file with improper macro defn; broke Bison 2.1

version graph

Package: flex; Maintainer for flex is Manoj Srivastava <srivasta@debian.org>; Source for flex is src:flex.

Reported by: Paul Eggert <eggert@CS.UCLA.EDU>

Date: Tue, 11 Oct 2005 00:18:15 UTC

Severity: wishlist

Tags: patch, upstream

Found in version flex/2.5.31-34

Fixed in version flex/2.5.31-36

Done: Manoj Srivastava <srivasta@debian.org>

Bug is archived. No further changes may be made.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Manoj Srivastava <srivasta@debian.org>:
Bug#333231; Package flex. Full text and rfc822 format available.

Acknowledgement sent to Paul Eggert <eggert@CS.UCLA.EDU>:
New Bug report received and forwarded. Copy sent to Manoj Srivastava <srivasta@debian.org>. Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Paul Eggert <eggert@CS.UCLA.EDU>
To: submit@bugs.debian.org
Subject: flex generates a C file with improper macro defn; broke Bison 2.1
Date: Mon, 10 Oct 2005 17:09:45 -0700
Package: flex
Version: 2.5.31-34
Tags: patch upstream

flex 2.5.31-34 generates a C file that does not conform to the C
standard, because the code invokes gram_wrap() even though the
gram_wrap macro expects an argument.

This bug is also present in Debian stable, and its presence broke
Bison 2.1 because I used Debian stable flex to build Bison.  The bug's
effect is described in
<http://lists.gnu.org/archive/html/bug-bison/2004-02/msg00026.html>;
that message is for Bison 1.875 but the same problem occurs in the
Bison 2.1 distribution because of the bug in flex 2.5.31-31.
(I used flex 2.5.31-31 to generate both Bison 1.875 and Bison 2.1.)

I see from
<http://ftp.debian.org/debian/pool/main/f/flex/flex_2.5.31-34.diff.gz>
that the file
flex-2.5.31/{arch}/flex/flex--debian/flex--debian--2.5/srivasta@debian.org--2003-primary/patch-log/patch-2
logs the presence of the bug, but there isn't any patch installed.  I
suspect the patch got lost somehow so I'm filing this bug report.
Here is a patch.

2005-10-10  Paul Eggert  <eggert@cs.ucla.edu>

	* main.c (readin): Don't output an invalid macro definition for
	yywrap; some C compilers complain about this.

--- flex-2.5.31-34/main.c	2005-10-10 16:43:31.000000000 -0700
+++ flex-2.5.31-34-yywrap/main.c	2005-10-10 16:49:11.000000000 -0700
@@ -1560,7 +1560,7 @@ void readin ()
     }
 
 	if (!do_yywrap) {
-		outn ("\n#define yywrap(n) 1");
+		outn ("\n#define yywrap() 1");
 		outn ("#define YY_SKIP_YYWRAP");
 	}
 



Reply sent to Manoj Srivastava <srivasta@debian.org (va, manoj)>:
You have taken responsibility. Full text and rfc822 format available.

Notification sent to Paul Eggert <eggert@CS.UCLA.EDU>:
Bug acknowledged by developer. Full text and rfc822 format available.

Message #10 received at 333231-done@bugs.debian.org (full text, mbox):

From: Manoj Srivastava <srivasta@debian.org (va, manoj)>
To: Paul Eggert <eggert@CS.UCLA.EDU>
Cc: 333231-done@bugs.debian.org
Subject: Re: Bug#333231: flex generates a C file with improper macro defn; broke Bison 2.1
Date: Fri, 21 Oct 2005 01:28:44 -0500
On Mon, 10 Oct 2005 17:09:45 -0700, Paul Eggert <eggert@CS.UCLA.EDU> said: 

> flex 2.5.31-34 generates a C file that does not conform to the C
> standard, because the code invokes gram_wrap() even though the
> gram_wrap macro expects an argument.

        You can leave macro arguments empty; this is not an error.  You
 cannot leave out arguments entirely; if a macro takes two arguments,
 there must be exactly one comma at the top level of its argument
 list. (See $6.10.2, and $6.10.3, (C)ISO/IEC ISO/IEC 9899:1999 (E) for
 details).

> I see from
>> http://ftp.debian.org/debian/pool/main/f/flex/flex_2.5.31-34.diff.gz>
> that the file
> flex-2.5.31/{arch}/flex/flex--debian/flex--debian--2.5/srivasta@debian.org--2003-primary/patch-log/patch-2
> logs the presence of the bug, but there isn't any patch installed.
> I suspect the patch got lost somehow so I'm filing this bug report.
> Here is a patch.

        Actually, no. The fix was removed since it caused all the
 tests to fail, and is strictly correct according to c99. Also, I did
 test to see if this was a serious problem: These are pretty
 aggressive warning flags
,----
| srivasta@cinder:~/flex-2.5.31/tests/test-array-nr$  i486-linux-gnu-gcc \
|   -std=gnu99 -pedantic -Wall -W -Wconversion -Wshadow  -Wpointer-arith \
|   -Wcast-qual -Wcast-align -Wunreachable-code  -Wbad-function-cast    \
|   -Wmissing-declarations -Wmissing-prototypes -Wwrite-strings         \
|   -Waggregate-return -Wpadded -W -Wfloat-equal -Wshadow -Wconversion  \
|   -Wchar-subscripts -Wnested-externs -Wstrict-prototypes  -Wformat    \
|   -Wcomment -fshort-enums -fno-common -Dgets=DONT_USE_GETS -Dlint     \
|   -Wpacked  -Winline -c -o scanner.o -I. -I../.. -I../..  -g -O2      \
|   scanner.c  
| scanner.l: In function 'testlex':
| scanner.l:43: warning: passing argument 2 of 'fwrite' as unsigned due to prototype
| scanner.c: In function 'test_scan_string':
| scanner.c:1430: warning: passing argument 2 of 'test_scan_bytes' as signed due to prototype
| scanner.c: At top level:
| scanner.c:1503: warning: no previous prototype for 'testget_lineno'
| scanner.c:1512: warning: no previous prototype for 'testget_in'
| scanner.c:1520: warning: no previous prototype for 'testget_out'
| scanner.c:1528: warning: no previous prototype for 'testget_leng'
| scanner.c:1537: warning: no previous prototype for 'testget_text'
| scanner.c:1546: warning: no previous prototype for 'testset_lineno'
| scanner.c:1558: warning: no previous prototype for 'testset_in'
| scanner.c:1563: warning: no previous prototype for 'testset_out'
| scanner.c:1568: warning: no previous prototype for 'testget_debug'
| scanner.c:1573: warning: no previous prototype for 'testset_debug'
| scanner.c:1579: warning: no previous prototype for 'testlex_destroy'
| scanner.c: In function 'testlex':
| scanner.c:823: warning: will never be executed
| scanner.c:693: warning: will never be executed
| scanner.c:673: warning: will never be executed
| scanner.c: In function 'testpop_buffer_state':
| scanner.c:1331: warning: will never be executed
| scanner.c: In function 'testlex_destroy':
| scanner.c:1583: warning: will never be executed
`----

        manoj

-- 
You're at the end of the road again.
Manoj Srivastava   <srivasta@debian.org>  <http://www.debian.org/%7Esrivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B  924B 21BA DABB BF24 424C



Information forwarded to debian-bugs-dist@lists.debian.org, Manoj Srivastava <srivasta@debian.org>:
Bug#333231; Package flex. Full text and rfc822 format available.

Acknowledgement sent to Paul Eggert <eggert@CS.UCLA.EDU>:
Extra info received and forwarded to list. Copy sent to Manoj Srivastava <srivasta@debian.org>. Full text and rfc822 format available.

Message #15 received at 333231@bugs.debian.org (full text, mbox):

From: Paul Eggert <eggert@CS.UCLA.EDU>
To: 333231@bugs.debian.org
Subject: Re: Bug#333231 acknowledged by developer (Re: Bug#333231: flex generates a C file with improper macro defn; broke Bison 2.1)
Date: Fri, 21 Oct 2005 00:58:25 -0700
Manoj Srivastava <srivasta@debian.org (va, manoj)> writes:

> and is strictly correct according to c99.

But the generated code is not correct according to C89.  In C89, the
behavior is undefined if a macro defined with one argument is called
with zero arguments.  Please see the official interpretation for
Defect Report #003, question 3, in
<http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_003.html#Question3>.

Surely flex is not intended to require C99; many compilers don't
support C99 yet, or don't fully support it.  GCC is in that category,
and people are reporting problems with GCC here.


> The fix was removed since it caused all the tests to fail

Sorry about that.  I forgot about the reentrant case.  Here is a
revised patch; I checked that "make check" works.

2005-10-21  Paul Eggert  <eggert@cs.ucla.edu>

	* main.c (readin): Don't output an invalid macro definition for
	yywrap; some C compilers complain about this.

--- flex-2.5.31-34/main.c	2005-10-10 16:43:31.000000000 -0700
+++ flex-2.5.31-34-yywrap/main.c	2005-10-21 00:53:40.000000000 -0700
@@ -1560,7 +1560,10 @@ void readin ()
     }
 
 	if (!do_yywrap) {
-		outn ("\n#define yywrap(n) 1");
+		if (reentrant)
+			outn ("\n#define yywrap(n) 1");
+		else
+			outn ("\n#define yywrap() 1");
 		outn ("#define YY_SKIP_YYWRAP");
 	}
 



Message #16 received at 333231-done@bugs.debian.org (full text, mbox):

From: Paul Eggert <eggert@CS.UCLA.EDU>
To: 333231-done@bugs.debian.org
Subject: Re: Bug#333231: flex generates a C file with improper macro defn; broke Bison 2.1
Date: Fri, 21 Oct 2005 00:57:10 -0700
Manoj Srivastava <srivasta@debian.org (va, manoj)> writes:

> and is strictly correct according to c99.

The generated code is not correct according to C89.  In C89, the
behavior is undefined if a macro defined with one argument is called
with zero arguments.  Please see the official interpretation for
Defect Report #003, question 3, in
<http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_003.html#Question3>.

Surely flex is not intended to require C99; many compilers don't
support C99 yet, or don't fully support it.  GCC is in that category,
and people are reporting problems with GCC here.


> The fix was removed since it caused all the tests to fail

Sorry about that.  I forgot about the reentrant case.  Here is a
revised patch; I checked that "make check" works.

2005-10-21  Paul Eggert  <eggert@cs.ucla.edu>

	* main.c (readin): Don't output an invalid macro definition for
	yywrap; some C compilers complain about this.

--- flex-2.5.31-34/main.c	2005-10-10 16:43:31.000000000 -0700
+++ flex-2.5.31-34-yywrap/main.c	2005-10-21 00:53:40.000000000 -0700
@@ -1560,7 +1560,10 @@ void readin ()
     }
 
 	if (!do_yywrap) {
-		outn ("\n#define yywrap(n) 1");
+		if (reentrant)
+			outn ("\n#define yywrap(n) 1");
+		else
+			outn ("\n#define yywrap() 1");
 		outn ("#define YY_SKIP_YYWRAP");
 	}
 



Information forwarded to debian-bugs-dist@lists.debian.org, Manoj Srivastava <srivasta@debian.org>:
Bug#333231; Package flex. Full text and rfc822 format available.

Acknowledgement sent to Manoj Srivastava <srivasta@debian.org (va, manoj)>:
Extra info received and forwarded to list. Copy sent to Manoj Srivastava <srivasta@debian.org>. Full text and rfc822 format available.

Message #21 received at 333231@bugs.debian.org (full text, mbox):

From: Manoj Srivastava <srivasta@debian.org (va, manoj)>
To: Paul Eggert <eggert@CS.UCLA.EDU>
Cc: 333231@bugs.debian.org, control@bugs.debian.org
Subject: Re: Bug#333231: acknowledged by developer (Re: Bug#333231: flex generates a C file with improper macro defn; broke Bison 2.1)
Date: Sat, 22 Oct 2005 00:33:08 -0500
reopen 333231
severity 333231 wishlist
thanks

Hi,

> But the generated code is not correct according to C89.  In C89, the
> behavior is undefined if a macro defined with one argument is called
> with zero arguments.  Please see the official interpretation for
> Defect Report #003, question 3, in
> <http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_003.html#Question3>.

        Quite so. C89, however, is 16 years old now. C99 is 6 years
 old. How long must we hold on to outdated and obsolete versions of
 the language?

> Surely flex is not intended to require C99; many compilers don't
> support C99 yet, or don't fully support it.

        Err, not so. From the upstream web site:

,----[ http://lex.sourceforge.net/ ]
| C99 is the latest C standard, and we aim to comply as much as is
| reasonably possible. Supporting non-standard compilers is very, very
| low on our priority list.
`----

> GCC is in that category, and people are reporting problems with GCC
> here.

        Err. I posted a report with the version of GCC shipping along
 side the version of flex we are talking about:


,----[ gcc options ]
| gcc -std=gnu99 -pedantic -Wall -W -Wconversion -Wshadow \
|     -Wpointer-arith -Wcast-qual -Wcast-align -Wunreachable-code \
|     -Wbad-function-cast  -Wmissing-declarations -Wmissing-prototypes \
|     -Wwrite-strings -Waggregate-return -Wpadded -W \
|     -Wfloat-equal -Wshadow -Wconversion -Wchar-subscripts \
|     -Wnested-externs -Wstrict-prototypes  -Wformat -Wcomment \
|     -fshort-enums -fno-common -Dgets=DONT_USE_GETS -Dlint -Wpacked \
|     -Winline 
`----

        With all the options turned on, the version of the compiler
 that Debian is shipping did not emit any diagnostics about the
 standards compliant macro usage in the file generated by flex.

        Now, having said that, I have not yet decided no not apply
 your patch --I just don't think this is a bug, just a feature request
 to support non-standards compliant compilers. I do see the utility of
 allowing people to develop on Debian and yet have their code compile
 on ancient platforms with non-compliant compilers.

        manoj
-- 
He who has experienced the end of his suffering here in this life, who
has set down the burden, freed! - that is what I call a brahmin. 402
Manoj Srivastava   <srivasta@debian.org>  <http://www.debian.org/%7Esrivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B  924B 21BA DABB BF24 424C



Bug reopened, originator not changed. Request was from Manoj Srivastava <srivasta@debian.org (va, manoj)> to control@bugs.debian.org. Full text and rfc822 format available.

Severity set to `wishlist'. Request was from Manoj Srivastava <srivasta@debian.org (va, manoj)> to control@bugs.debian.org. Full text and rfc822 format available.

Reply sent to Manoj Srivastava <srivasta@debian.org>:
You have taken responsibility. Full text and rfc822 format available.

Notification sent to Paul Eggert <eggert@CS.UCLA.EDU>:
Bug acknowledged by developer. Full text and rfc822 format available.

Message #30 received at 333231-close@bugs.debian.org (full text, mbox):

From: Manoj Srivastava <srivasta@debian.org>
To: 333231-close@bugs.debian.org
Subject: Bug#333231: fixed in flex 2.5.31-36
Date: Sun, 23 Oct 2005 13:47:52 -0700
Source: flex
Source-Version: 2.5.31-36

We believe that the bug you reported is fixed in the latest version of
flex, which is due to be installed in the Debian FTP archive:

flex-doc_2.5.31-36_all.deb
  to pool/main/f/flex/flex-doc_2.5.31-36_all.deb
flex_2.5.31-36.diff.gz
  to pool/main/f/flex/flex_2.5.31-36.diff.gz
flex_2.5.31-36.dsc
  to pool/main/f/flex/flex_2.5.31-36.dsc
flex_2.5.31-36_i386.deb
  to pool/main/f/flex/flex_2.5.31-36_i386.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 333231@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Manoj Srivastava <srivasta@debian.org> (supplier of updated flex package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Sun, 23 Oct 2005 14:59:05 -0500
Source: flex
Binary: flex-doc flex
Architecture: source i386 all
Version: 2.5.31-36
Distribution: unstable
Urgency: low
Maintainer: Manoj Srivastava <srivasta@debian.org>
Changed-By: Manoj Srivastava <srivasta@debian.org>
Description: 
 flex       - A fast lexical analyzer generator.
 flex-doc   - Documentation for flex (a fast lexical analyzer generator)
Closes: 320134 333231
Changes: 
 flex (2.5.31-36) unstable; urgency=low
 .
   * Bug fix: "flex: doxygen warning: argument `str' of command @param is
     not found in the argumentlist", thanks to Ludovic Rousseau
                                                            (Closes: #320134).
   * Bug fix: "flex generates a C file with improper macro defn; broke
     Bison 2.1", thanks to Paul Eggert. Well, even though it is legal C99
     to have empty arguments in macros (so it is legal to call a macro
     taking one argument with no arguments), it was not legal C89 --
     therefore, emit tqo different macros, with and without arguments -- to
     cater to users running obsolete compilers.             (Closes: #333231).
Files: 
 a211839fdc8470dcd9fe53ab0ed80357 670 devel standard flex_2.5.31-36.dsc
 7ed573f6e5f9cf7ddea521bc06bedb8a 211072 devel standard flex_2.5.31-36.diff.gz
 63bb249889420feb37e0fccf82ad0eb0 149646 doc optional flex-doc_2.5.31-36_all.deb
 f57e8713f0e4302dc3ad34e1b7c1f66c 268566 devel standard flex_2.5.31-36_i386.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)

iD8DBQFDW/QjIbrau78kQkwRAn+TAKCUfNl6OP+DQYZuRlGPYrghbKfCaQCbBjwr
e3pd/yOGyPYgk3LwyZL2e58=
=yK3x
-----END PGP SIGNATURE-----




Information forwarded to debian-bugs-dist@lists.debian.org, Manoj Srivastava <srivasta@debian.org>:
Bug#333231; Package flex. Full text and rfc822 format available.

Acknowledgement sent to Paul Eggert <eggert@CS.UCLA.EDU>:
Extra info received and forwarded to list. Copy sent to Manoj Srivastava <srivasta@debian.org>. Full text and rfc822 format available.

Message #35 received at 333231@bugs.debian.org (full text, mbox):

From: Paul Eggert <eggert@CS.UCLA.EDU>
To: 333231@bugs.debian.org
Subject: Re: Bug#333231: acknowledged by developer (Re: Bug#333231: flex generates a C file with improper macro defn; broke Bison 2.1)
Date: Sun, 23 Oct 2005 21:55:11 -0700
Thanks for installing that patch into Debian; that will simplify my
life once it gets into Debian stable.

Here is some followup, which is now only of historical interest:

Manoj Srivastava <srivasta@debian.org (va, manoj)> writes:

>  C89, however, is 16 years old now. C99 is 6 years old. How long
>  must we hold on to outdated and obsolete versions of the language?

So long as any project that uses 'flex' wants to generate
distributions that can be built by older compilers.

To give you an idea of how long that might be, when operating in yacc
mode, Bison still supports K&R C compilers, even though these have
been obsolete since 1989.  We are thinking of removing the K&R support
from Bison next year, which would be about 17 years after C89 came
out.  But this is because we've done a preliminary survey of
applications using Bison, and nobody cares about K&R C compilers any
more.  There are still a few people who care about porting to K&R C
libraries like the SunOS 4.x library, so we'll continue to be be
portable to those for a while longer; this is a separate issue from
the compiler proper.  (I also plan to ask on the Bison forums about
this before making the change.)

>  I posted a report with the version of GCC shipping along
>  side the version of flex we are talking about:

I'm more worried about the case where people like me build
distributions on Debian stable, and people on less-advanced systems
try to use their system's C compiler.  Older GCC versions complain
about the code in question, as do many non-GCC implementations that
are still widely-used enough to cause bug reports to be sent to us.



Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Mon, 25 Jun 2007 02:41:45 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Wed Apr 16 04:12:55 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.