Debian Bug report logs - #259887
[PR 16601] [3.3 regression] miscompilation of automatic dynamic arrays in crypto/IPsec subsystems of the Linux kernel

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: <herbert@gondor.apana.org.au>

To: submit@bugs.debian.org

Subject: gcc-3.3: Miscompiles automatic dynamic arrays

Date: Sat, 17 Jul 2004 15:34:09 +1000

Package: gcc-3.3
Version: 1:3.3.4-3
Severity: critical

With the option -mpreferred-stack-boundary=2, gcc 3.3.4 is miscompiling
automatic dynamic arrays.  Unfortunately both are used in the
crypto/IPsec subsystems of the Linux kernel.

Here is a sample program:

#include <string.h>

int bar(char *s);

int foo(char *s, int len, int x)
{
	char buf[x ? len : 0];

	if (x) {
		memcpy(buf, s, len);
		s = buf;
	}

	return bar(s);
}

With gcc 3.3.4, this produces:

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
	.file	"b.c"
	.text
	.p2align 4,,15
.globl foo
	.type	foo, @function
foo:
	pushl	%ebp
	xorl	%eax, %eax
	movl	%esp, %ebp
	subl	$24, %esp
	movl	16(%ebp), %ecx
	movl	%edi, -4(%ebp)
	movl	12(%ebp), %edx
	movl	%esp, %edi
	movl	%ebx, -12(%ebp)
	movl	%esi, -8(%ebp)
	decl	%edx
	movl	8(%ebp), %esi
	testl	%ecx, %ecx
	setne	%al
	decl	%eax
	orl	%eax, %edx
	addl	$19, %edx
	andl	$-4, %edx
---------------------------------------------------------------------
	subl	%edx, %esp
	leal	27(%esp), %ebx
	andl	$-16, %ebx

Note the offset 27.  The same program when compiled with gcc 3.2.3
produces similar output but it uses an offset of 15.

Suppose that len = 16, x != 0, and %esp & 15 = 8 before the subl.

That means %edx = (15 + 19) & ~3 = 32.  So %esp & 15 is still 8
after the subtraction.  That is, %esp = 16x + 8.  Hence
%ebx = (%esp + 27) & ~15 = (16x + 35) & ~15 = 16x + 32 = %esp + 24.

Therefore buf will only contain 8 bytes of space instead of 16
bytes.
---------------------------------------------------------------------
	testl	%ecx, %ecx
	jne	.L5
.L4:
	movl	%esi, (%esp)
	call	bar
	movl	%edi, %esp
	movl	-12(%ebp), %ebx
	movl	-8(%ebp), %esi
	movl	-4(%ebp), %edi
	movl	%ebp, %esp
	popl	%ebp
	ret
	.p2align 4,,7
.L5:
	movl	12(%ebp), %eax
	movl	%esi, 4(%esp)
	movl	%ebx, %esi
	movl	%eax, 8(%esp)
	movl	%ebx, (%esp)
	call	memcpy
	jmp	.L4
	.size	foo, .-foo
	.section	.note.GNU-stack,"",@progbits
	.ident	"GCC: (GNU) 3.3.4 (Debian 1:3.3.4-3)"
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Since this bug can lead to remotely triggered crashes and possibly
exploits I'm rating it as critical.

-- System Information
Debian Release: testing/unstable
Kernel Version: Linux gondolin 2.4.26-1-686-smp #1 SMP Sat May 1 19:17:11 EST 2004 i686 GNU/Linux

Versions of the packages gcc-3.3 depends on:
ii  binutils       2.14.90.0.7-8  The GNU assembler, linker and binary utiliti
ii  cpp-3.3        3.3.4-1        The GNU C preprocessor
ii  gcc-3.3-base   3.3.4-1        The GNU Compiler Collection (base package)
ii  libc6          2.3.2.ds1-13   GNU C Library: Shared libraries and Timezone
ii  libgcc1        3.3.4-1        GCC support library

Message #10 received at 259887@bugs.debian.org (full text, mbox, reply):

From: Matthias Klose <doko@cs.tu-berlin.de>

To: <herbert@gondor.apana.org.au>, 259887@bugs.debian.org

Subject: Re: Bug#259887: gcc-3.3: Miscompiles automatic dynamic arrays

Date: Sat, 17 Jul 2004 09:27:13 +0200

herbert@gondor.apana.org.au writes:
> Package: gcc-3.3
> Version: 1:3.3.4-3
> Severity: critical
> 
> With the option -mpreferred-stack-boundary=2, gcc 3.3.4 is miscompiling
> automatic dynamic arrays.  Unfortunately both are used in the
> crypto/IPsec subsystems of the Linux kernel.

I assume the complete flags are -O2 -mpreferred-stack-boundary=2 ? Can
you provide a workaround for the kernel source?

> Here is a sample program:
> 
> #include <string.h>
> 
> int bar(char *s);
> 
> int foo(char *s, int len, int x)
> {
> 	char buf[x ? len : 0];
> 
> 	if (x) {
> 		memcpy(buf, s, len);
> 		s = buf;
> 	}
> 
> 	return bar(s);
> }

Message #21 received at 259887-quiet@bugs.debian.org (full text, mbox, reply):

From: Matthias Klose <doko@mail.net.local>

To: control@bugs.debian.org

Cc: 259887-quiet@bugs.debian.org

Subject: gcc: submitted Debian report #259887 to gcc-gnats as PR 16601

Date: Sat, 17 Jul 2004 09:23:20 +0200

# submitted Debian report #259887 to gcc-gnats as PR 16601
# http://gcc.gnu.org/PR16601

forwarded 259887 http://gcc.gnu.org/PR16601
retitle 259887 [PR 16601] [3.3 regression] miscompilation of automatic dynamic arrays in crypto/IPsec subsystems of the Linux kernel
tags 259887 + upstream
thanks

Message #26 received at 259887@bugs.debian.org (full text, mbox, reply):

From: Herbert Xu <herbert@gondor.apana.org.au>

To: Matthias Klose <doko@cs.tu-berlin.de>

Cc: 259887@bugs.debian.org

Subject: Re: Bug#259887: gcc-3.3: Miscompiles automatic dynamic arrays

Date: Sat, 17 Jul 2004 17:36:59 +1000

On Sat, Jul 17, 2004 at 09:27:13AM +0200, Matthias Klose wrote:
> 
> I assume the complete flags are -O2 -mpreferred-stack-boundary=2 ? Can

Sorry, yes that's what I used.  The kernel adds a few more options like
-fomit-frame-pointer but it doesn't make any differences to the problem.

> you provide a workaround for the kernel source?

The simplest work around is to allocate things using kmalloc instead
of the stack.  But I need to discuss it with the others first.

Thanks,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Message #31 received at 259887@bugs.debian.org (full text, mbox, reply):

From: Matthias Klose <doko@cs.tu-berlin.de>

To: Herbert Xu <herbert@gondor.apana.org.au>

Cc: control@bugs.debian.org.259887@bugs.debian.org

Subject: Re: Bug#259887: gcc-3.3: Miscompiles automatic dynamic arrays

Date: Sat, 17 Jul 2004 09:43:06 +0200 (MEST)

severity 259887 important
thanks

lowering the severity, so that the current packages can enter testing
tonight (the bug is present in the 3.3.4-2 package as well).
re-raising the severity after the package has reached testing.

Message #38 received at 259887-done@bugs.debian.org (full text, mbox, reply):

From: Herbert Xu <herbert@gondor.apana.org.au>

To: 259887-done@bugs.debian.org

Subject: Re: Bug#259887: gcc-3.3: Miscompiles automatic dynamic arrays

Date: Sat, 17 Jul 2004 19:47:22 +1000

On Sat, Jul 17, 2004 at 05:36:59PM +1000, herbert wrote:
> On Sat, Jul 17, 2004 at 09:27:13AM +0200, Matthias Klose wrote:
> > 
> > I assume the complete flags are -O2 -mpreferred-stack-boundary=2 ? Can
> 
> Sorry, yes that's what I used.  The kernel adds a few more options like
> -fomit-frame-pointer but it doesn't make any differences to the problem.

I'm sorry but I got it wrong.

gcc 3.3.4 is unconditionally allocating 12 bytes of extra room at
the start of the function.  Since the most it can go over by is
11 bytes (when %esp & ~15 = 5), this is safe.

Sorry for the noise.
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Send a report that this bug log contains spam.