Debian Bug report logs - #641146
openttd: Swapped colors on sparc64

version graph

Package: openttd; Maintainer for openttd is Matthijs Kooijman <matthijs@stdin.nl>; Source for openttd is src:openttd.

Reported by: BERTRAND Joël <joel.bertrand@systella.fr>

Date: Sat, 10 Sep 2011 19:33:02 UTC

Severity: normal

Found in version openttd/1.1.2-1

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Matthijs Kooijman <matthijs@stdin.nl>:
Bug#641146; Package openttd. (Sat, 10 Sep 2011 19:33:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to BERTRAND Joël <joel.bertrand@systella.fr>:
New Bug report received and forwarded. Copy sent to Matthijs Kooijman <matthijs@stdin.nl>. (Sat, 10 Sep 2011 19:33:05 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: BERTRAND Joël <joel.bertrand@systella.fr>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: openttd: Swapped colors on sparc64
Date: Sat, 10 Sep 2011 21:21:40 +0200
Package: openttd
Version: 1.1.2-1
Severity: normal

Dear Maintainer,

On sparc64, openttd colors are swapped (green is green, but red seems to be
replaced by blue and blue by red). I think there is an endianess trouble in
openttd.

Regards,

JB

-- System Information:
Debian Release: wheezy/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable'), (500, 'stable'), (1, 'experimental')
Architecture: sparc (sparc64)

Kernel: Linux 2.6.36.2 (SMP w/1 CPU core)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages openttd depends on:
ii  libc6            2.13-18              
ii  libfontconfig1   2.8.0-3              
ii  libfreetype6     2.4.6-2              
ii  libgcc1          1:4.6.1-4            
ii  libicu44         4.4.2-2              
ii  liblzma2         5.1.1alpha+20110809-2
ii  liblzo2-2        2.05-2               
ii  libpng12-0       1.2.46-3             
ii  libsdl1.2debian  1.2.14-6.4           
ii  libstdc++6       4.6.1-4              
ii  openttd-data     1.1.2-1              
ii  zlib1g           1:1.2.3.4.dfsg-3     

Versions of packages openttd recommends:
ii  openttd-opengfx  0.3.5-1     
ii  openttd-openmsx  0.3.1-1     
ii  timidity         2.13.2-39+b1
ii  x11-utils        7.6+3       

Versions of packages openttd suggests:
pn  openttd-opensfx  <none>

-- no debconf information




Information forwarded to debian-bugs-dist@lists.debian.org, Matthijs Kooijman <matthijs@stdin.nl>:
Bug#641146; Package openttd. (Sat, 17 Sep 2011 20:22:33 GMT) Full text and rfc822 format available.

Acknowledgement sent to rubidium@openttd.org:
Extra info received and forwarded to list. Copy sent to Matthijs Kooijman <matthijs@stdin.nl>. (Sat, 17 Sep 2011 20:22:33 GMT) Full text and rfc822 format available.

Message #10 received at 641146@bugs.debian.org (full text, mbox):

From: Rubidium <rubidium@openttd.org>
To: 641146@bugs.debian.org
Subject: re: openttd: Swapped colors on sparc64
Date: Sat, 17 Sep 2011 21:53:29 +0200
Hi,

I've just tried both OpenTTD 1.0.4-3 and 1.1.2-1 on a PowerPC install of 
Squeeze (in QEMU using the image from aurel32's site). For 1.1.2-1 I 
installed a newer libstdc++, gcc-4.6-base and multiarchsupport. The 
installed SDL version is libsdl1.2debian-alsa 1.2.14-6.1.

In any case, in both cases OpenTTD colours are not messed up, i.e. they 
work fine. Given PowerPC is BE like Sparc, I'm assuming that OpenTTD is 
not having an endianness issue here but rather libsdl. If you are able 
to compile OpenTTD yourself it might be worth to try OpenTTD compiled 
with allegro. Then make sure allegro gets chosen by using openttd -v 
allegro.

Alternatively you can see whether openttd -b 32bpp-optimized works 
around the issue. If that's the case, then I fear libsdl is doing 
something incorrectly with the palette. Or at least inconsistently over 
the different platforms.

Is there a QEMU image for sparc64, or does QEMU still not support 
sparc64? I've read something about them not supporting 64 bits well 
enough when the sparc kernel became 64 bits, but could not find anything 
about the recent state. Without that I, sadly enough, can't be of much 
help in figuring out what goes wrong exactly.

Regards,
Rubidium




Information forwarded to debian-bugs-dist@lists.debian.org, Matthijs Kooijman <matthijs@stdin.nl>:
Bug#641146; Package openttd. (Sat, 07 Apr 2012 06:00:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to rbijker@rbijker.net:
Extra info received and forwarded to list. Copy sent to Matthijs Kooijman <matthijs@stdin.nl>. (Sat, 07 Apr 2012 06:00:03 GMT) Full text and rfc822 format available.

Message #15 received at 641146@bugs.debian.org (full text, mbox):

From: "R. Bijker" <rbijker@rbijker.net>
To: 641146@bugs.debian.org
Subject: re: openttd: Swapped colors on sparc64
Date: Sat, 07 Apr 2012 07:29:08 +0200
Hi,

trying the same on a SPARC (not 64) QEMU shows something somewhat 
interesting. First of all QEMU only supports 8bpp colours. Secondly, 
when starting OpenTTD via either Allegro or SDL in 8bpp mode makes the 
colours of the rest of the interface go totally haywire whereas the 
colours in-game are correct. Thirdly, using 32bpp the colours are 
definitely wrong, however changing the order from ARGB to ABGR makes the 
colours look better but they are still wrong. I have no idea what is 
going on there though. In any case, x86 uses BGRA (i.e. ARGB reversed), 
PPC uses ARGB and now it seems SPARC uses ABGR. So it's not an Endian 
problem, but some strange SPARC implementation detail issue.

The only caveat of changing ARGB -> ABGR is that the screenshots will 
then be incorrectly coloured, so somewhere during writing data to the 
screen the bits need to be reordered. The main question is, however, how 
can we detect that we need to reorder bits? As when we don't need it, we 
shouldn't waste CPU on doing basically a no-op, and we would like to 
know whether there are more peculiar targets.

In any case, I have no idea whether this also happens on SPARC64. For 
example, I don't even know whether you used 32bpp or 8bpp OpenTTD. If it 
is 8bpp OpenTTD, which I assume to be the case as that's default and you 
didn't mention switching to the 32bpp blitter, then this issue isn't 
your issue and I'd rather argue that it is a bug in SDL given we set the 
right values in the SDL_Color structure.

Regards,
Rubidium




Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#641146; Package openttd. (Mon, 04 Jun 2012 11:06:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Matthijs Kooijman <matthijs@stdin.nl>:
Extra info received and forwarded to list. (Mon, 04 Jun 2012 11:06:11 GMT) Full text and rfc822 format available.

Message #20 received at 641146@bugs.debian.org (full text, mbox):

From: Matthijs Kooijman <matthijs@stdin.nl>
To: BERTRAND Joël <joel.bertrand@systella.fr>, 641146@bugs.debian.org
Subject: Re: Bug#641146: openttd: Swapped colors on sparc64
Date: Mon, 4 Jun 2012 12:23:33 +0200
[Message part 1 (text/plain, inline)]
Hi Joël,

> On sparc64, openttd colors are swapped (green is green, but red seems to be
> replaced by blue and blue by red). I think there is an endianess trouble in
> openttd.
> 
I've managed to reproduce the problem (or at least _a_ problem) on an
old Ultra1 I managed to get my hands on for this problem. I'm not quite
sure what's wrong yet, it seems the palette OpenTTD is setting is
somehow not (completely) used. I'll have to look into this a bit
further, but it might be some kind of bug/limitation in libSDL or the
hardware...

Could you tell me on what hardware you observed this problem?

In any case, this bug is not forgotten.

Gr.

Matthijs
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Matthijs Kooijman <matthijs@stdin.nl>:
Bug#641146; Package openttd. (Mon, 04 Jun 2012 18:09:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to BERTRAND Joel <joel.bertrand@systella.fr>:
Extra info received and forwarded to list. Copy sent to Matthijs Kooijman <matthijs@stdin.nl>. (Mon, 04 Jun 2012 18:09:03 GMT) Full text and rfc822 format available.

Message #25 received at 641146@bugs.debian.org (full text, mbox):

From: BERTRAND Joel <joel.bertrand@systella.fr>
To: Matthijs Kooijman <matthijs@stdin.nl>, 641146@bugs.debian.org
Subject: Re: Bug#641146: openttd: Swapped colors on sparc64
Date: Mon, 04 Jun 2012 19:53:09 +0200
Matthijs Kooijman a écrit :
> Hi Joël,
>
>> On sparc64, openttd colors are swapped (green is green, but red seems to be
>> replaced by blue and blue by red). I think there is an endianess trouble in
>> openttd.
>>
> I've managed to reproduce the problem (or at least _a_ problem) on an
> old Ultra1 I managed to get my hands on for this problem. I'm not quite
> sure what's wrong yet, it seems the palette OpenTTD is setting is
> somehow not (completely) used. I'll have to look into this a bit
> further, but it might be some kind of bug/limitation in libSDL or the
> hardware...
>
> Could you tell me on what hardware you observed this problem?

	Of course, Matthijs. I have seen this bug on my Blade2000 with 
Creator3D (ffb2+) in UPA slot.

	Regards,

	JKB




Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#641146; Package openttd. (Fri, 04 Jan 2013 18:12:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Matthijs Kooijman <matthijs@stdin.nl>:
Extra info received and forwarded to list. (Fri, 04 Jan 2013 18:12:03 GMT) Full text and rfc822 format available.

Message #30 received at 641146@bugs.debian.org (full text, mbox):

From: Matthijs Kooijman <matthijs@stdin.nl>
To: BERTRAND Joel <joel.bertrand@systella.fr>
Cc: 641146@bugs.debian.org, rubidium@openttd.org
Subject: Re: Bug#641146: openttd: Swapped colors on sparc64
Date: Fri, 4 Jan 2013 18:43:09 +0100
Hi Bertrand,

I did a bit more digging into the palette code of SDL, trying to figure
out what is happening and what we could be expecting from this. What
follows is a fairly detailed description of what I've learned, which is
probably not so interesting for you, but I wanted to write it down for
future reference. You might want to skip ahead to the conclusion. at the
end, where there's also a few questions for you.


Pretty much all of the discussion below is platform-independent and has
been tested both on sparc64 (with 32bit userland) and amd64. On sparc64,
I've only tested with X running in 8bit mode since my hardware can't do
anything else, on amd64 I've run in 32bit and 8bit modes (the latter
needed the vesa driver to work, since apparently 8bit mode is broken in
the modern intel drivers...).


When a screen surface is created using SDL_SetVideoMode, there are a
number of different palettes that can be created.
 - When the X11 visual is PseudoColor, and SDL_HWPALETTE is set, SDL
   allocates aprivate, changeable colormap (using XCreateColormap with AllocAll
   passed).

   This means that SDL has complete control over all the color values in
   the colormap and they can always be set to whatever value the
   application likes.

   The colormap allocated like this, is a virtual colormap. This means
   that even though we have complete control over our colormap, the
   colormap might not always be applied (since most hardware can only
   apply a single pixel -> RGB colormap translation over the entire
   display, so if we control the entire colormap, there are no color
   cells left for other applications to use).

   According to [1], the window manager controls what application's
   colormap is installed at what time. I have found that on my Ultra1,
   the colormap is installed (that is, the colors become correcnt) when
   switching to fullscreen.

[1]: http://menehune.opt.wfu.edu/Kokua/Irix_6.5.21_doc_cd/usr/share/Insight/library/SGI_bookshelves/SGI_Developer/books/XLib_PG/sgi_html/ch07.html#S1-1002-7-10

   Note that this needs to be SDL fullscreen, switching to fullscreen
   using my window manager (awesome, meta+f) doesn't work. Looking at
   the SDL code, it seems when switching to fullscreen in
   X11_EnterFullScreen, it calls XInstallColormap to install the virtual
   color map, causing it to become active. However, when switching out
   of full screen, the colormap is _not_ uninstalled, causing the
   palette to remain correctly (for the SDL app, they are messed up for
   all other apps) until the SDL app quits or another SDL app starts in
   fullscreen.

   This behaviour does seem very window-manager specific. It seems
   awesome doesn't attempt to manage the installed colormap at all (so
   it looks broken until you switch to fullscreen, then it remains
   correct when switch back to windowed mode). xfwm4 seems to handle
   colormaps more sanely: the colormap of whatever window is
   focused is installed. This means that when starting OpenTTD under
   xfwm4, it is displayed correctly out of the box (but everything else,
   including the window decorations on the OpenTTD window itself, are
   messed up of course).

   Note that this virtual colormap thing (probably) does not apply when
   running an 8-bit window on 24/32 bit X. In this case, the display can
   actually display all colors, so only a translation from pixel to real
   rgb values needs to happen when rendering to the screen. I'm not
   entirely sure how this happens, since this is handled inside X, which I
   didn't investigate.

 - For PseudoColor without SDL_HWPALLETE, SDL uses the default X11
   colormap. This means that a number of colors are already allocated by
   other applications, so there is only a limited amount of cells left.
   When allocating color cells, the application can only request
   specific colors and X11 will decide which pixel value to allocate to
   that color (or even an approximation of that color, if a similar
   color is already available, to save color cells).

   This means that when setting the pallete, you request a number of
   colors, but should check the palette after the request to see which
   colors actually ended up where.

   This mode is not very useful for OpenTTD, since you don't have
   complete control over the allocated colors (so no palette animation),
   there are not as many colors as OpenTTD needs and we would need to
   build our own OpenTTD pixel value -> allocated color pixel value
   mapping. However, if implemented properly, this mode could be used to make
   OpenTTD work in windowed mode on 8-bit systems looking at least
   somewhat like the real deal.

   However, further code review shows that when you create a 32bit
   screen surface (e.g., SDL_SetVideoMode with a depth of 32) on an 8
   bit X, SDL will create an 8bit screen surface and a 32bit shadow
   surface, and then do all this complicated mapping stuff when
   rendering the shadow surface to the screen surface (see
   SDL_MapSurface, MapNto1, SDL_CalculateBlit, SDL_CalculateBlitN,
   Blit_RGB888_index8 and BlitNto1). This is exactly what happens when
   you run openttd with a 32bpp blitter on an 8bit system: It works and
   gives very decent results, without messing up the colours of other
   systems.

   Note that this 32bpp blitting is a bit more complicated than
   necessary, since everytime the image is blitted to the screen
   surface, each RGB pixel is again translated into a 8-bit "normal
   form", which is then looked up in a mapping. In theory, it should be
   possible to achieve the same thing with the 8bit blitters, using an
   8bit shadow surface, where each 8-bit pixel value in the OpenTTD
   palette is mapped to the closed colour in the system palette instead.
   However, it seems there's no real way to force a shadow surface in
   SDL, it's only used when needed in very specific situations
   (different depths, most notably). We could manually emulate a shadow
   surface, not sure how efficient this is.

 - For DirectColor, SDL always allocates a changeable colormap (again
   using XCreateColormap with AllocAll), regardless of SDL_HWPALLETTE.
   Here, it also does fany things with a gamma ramp which I don't
   understand (but look like they set up a grayscale palette, not sure).

 - For all other visual types, SDL allocates an empty private colormap
   (using XCreateColormap with AllocNone). I'm not so sure how this
   colormap is supposed to be used (since SDL_HWPALETTE will be disabled
   in this mode, colors will be allocated on a best-effort basis like
   with PseudoColor without SDL_HWPALETTE above).

Note that SDL_HWPALETTE is always automatically enabled when the X11
visual for the SDL window is PseudoColor and is not the same as the
default visual (e.g., when using an 8-bit window on a 24-bit X).

Furthermore, when SDL window has > 8 bpp or the window visual is not
PseudoColor, SDL_HWPALETTE is always disabled on the resulting surface.

There is also the distinction between a surface's logical and physical
palettes, but only a screen surface can have a physical palette, and
only a screen surface with SDL_HWPALETTE can have different physical and
logical palettes. I suspect that this different logical palette would
then only be used when looking up colors on the surface (e.g., when
blitting a 32bit surface to a 8bit surface, the logical palette is
used), while using a different palette for displaying (the physical
palette). I don't think any of this stuff is useful for OpenTTD, though.

Making screenshots with xwd is also interesting. Using a
simple sdl test program, when SDL_HWPALETTE is on the screenshot is
perfect, even when the display (in windowed mode) is not. I presume this
is because xwd takes into account the window's virtual colormap when
making the screenshot, even when that colormap is not installed in the
current display.

When making the screenshot without SDL_HWPALETTE, the screenshot shows
the same (broken) image as is shown on display. This is presumably
because the colormap for the window is the default one which is also
installed in the screen. See correct.png attached.

Also note that in this mode, only part of the colours in the palette my
test app tries to set are allocated (in particular, the first 64 color
cells seem fixed, and the next 10-20 as well, depending on which
applications are running). Furthermore, the remaining color cells can be
allocated, but the pixel -> rgb mapping is not as requested, since X
starts allocating cells from cell 80 (or something) and onwards. See
wrong.png attached

As a final confirmation that this works like this: If you start the test
program when no terminals are running, and then start a shell, the
colors in ls will be wrong (directories are white, because there is no
color cell available for the particular blue normally used).

During testing, I have not observed any differences between using
SDL_SWSURFACE and SDL_HWSURFACE (though I have not tested this
extensively).


As for the 32bpp blitters, it seems that these are currently broken on
8bit X (both on sparc and on amd64). However, when removing the
SDL_HWPALETTE, the 32bpp bliters magically start working, letting SDL
do all kinds of mapping complexities under the hood. Because in this
mode, SDL tries to approximate every colour in the rendered graphics
with the closest color available from a shared palette, OpenTTD can be
displayed fairly well without messing up the colors of other
applications.


In summary:

 - The 8 bit paletted rendering seems to work fine for me, both on amd64
   and sparc. Depending on the window manager, the colours are not
   always properly shown in windowed mode (xfwm4 behaves properly), but
   they are always correct in full screen mode.

   Bertrand, I suspect that this might mean that I'm not observing the
   bug you are seeing after all. Does your bug also occur in fullscreen?
   Could you perhaps make a picture of the incorrect graphics (using a
   camera if screenshots do not capture it right)? Also, could you
   attach the output of "xpdyinfo" on your sparc?

 - We could perhaps change OpenTTD to approximate its colours using the
   system palette instead of forcing its own palette, when using an 8bit
   blitter on an 8bit X. This would result in 8bit rendering without
   messing up other application's colors, at the expense of extra
   performance compared to an exclusive 8bit palette (especially since
   animation needs to happen manually). I'm testing some patches for
   that rignt now and I'll discuss this option with upstream.

 - The 32bit rendering is broken on 8bit X, but this is trivial to fix
   in OpenTTD. I'll see about getting this fix into upstream.


Gr.

Matthijs




Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#641146; Package openttd. (Fri, 04 Jan 2013 18:39:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Matthijs Kooijman <matthijs@stdin.nl>:
Extra info received and forwarded to list. (Fri, 04 Jan 2013 18:39:03 GMT) Full text and rfc822 format available.

Message #35 received at 641146@bugs.debian.org (full text, mbox):

From: Matthijs Kooijman <matthijs@stdin.nl>
To: BERTRAND Joel <joel.bertrand@systella.fr>
Cc: 641146@bugs.debian.org, rubidium@openttd.org
Subject: Re: Bug#641146: openttd: Swapped colors on sparc64
Date: Fri, 4 Jan 2013 19:34:03 +0100
[Message part 1 (text/plain, inline)]
And here are the attachments I promised in the previous mail...
[correct.png (image/png, attachment)]
[wrong.png (image/png, attachment)]

Information forwarded to debian-bugs-dist@lists.debian.org:
Bug#641146; Package openttd. (Thu, 21 Mar 2013 14:33:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Matthijs Kooijman <matthijs@stdin.nl>:
Extra info received and forwarded to list. (Thu, 21 Mar 2013 14:33:04 GMT) Full text and rfc822 format available.

Message #40 received at 641146@bugs.debian.org (full text, mbox):

From: Matthijs Kooijman <matthijs@stdin.nl>
To: BERTRAND Joel <joel.bertrand@systella.fr>
Cc: 641146@bugs.debian.org
Subject: Re: Bug#641146: openttd: Swapped colors on sparc64
Date: Thu, 21 Mar 2013 15:11:00 +0100
Hi Joël,

a while ago you reported a color problem with OpenTTD on your sparc64.
Some palette-related fixes that made things work on my sparc64 with 8bit
video were included in the 1.3.0 OpenTTD release, which was recently
uploaded to experimental.

Could you test the OpenTTD version in experimental and report if it
fixes the problems you were seeing?

Thanks,

Matthijs



Information forwarded to debian-bugs-dist@lists.debian.org, Matthijs Kooijman <matthijs@stdin.nl>:
Bug#641146; Package openttd. (Sat, 23 Mar 2013 22:27:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to BERTRAND Joël <joel.bertrand@systella.fr>:
Extra info received and forwarded to list. Copy sent to Matthijs Kooijman <matthijs@stdin.nl>. (Sat, 23 Mar 2013 22:27:04 GMT) Full text and rfc822 format available.

Message #45 received at 641146@bugs.debian.org (full text, mbox):

From: BERTRAND Joël <joel.bertrand@systella.fr>
To: Matthijs Kooijman <matthijs@stdin.nl>, 641146@bugs.debian.org
Subject: Re: Bug#641146: openttd: Swapped colors on sparc64
Date: Sat, 23 Mar 2013 23:10:27 +0100
Matthijs Kooijman a écrit :
> Hi Joël,
>
> a while ago you reported a color problem with OpenTTD on your sparc64.
> Some palette-related fixes that made things work on my sparc64 with 8bit
> video were included in the 1.3.0 OpenTTD release, which was recently
> uploaded to experimental.
>
> Could you test the OpenTTD version in experimental and report if it
> fixes the problems you were seeing?

	I will try as soon as possible. I have halted my sparc due to some 
kernel instabilities (3.2 is not stable on all sun4u I have).

	Regards,

	JKB



Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Sun Apr 20 08:38:47 2014; Machine Name: buxtehude.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.