Debian Bug report logs - #539409
ejabberd: ipv6 nameservers in resolv.conf breaks SRV lookups

Package: ejabberd; Maintainer for ejabberd is Konstantin Khomoutov <flatworm@users.sourceforge.net>; Source for ejabberd is src:ejabberd.

Reported by: Alexander Clouter <alex@digriz.org.uk>

Date: Thu, 30 Jul 2009 10:09:02 UTC

Severity: normal

Tags: ipv6, jessie, sid, squeeze, wheezy

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Torsten Werner <twerner@debian.org>:
Bug#539269; Package ejabberd. (Thu, 30 Jul 2009 10:09:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Alexander Clouter <alex@digriz.org.uk>:
New Bug report received and forwarded. Copy sent to Torsten Werner <twerner@debian.org>. (Thu, 30 Jul 2009 10:09:06 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Alexander Clouter <alex@digriz.org.uk>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: ejabberd: ipv6 nameservers in resolv.conf breaks SRV lookups
Date: Thu, 30 Jul 2009 10:33:32 +0100
Package: ejabberd
Version: 2.0.1-6+lenny1
Severity: normal

I recently upgraded our server hardware and started afresh with networking and the lot.  The 
previous box although IPv6 capable, it was pretty focused around just IPv4, this box I get a clean 
slate with.

My /etc/resolv.conf file used to look like:
----
domain wormnet.eu
search wormnet.eu
nameserver ::1
----

Migrating from our old ejabberd server to the same version on the new hardware we found most s2s 
connections, particularly the gBorg ones, were uncontactable.  We thought it could be DNS 
propagation issues, as we slightly borked that up originally, but noticed that the s2s connections 
to some of the minor Jabber servers (@jabber.earth.li for example) were working fine.

I cracked out the packet sniffer and saw TCP SYN packets going our to port 5269 but no replying 
SYN/ACK's.  Strange I thought, then after I while, once I looked at the DNS lookups being made too, 
I saw that no SRV records were being looked up and it was just making raw A record lookups to make 
the s2s connection.  Eeek.

The A record results for gmail.com are very different to the SRV records you get for 
_xmpp-server._tcp.gmail.com.  The 5269/tcp packets were going to the wrong place.  This explains 
why for the smaller Jabber installations it just worked as it so happened that the SRV records come 
back with identical results to the raw A record lookup.

As a result my resolv.conf file now looks like:
----
domain wormnet.eu
search wormnet.eu
nameserver 127.0.0.1
#nameserver ::1
----

This fixes the problem, but makes for an IPv6 bug in ejabberd (or possibly even erlang?).

Cheers

-- System Information:
Debian Release: 5.0.2
  APT prefers stable
  APT policy: (500, 'stable')
Architecture: sparc (sparc64)

Kernel: Linux 2.6.26-2-sparc64-smp (SMP w/1 CPU core)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages ejabberd depends on:
ii  adduser                3.110             add and remove users and groups
ii  debconf [debconf-2.0]  1.5.24            Debian configuration management sy
ii  erlang-base [erlang-ab 1:12.b.3-dfsg-4   Concurrent, real-time, distributed
ii  erlang-nox             1:12.b.3-dfsg-4   Concurrent, real-time, distributed
ii  libc6                  2.7-18            GNU C Library: Shared libraries
ii  libexpat1              2.0.1-4           XML parsing C library - runtime li
ii  libpam0g               1.0.1-5+lenny1    Pluggable Authentication Modules l
ii  libssl0.9.8            0.9.8g-15+lenny1  SSL shared libraries
ii  openssl                0.9.8g-15+lenny1  Secure Socket Layer (SSL) binary a
ii  ucf                    3.0016            Update Configuration File: preserv
ii  zlib1g                 1:1.2.3.3.dfsg-12 compression library - runtime

ejabberd recommends no packages.

Versions of packages ejabberd suggests:
pn  libunix-syslog-perl           <none>     (no description available)

-- debconf information:
  ejabberd/user:
  ejabberd/hostname: localhost
  ejabberd/nomatch:




Information forwarded to debian-bugs-dist@lists.debian.org, Torsten Werner <twerner@debian.org>:
Bug#539269; Package ejabberd. (Thu, 30 Jul 2009 17:18:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Konstantin Khomoutov <flatworm@users.sourceforge.net>:
Extra info received and forwarded to list. Copy sent to Torsten Werner <twerner@debian.org>. (Thu, 30 Jul 2009 17:18:02 GMT) Full text and rfc822 format available.

Message #10 received at 539269@bugs.debian.org (full text, mbox):

From: Konstantin Khomoutov <flatworm@users.sourceforge.net>
To: Alexander Clouter <alex@digriz.org.uk>, 539269@bugs.debian.org
Subject: Re: Bug#539269: ejabberd: ipv6 nameservers in resolv.conf breaks SRV lookups
Date: Thu, 30 Jul 2009 21:16:43 +0400
Alexander Clouter wrote:

[...]
> My /etc/resolv.conf file used to look like:
> ----
> domain wormnet.eu
> search wormnet.eu
> nameserver ::1
> ----
[...]
> As a result my resolv.conf file now looks like:
> ----
> domain wormnet.eu
> search wormnet.eu
> nameserver 127.0.0.1
> #nameserver ::1
> ----
> This fixes the problem, but makes for an IPv6 bug in ejabberd (or possibly even erlang?).

Could you please do the following tests for me?

1) Keeping the current setup, run

# ejabberdctl debug

This will attach an Erlang shell to the live ejabberd instance.

2) At the Erlang shell prompt, execute the following two commands 
(commands are executed when you hit return, and also notice the trailing 
dots):

inet:get_rc().

and

inet_res:getbyname("_xmpp-server._tcp.jabber.ru", srv).

(change the jabber.ru server to whatever server you had troubles with, 
if you wish so).

Save the results, quit the shell by executing q(). or by entering Ctrl-g
followed by entering q and hitting return.

3) Revert the "nameserver" setting back to "::1", restart ejabberd and 
repeat the step (2).

4) Change everything back to the working state and post the output of 
those commands.

At least this could give a clue on how the Erlang DNS resolver is 
configured for you in both cases, and whether is is able to correctly 
parse the IPv6 address from the system resolver config.




Information forwarded to debian-bugs-dist@lists.debian.org, Torsten Werner <twerner@debian.org>:
Bug#539269; Package ejabberd. (Thu, 30 Jul 2009 17:42:07 GMT) Full text and rfc822 format available.

Acknowledgement sent to Konstantin Khomoutov <flatworm@users.sourceforge.net>:
Extra info received and forwarded to list. Copy sent to Torsten Werner <twerner@debian.org>. (Thu, 30 Jul 2009 17:42:07 GMT) Full text and rfc822 format available.

Message #15 received at 539269@bugs.debian.org (full text, mbox):

From: Konstantin Khomoutov <flatworm@users.sourceforge.net>
To: Alexander Clouter <alex@digriz.org.uk>, 539269@bugs.debian.org
Subject: Re: Bug#539269: ejabberd: ipv6 nameservers in resolv.conf breaks SRV lookups
Date: Thu, 30 Jul 2009 21:40:51 +0400
A quick followup to my previous post.

First, it's not needed to mess with ejabberd to see what Erlang resolver 
configuration it sees -- it suffices to start the standalone Erlang 
shell with the right parameters.

Second, this post [1] suggests to tweak the Erlang resolver config -- 
worth trying out.

How to do this.

1) As root, copy /etc/ejabberd/inetrc someplace (say, /tmp) and chown it 
to the user you usually log in with.

2) Assumed the resulting file is /tmp/inetrc, and you have the 
permission to read it, start the Erlang shell like this:

$ erl -kernel inetrc '"/tmp/inetrc"'

(notice the usage of quotes -- the Erlang itself must see "" around the 
file name, so you have to protect them from the Unix shell).

3) At the Erlang shell's prompt, enter the commands mentioned in my 
previous post:

inet:get_rc().

and

inet_res:getbyname("_xmpp-server._tcp.jabber.ru", srv).

4) Quit the Erlang shell, change the "nameserver" setting in 
/etc/resolv.conf to "::1", restart the shell as shown above and retry 
the commands.

5) Repeat steps (3) and (4) with

{inet6, true}.

added to the /tmp/inetrc so that it contents look like

{file, resolv, "/etc/resolv.conf"}.
{inet6, true}.

This setting is explained in [2].

In particular, it's interesting to know:
* Whether the resolving works for you on step (5).
* Whether the {inet6, true}. setting makes any difference.

1. http://erlang.org/pipermail/erlang-questions/2008-April/034230.html
2. http://erlang.org/doc/apps/erts/inet_cfg.html





Information forwarded to debian-bugs-dist@lists.debian.org, Torsten Werner <twerner@debian.org>:
Bug#539269; Package ejabberd. (Thu, 30 Jul 2009 19:42:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Alexander Clouter <alex@digriz.org.uk>:
Extra info received and forwarded to list. Copy sent to Torsten Werner <twerner@debian.org>. (Thu, 30 Jul 2009 19:42:03 GMT) Full text and rfc822 format available.

Message #20 received at 539269@bugs.debian.org (full text, mbox):

From: Alexander Clouter <alex@digriz.org.uk>
To: Konstantin Khomoutov <flatworm@users.sourceforge.net>
Cc: 539269@bugs.debian.org
Subject: Re: Bug#539269: ejabberd: ipv6 nameservers in resolv.conf breaks SRV lookups
Date: Thu, 30 Jul 2009 20:35:18 +0100
Hi,

* Konstantin Khomoutov <flatworm@users.sourceforge.net> [2009-07-30 21:40:51+0400]:
> 
> A quick followup to my previous post.
> 
> First, it's not needed to mess with ejabberd to see what Erlang resolver  
> configuration it sees -- it suffices to start the standalone Erlang shell 
> with the right parameters.
> 
> Second, this post [1] suggests to tweak the Erlang resolver config --  
> worth trying out.
> 
> How to do this.
> 
> [snipped]
>
This is what I ran:
----
alex@chipmunk:~$ cat /tmp/run 
inet:get_rc().
inet_res:getbyname("_xmpp-server._tcp.gmail.com", srv).
----

It seems that "nameserver ::1" does not work at all, probably an erlang 
bug instead?

-- nameserver 127.0.0.1 + !inet6 --
alex@chipmunk:~$ erl -kernel inetrc '"/tmp/inetrc"' < /tmp/run
Eshell V5.6.3  (abort with ^G)
1> [{domain,"wormnet.eu"},
 {nameserver,{127,0,0,1}},
 {search,["wormnet.eu"]},
 {lookup,[native]}]
2> {ok,{hostent,"_xmpp-server._tcp.gmail.com",[],srv,5,
             [{20,0,5269,"xmpp-server2.l.google.com"},
              {20,0,5269,"xmpp-server3.l.google.com"},
              {20,0,5269,"xmpp-server1.l.google.com"},
              {5,0,5269,"xmpp-server.l.google.com"},
              {20,0,5269,"xmpp-server4.l.google.com"}]}}
3> *** Terminating erlang (nonode@nohost)
----------------------------------

-- nameserver ::1 + !inet6 --
alex@chipmunk:~$ erl -kernel inetrc '"/tmp/inetrc"' < /tmp/run
Eshell V5.6.3  (abort with ^G)
1> [{domain,"wormnet.eu"},
 {search,["wormnet.eu"]},
 {lookup,[native]}]
2> {error,timeout}
3> *** Terminating erlang (nonode@nohost)
----------------------------------

-- nameserver 127.0.0.1 + inet6 --
alex@chipmunk:~$ erl -kernel inetrc '"/tmp/inetrc"' < /tmp/run
Eshell V5.6.3  (abort with ^G)
1> [{domain,"wormnet.eu"},
 {nameserver,{127,0,0,1}},
 {search,["wormnet.eu"]},
 {inet6,true},
 {lookup,[native]}]
2> {ok,{hostent,"_xmpp-server._tcp.gmail.com",[],srv,5,
             [{20,0,5269,"xmpp-server4.l.google.com"},
              {20,0,5269,"xmpp-server1.l.google.com"},
              {20,0,5269,"xmpp-server2.l.google.com"},
              {5,0,5269,"xmpp-server.l.google.com"},
              {20,0,5269,"xmpp-server3.l.google.com"}]}}
3> *** Terminating erlang (nonode@nohost)
----------------------------------

-- nameserver ::1 + inet6 --
alex@chipmunk:~$ erl -kernel inetrc '"/tmp/inetrc"' < /tmp/run
Eshell V5.6.3  (abort with ^G)
1> [{domain,"wormnet.eu"},
 {search,["wormnet.eu"]},
 {inet6,true},
 {lookup,[native]}]
2> {error,timeout}
3> *** Terminating erlang (nonode@nohost)
----------------------------------

In the next posting[1] in the 'inet6' thread I see:
----
inet_db:add_ns({A,B,C,D}).  %% Name server must be IPv4, there is a fixme in inet_db.
----

Looking at inet_db.erl I see "Fix IPv6 nameservers" all over the place, 
so it's a bug in upstream erlang it seems?

It is interesting that an A record lookup is made by ejabberd though in 
this situation, is there some fallback code being involked?  Should it 
be?

Cheers

[1] http://erlang.org/pipermail/erlang-questions/2008-April/034254.html

-- 
Alexander Clouter
.sigmonster says: Tuesday is the Wednesday of the rest of your life.




Information forwarded to debian-bugs-dist@lists.debian.org, Torsten Werner <twerner@debian.org>:
Bug#539269; Package ejabberd. (Fri, 31 Jul 2009 12:00:09 GMT) Full text and rfc822 format available.

Acknowledgement sent to Konstantin Khomoutov <flatworm@users.sourceforge.net>:
Extra info received and forwarded to list. Copy sent to Torsten Werner <twerner@debian.org>. (Fri, 31 Jul 2009 12:00:09 GMT) Full text and rfc822 format available.

Message #25 received at 539269@bugs.debian.org (full text, mbox):

From: Konstantin Khomoutov <flatworm@users.sourceforge.net>
To: Alexander Clouter <alex@digriz.org.uk>, 539269@bugs.debian.org
Subject: Re: Bug#539269: ejabberd: ipv6 nameservers in resolv.conf breaks SRV lookups
Date: Fri, 31 Jul 2009 15:59:28 +0400
Alexander Clouter wrote:

> -- nameserver 127.0.0.1 + !inet6 --
> 1> [{domain,"wormnet.eu"},
>  {nameserver,{127,0,0,1}},
>  {search,["wormnet.eu"]},
>  {lookup,[native]}]
[...]
> -- nameserver ::1 + !inet6 --
> 1> [{domain,"wormnet.eu"},
>  {search,["wormnet.eu"]},
>  {lookup,[native]}]
[...]
> -- nameserver 127.0.0.1 + inet6 --
> 1> [{domain,"wormnet.eu"},
>  {nameserver,{127,0,0,1}},
>  {search,["wormnet.eu"]},
>  {inet6,true},
>  {lookup,[native]}]
[...]
> -- nameserver ::1 + inet6 --
> 1> [{domain,"wormnet.eu"},
>  {search,["wormnet.eu"]},
>  {inet6,true},
>  {lookup,[native]}]

I think the pattern here is that whenever you have "nameserver" set to 
"::1", the Erlang inet code fails to parse it from /etc/resolv.conf, and 
the Erlang DNS reslover ends up being left without a usable nameserver.

> In the next posting[1] in the 'inet6' thread I see:
> ----
> inet_db:add_ns({A,B,C,D}).  %% Name server must be IPv4, there is a fixme in inet_db.
> ----
> Looking at inet_db.erl I see "Fix IPv6 nameservers" all over the place, 
> so it's a bug in upstream erlang it seems?
Yes, it seems like a bug upstream -- while the inets module itself seems 
not to have any problems using ipv6 addresses, the DNS code seems
to lag behind the rest in this respect.

Now we have to figure out whether it is fixed in Erlang R13B which is 
already in Squeeze.

> It is interesting that an A record lookup is made by ejabberd though in 
> this situation, is there some fallback code being involked?  Should it 
> be?
Yes and no: the fallback to the A-record resolution is specified in the 
spec (see the fourth paragraph of [1]), but the spec seems to only talk 
about the case of a failed SRV query, in the sense of the query returned 
the NXDOMAIN (or a similar) error, as I understand this.
So what's the correct behaviour in the presense of a misconfigured DNS 
resolver is not clear to me.

1. http://xmpp.org/rfcs/rfc3920.html#rfc.section.14.4





Information forwarded to debian-bugs-dist@lists.debian.org, Torsten Werner <twerner@debian.org>:
Bug#539269; Package ejabberd. (Fri, 31 Jul 2009 15:27:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Konstantin Khomoutov <flatworm@users.sourceforge.net>:
Extra info received and forwarded to list. Copy sent to Torsten Werner <twerner@debian.org>. (Fri, 31 Jul 2009 15:27:05 GMT) Full text and rfc822 format available.

Message #30 received at 539269@bugs.debian.org (full text, mbox):

From: Konstantin Khomoutov <flatworm@users.sourceforge.net>
To: control@bugs.debian.org, 539269@bugs.debian.org
Subject: Reassigning #539269 to erlang and retitling
Date: Fri, 31 Jul 2009 19:25:36 +0400
retitle 539269 Erlang DNS resolver cannot use IPv6 DNS servers to resolve SRV records
reassign 539269 erlang
thanks

Sergei Golovan verified the bug still exists in Erlang/OTP R13B,
so reassigning it.

It seems that the bug is about differences in how simple (A) lookups
and complex (SRV) lookups are performed.
If I understand correctly, it is done by different bits of the code:
simple lookups seem to use the system resolver (and so ejabberd
was able to successfully communicate with the servers whose XMPP
domains resolved to actual server IPs, and which used the standard
s2s port for communication), but complex lookups (SRV in this case)
are probably performed by directly sending/receiving data, which
requires knowledge of the address of at least one nameserver to work
properly, and that nameserver is not parsed correctly from
/etc/resolv.conf.





Changed Bug title to 'Erlang DNS resolver cannot use IPv6 DNS servers to resolve SRV records' from 'ejabberd: ipv6 nameservers in resolv.conf breaks SRV lookups' Request was from Konstantin Khomoutov <flatworm@users.sourceforge.net> to control@bugs.debian.org. (Fri, 31 Jul 2009 15:27:06 GMT) Full text and rfc822 format available.

Bug reassigned from package 'ejabberd' to 'erlang'. Request was from Konstantin Khomoutov <flatworm@users.sourceforge.net> to control@bugs.debian.org. (Fri, 31 Jul 2009 15:27:07 GMT) Full text and rfc822 format available.

Bug No longer marked as found in versions ejabberd/2.0.1-6+lenny1. Request was from Konstantin Khomoutov <flatworm@users.sourceforge.net> to control@bugs.debian.org. (Fri, 31 Jul 2009 15:27:08 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Erlang Packagers <erlang-pkg-devel@lists.berlios.de>:
Bug#539269; Package erlang. (Fri, 31 Jul 2009 15:39:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Konstantin Khomoutov <flatworm@users.sourceforge.net>:
Extra info received and forwarded to list. Copy sent to Erlang Packagers <erlang-pkg-devel@lists.berlios.de>. (Fri, 31 Jul 2009 15:39:02 GMT) Full text and rfc822 format available.

Message #41 received at 539269@bugs.debian.org (full text, mbox):

From: Konstantin Khomoutov <flatworm@users.sourceforge.net>
To: Alexander Clouter <alex@digriz.org.uk>, 539269@bugs.debian.org
Subject: Re: Bug#539269: ejabberd: ipv6 nameservers in resolv.conf breaks SRV lookups
Date: Fri, 31 Jul 2009 19:34:03 +0400
Alexander Clouter wrote:
> Package: ejabberd
> Version: 2.0.1-6+lenny1
> Severity: normal
> 
> I recently upgraded our server hardware and started afresh with networking and the lot.  The 
> previous box although IPv6 capable, it was pretty focused around just IPv4, this box I get a clean 
> slate with.
> 
> My /etc/resolv.conf file used to look like:
> ----
> domain wormnet.eu
> search wormnet.eu
> nameserver ::1
> ----

By the way, I read through man resolv.conf and noticed that the 
"nameserver" option is optional; if it is missing, the nameserver is 
assumed to be on localhost.

With this in mind, and provided your nameserver is configured to listen 
both on 127.0.0.1 and ::1, may be it worth trying this setup out?




Bug 539269 cloned as bug 539409. Request was from Konstantin Khomoutov <flatworm@users.sourceforge.net> to control@bugs.debian.org. (Fri, 31 Jul 2009 15:45:08 GMT) Full text and rfc822 format available.

Bug reassigned from package 'erlang' to 'ejabberd'. Request was from Konstantin Khomoutov <flatworm@users.sourceforge.net> to control@bugs.debian.org. (Fri, 31 Jul 2009 15:45:11 GMT) Full text and rfc822 format available.

Changed Bug title to 'ejabberd: ipv6 nameservers in resolv.conf breaks SRV lookups' from 'Erlang DNS resolver cannot use IPv6 DNS servers to resolve SRV records' Request was from Konstantin Khomoutov <flatworm@users.sourceforge.net> to control@bugs.debian.org. (Fri, 31 Jul 2009 15:45:11 GMT) Full text and rfc822 format available.

Added blocking bug(s) 539269 Request was from Konstantin Khomoutov <flatworm@users.sourceforge.net> to control@bugs.debian.org. (Fri, 31 Jul 2009 15:45:12 GMT) Full text and rfc822 format available.

Added tag(s) ipv6. Request was from Konstantin Khomoutov <flatworm@users.sourceforge.net> to control@bugs.debian.org. (Sun, 28 Feb 2010 21:30:06 GMT) Full text and rfc822 format available.

Severity set to 'serious' from 'normal' Request was from Clint Adams <schizo@debian.org> to control@bugs.debian.org. (Tue, 23 Mar 2010 01:03:26 GMT) Full text and rfc822 format available.

Added tag(s) sid and squeeze. Request was from Gerfried Fuchs <rhonda@debian.at> to control@bugs.debian.org. (Tue, 23 Mar 2010 07:51:19 GMT) Full text and rfc822 format available.

Severity set to 'normal' from 'serious' Request was from Gerfried Fuchs <rhonda@deb.at> to control@bugs.debian.org. (Tue, 23 Mar 2010 08:30:30 GMT) Full text and rfc822 format available.

Added tag(s) wheezy. Request was from Kurt Roeckx <kurt@roeckx.be> to control@bugs.debian.org. (Wed, 16 Feb 2011 19:03:27 GMT) Full text and rfc822 format available.

Added tag(s) jessie. Request was from Julien Cristau <jcristau@debian.org> to control@bugs.debian.org. (Thu, 18 Apr 2013 17:42:36 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Wed Apr 16 20:12:23 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.