Debian Bug report logs - #375787
sendmail-base: bad dialup.m4

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: David Fries <david@fries.net>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: sendmail-base: bad dialup.m4

Date: Tue, 27 Jun 2006 22:19:55 -0500

Package: sendmail-base
Version: 8.13.7-2
Severity: important


For the last four or five days I've been tracking down a problem.  I
could receive e-mail, but only some of the e-mail would go out.  I
also saw the following,

telnet ::1 25
Trying ::1...
Connected to ::1.
Escape character is '^]'.
220 ;; ESMTP connection timed out; no servers could be reached Sendmail 8.13.7/8.13.7/Debian-2; Mon, 26 Jun 2006 21:55:05 -0500; (No UCE/UBE) logging access from: ip6-localhost(OK)-ip6-localhost [IPv6:::1]

I finally tracked it down to /etc/mail/m4/dialup.m4
(ip replaced with <!-- ip --> only one change)

LOCAL_CONFIG
#------------------------------------------------------------
#
# Dynamic host/ip updates from /etc/network/if-up.d/sendmail
#
# NOTE: the following line *MUST* be in /etc/mail/sendmail.mc
dnl include(`/etc/mail/dialup.m4')dnl
# This should *NOT* be included in submit.mc !
#
# Make sure we accept mail as this ip (for bounces, etc)
Cw<!-- ip -->
#
# Define our true hostname (from our ISP) - becomes $j
define(`confDOMAIN_NAME', `;; connection timed out; no servers could be reached')dnl
#
# Make sure we accept mail as this name (for bounces, etc)
Cw;; connection timed out; no servers could be reached
#
# Add our hostname to class G for genericstable support
CG;; connection timed out; no servers could be reached
#------------------------------------------------------------


I was watching for failed DNS queries, using strace and ltrace to see
what sendmail was doing starting up, and finally figured out the
above.  This is a server and doesn't have dynamically changing ip
addresses, so I've disabled dialup.m4 in the sendmail configuration
file.  Maybe in the future the script could avoid setting
confDOMAIN_NAME if the DNS query failed.


-- System Information:
Debian Release: 3.0
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'testing')
Architecture: i386 (i586)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.4.26
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)

Versions of packages sendmail-base depends on:
ii  adduser                       3.45       Add and remove users and groups
ii  m4                            1.4-15     a macro processing language
ii  make                          3.79.1-15  The GNU version of the "make" util
ii  netbase                       4.18       Basic TCP/IP networking system
ii  perl                          5.8.7-4    Larry Wall's Practical Extraction 

Versions of packages sendmail-base recommends:
pn  resolvconf                    <none>     (no description available)

Versions of packages sensible-mda depends on:
ii  libc6                         2.3.6-15   GNU C Library: Shared libraries
ii  procmail                      3.22-2     Versatile e-mail processor.
ii  sendmail-bin [mail-transport- 8.13.7-2   powerful, efficient, and scalable 

Versions of packages rmail depends on:
ii  libc6                       2.3.6-15     GNU C Library: Shared libraries
ii  libldap2                    2.1.30-13+b1 OpenLDAP libraries
ii  sendmail-bin [mail-transpor 8.13.7-2     powerful, efficient, and scalable 

-- no debconf information

Message #10 received at 375787@bugs.debian.org (full text, mbox, reply):

From: Richard Kay <richard.kay@tic.ac.uk>

To: 375787@bugs.debian.org

Cc: cowboy@debian.org, david@fries.net, cbl@cbl.abuseat.org

Subject: Debian Sendmail bug 375787 recurring on Etch

Date: Thu, 27 Sep 2007 13:05:16 +0100

My /etc/mail/m4/dialup.m4 contained:

define(`confDOMAIN_NAME', `;; connection timed out; no servers could be
reached')dnl

This seems to end up within the HELO/EHLO greeting for sent messages,
breaking 
RFC2821 .

This resulted in my server ( 80.68.90.112 ) getting a cbl.abuseat.org
blacklisting which was picked up by zen.spamhaus.org causing my
outgoing mail to be blocked.

Changing the error message to my valid domain name: copsewood.net 
seems to have fixed the problem. However, every time Sendmail gets
upgraded I will have to reapply my local patch until this bug is fixed.
Isn't there a better message Sendmail can use as the default, or
wouldn't it work better for sendmail or the Debian sendmail installation
script to try to figure out the domain name, e.g. using the hostname
command ?

Message #15 received at 375787@bugs.debian.org (full text, mbox, reply):

From: Kees-Jan Dijkzeul <kees-jan.dijkzeul@iae.nl>

To: 375787@bugs.debian.org

Subject: I've analysed this some more...

Date: Thu, 18 Oct 2007 21:32:47 +0200

I've just recently had the same problem (sendmail 8.13.8-3 / Debian Etch)

The problem first occurred when I booted the box without the network
cable plugged in. The network interface got configured with a remembered
ip adress, but obviously, DNS is not available. Hence, the
/etc/network/if-up.d/sendmail script failed (?) to determine my
hostname, and messed up dialup.m4 by inserting a dns-error message

I expected the box to recover when rebooting with the network connected,
but it didn't. Manually calling /etc/network/if-up.d/sendmail as

/etc/network/if-up.d/sendmail 192.168.11.2 '' eth0

seems to solve the issue for me (dialup.m4 gets rewritten correctly, and
isn't messed up again by subsequent ifup/ifdowns

I did not yet investigate why /etc/network/if-up.d/sendmail feels the
need to update my hostname only when dns is absent, but not when it is
present.

Message #20 received at 375787@bugs.debian.org (full text, mbox, reply):

From: Richard A Nelson <cowboy@debian.org>

To: Kees-Jan Dijkzeul <kees-jan.dijkzeul@iae.nl>, David Fries <david@fries.net>, 375787@bugs.debian.org

Subject: Re: Bug#375787: I've analysed this some more...

Date: Thu, 18 Oct 2007 19:01:59 -0700 (PDT)

On Thu, 18 Oct 2007, Kees-Jan Dijkzeul wrote:

> The problem first occurred when I booted the box without the network
> cable plugged in. The network interface got configured with a remembered
> ip adress, but obviously, DNS is not available. Hence, the
> /etc/network/if-up.d/sendmail script failed (?) to determine my
> hostname, and messed up dialup.m4 by inserting a dns-error message

I can see that happening, and the other cause is slow DNS - though
there are a plethora of attempts to hopefully work around that case.

> I expected the box to recover when rebooting with the network connected,
> but it didn't. Manually calling /etc/network/if-up.d/sendmail as
>
> /etc/network/if-up.d/sendmail 192.168.11.2 '' eth0
>
> seems to solve the issue for me (dialup.m4 gets rewritten correctly, and
> isn't messed up again by subsequent ifup/ifdowns

Exactly - you told it what your new address was, and that the domain
(provider) has not changed.

> I did not yet investigate why /etc/network/if-up.d/sendmail feels the
> need to update my hostname only when dns is absent, but not when it is
> present.

Actually, updating has nothing to do with DNS availability !

provider.m4 and dialup.m4 are re-written in the following cases:

	*) Network goes down
	   This event causes /etc/mail/m4/dialup.m4 to updated

	*) Network comes up and one or more of the following has changed

		+) Network domain/provider (ISP) via PPP or provider tag in
		   /etc/network/interfaces, or DHCP lease information
		   This event causes /etc/mail/m4/provider.m4 to be updated

		+) The IP has changed
		   This event causes /etc/mail/m4/dialup.m4 to be updated

	  Note that both of these can happen at the same time

These changes are handled by one or more of /etc/ppp/ip-*/sendmail,
/etc/network/if-*/sendmail and /etc/dhcp3/dhclient-exit-hooks.d/sendmail

The real logic (or lack thereof) comes in two places:

	*) /etc/mail/sendmail.conf
	   Here, you tell sendmail if you have a static or dynamic IP,
	   and if dynamic, which interface(s) to monitor for changes.

	   If you have a static IP, please tell sendmail that, and update
	   the dialup.m4 by hand to reflect what the values should be - then
	   sendmail will keep its grubby (not so) little fingers out the
	   pie.

	*) /usr/share/sendmail/dynamic
	   Here it attempts to 'do the right thing' and not restart sendmail every
	   time your DHCP lease is renewed, etc...  It will only update information
	   it has a good chance of knowing changed.

The above really should be put into README.Debian

So, in your case, after you set the IP, sendmail has no way of knowing
when/if the resolvability of your host changes - and can't change anything.

This whole mess was concocted by me and a few others back when we were
forced to live on PPP/dialup or DHCP dsl/cable - and it worked just fine
for us, but at least I always ran a split horizon/view local DNS on my
mail servers - and thus never really ran into this issue.

I found a bug in the dynamic script that can cause early termination
of the host lookup loop - and explains the ';;....'  edit.

I'm going to fix that, but before I put it out, I would like your
opinions on how to handle the remaining issue:

/usr/share/sendmail/dynamic::update_host is called when the IP has
changed, and it calls find_host to do the resolution...  just blindly
assuming it has done the job properly.

If find_host works, there is no problem...

If it fails, however, there are some options - none good ;(

	*) Keep sendmail from starting in an unknown state, and invent
	   a means to do the needed DNS lookup during
	       /etc/init.d/sendmail start
	   This likely means actually keeping some state around - and
	   we could even retry later on DHCP RENEWs !

	*) Keep sendmail the 'logically offline state' - mail will
	   not be sent, I will have to investigate if it will accept
	   mail in this state or not (doing so could be *bad*)

	*) Use the prior hostname, which *may* be invalid - potentially
	   causing bounces, DNSBL additions, etc.

-- 
Rick Nelson
I stopped a long time ago to try to find anything in the bug list of dpkg.
We should run for an entry in the Guinness Book of Records.
		-- Stephane Bortzmeyer

Message #25 received at 375787@bugs.debian.org (full text, mbox, reply):

From: "Kees-Jan Dijkzeul" <kees-jan.dijkzeul@iae.nl>

To: "Richard A Nelson" <cowboy@debian.org>

Cc: "David Fries" <david@fries.net>, 375787@bugs.debian.org

Subject: Re: Bug#375787: I've analysed this some more...

Date: Fri, 19 Oct 2007 09:07:22 +0200 (CEST)

Richard A Nelson wrote:
[...]
>> /etc/network/if-up.d/sendmail 192.168.11.2 '' eth0
>>
>> seems to solve the issue for me (dialup.m4 gets rewritten correctly, and
>> isn't messed up again by subsequent ifup/ifdowns
>
> Exactly - you told it what your new address was, and that the domain
> (provider) has not changed.

This is not entirely accurate. During this entire exercise, my ip address
never changed. I get it from my broadband router that is configured to
always serve this ip-address to the mac-address of my box. The only thing
that ever changes is the ability to do reverse-DNS lookups of my IP
address.

>> I did not yet investigate why /etc/network/if-up.d/sendmail feels the
>> need to update my hostname only when dns is absent, but not when it is
>> present.
>
> Actually, updating has nothing to do with DNS availability !

Hmm...
You are contradicting my observations... This is where things get
interesting.

If I recall correctly (didn't re-test yet), the following happened:
1. Start with a perfectly working configuration.
2. Shut down
   - I didn't verify that dialup.m4 gets updated
3. Disconnect ethernet
4. Start up again
   - DHCP fails (obviously), so I'm given a remembered address,
     which is the same as the one I've always had (192.168.11.2)
   - dialup.m4 gets updated, even though my IP is still the same (?).
     Because DNS-resolvability has changed, dialup.m4 gets messed up.
5. Shut down again
   - I didn't verify that dialup.m4 gets updated
6. Reconnect ethernet
7. Start up yet again
   - DHCP succeeds again. I'm still getting the same ip as before.
   - For some reason dialup.m4 doesn't get updated.
8. End up with a broken configuration

So, in my opinion, the problem is that dialup.m4 should get updated either
both at step 4 and step 7, or not at all. Current observation is that it
only gets updated at step 4, and not at step 7.

> So, in your case, after you set the IP, sendmail has no way of knowing
> when/if the resolvability of your host changes - and can't change
> anything.

As expained above, I don't think this is the cause of my problem, as I am
doing reboots in between.

Still, if you depend on resolvability and don't get notifications when
resolvability changes, you have a design flaw.

> This whole mess was concocted by me and a few others back when we were
> forced to live on PPP/dialup or DHCP dsl/cable - and it worked just fine
> for us, but at least I always ran a split horizon/view local DNS on my
> mail servers - and thus never really ran into this issue.

[...]

> I found a bug in the dynamic script that can cause early termination
> of the host lookup loop - and explains the ';;....'  edit.
>
> I'm going to fix that, but before I put it out, I would like your
> opinions on how to handle the remaining issue:

I'm not really the right person to ask, since I don't think I fully
appreciate what you are trying to do in the first place.

I'm guessing you are trying to make the sendmail configuration robust
against hostname changes, basically allowing sendmail to accept mail for
whatever hostname it happens to find itself at.

I currently don't see the point of implementing such a feature, since I
would never actually be able to hand out such an e-mail address, because
it is liable to change at unexpected times. In any (AFAIK) useful
configuration, the domain name(s) sendmail receives mail at is fixed (such
that you can hand out your e-mail address), and the real challenge is to
make mail addressed to that (fixed) domain reach you.

Back in the PPP/dialup days, I was using UUCP (which also allowed me to go
offline every now and then). Today, I have cable and a dyndns account,
making sure that my hostname points to the correct ip address.

Anyway... Returning on topic. I think we are discussing two different
issues. As far as I can tell, this bug is about the case where
resolvability changes (but ip-address doesn't) at ifup/ifdown time. This
is currently not (always) handled correctly, and I don't think we know the
cause yet.

You are asking me what to do in a case where dynamic domain names are
required and you don't have resolvability. As you observe, there is no
'correct' way to handle this case. I'm not sure, though, that the
requirement still exists (or should have ever existed ;-)

Groetjes,

Kees-Jan

Message #30 received at 375787@bugs.debian.org (full text, mbox, reply):

From: David Fries <david@fries.net>

To: Richard A Nelson <cowboy@debian.org>

Cc: Kees-Jan Dijkzeul <kees-jan.dijkzeul@iae.nl>, 375787@bugs.debian.org

Subject: Re: Bug#375787: I've analysed this some more...

Date: Sat, 20 Oct 2007 12:48:06 -0500

On Thu, Oct 18, 2007 at 07:01:59PM -0700, Richard A Nelson wrote:
> On Thu, 18 Oct 2007, Kees-Jan Dijkzeul wrote:
> 
> This whole mess was concocted by me and a few others back when we were
> forced to live on PPP/dialup or DHCP dsl/cable - and it worked just fine
> for us, but at least I always ran a split horizon/view local DNS on my
> mail servers - and thus never really ran into this issue.

I manged to run for years without it being a problem.  I don't know
what happened that I got bitten that time either.  The system serves
DHCP and has static ip addresses.  My fault for having dialup.m4
referenced.

> I found a bug in the dynamic script that can cause early termination
> of the host lookup loop - and explains the ';;....'  edit.
> 
> I'm going to fix that, but before I put it out, I would like your
> opinions on how to handle the remaining issue:
> 
> /usr/share/sendmail/dynamic::update_host is called when the IP has
> changed, and it calls find_host to do the resolution...  just blindly
> assuming it has done the job properly.
> 
> If find_host works, there is no problem...
> 
> If it fails, however, there are some options - none good ;(
> 
> 	*) Keep sendmail from starting in an unknown state, and invent
> 	   a means to do the needed DNS lookup during
> 	       /etc/init.d/sendmail start
> 	   This likely means actually keeping some state around - and
> 	   we could even retry later on DHCP RENEWs !
> 
> 	*) Keep sendmail the 'logically offline state' - mail will
> 	   not be sent, I will have to investigate if it will accept
> 	   mail in this state or not (doing so could be *bad*)
> 
> 	*) Use the prior hostname, which *may* be invalid - potentially
> 	   causing bounces, DNSBL additions, etc.

I would say a modified last one.  Keep the current configuration and
sendmail running, but invalidate the peer and IP address cache, so
that they don't have to change for the script to try again.  Though I
was looking through the /usr/share/sendmail/dynamic script I have and
I don't see how it compares the current ip address to the last one.

> -- 
> Rick Nelson
> I stopped a long time ago to try to find anything in the bug list of dpkg.
> We should run for an entry in the Guinness Book of Records.
> 		-- Stephane Bortzmeyer

-- 
David Fries <david@fries.net>
http://fries.net/~david/ (PGP encryption key available)

Message #35 received at 375787-close@bugs.debian.org (full text, mbox, reply):

From: Richard A Nelson (Rick) <cowboy@debian.org>

To: 375787-close@bugs.debian.org

Subject: Bug#375787: fixed in sendmail 8.14.1-11

Date: Sun, 21 Oct 2007 02:47:03 +0000

Source: sendmail
Source-Version: 8.14.1-11

We believe that the bug you reported is fixed in the latest version of
sendmail, which is due to be installed in the Debian FTP archive:

libmilter-dev_8.14.1-11_amd64.deb
  to pool/main/s/sendmail/libmilter-dev_8.14.1-11_amd64.deb
libmilter1-dbg_8.14.1-11_amd64.deb
  to pool/main/s/sendmail/libmilter1-dbg_8.14.1-11_amd64.deb
libmilter1_8.14.1-11_amd64.deb
  to pool/main/s/sendmail/libmilter1_8.14.1-11_amd64.deb
rmail_8.14.1-11_amd64.deb
  to pool/main/s/sendmail/rmail_8.14.1-11_amd64.deb
sendmail-base_8.14.1-11_all.deb
  to pool/main/s/sendmail/sendmail-base_8.14.1-11_all.deb
sendmail-bin_8.14.1-11_amd64.deb
  to pool/main/s/sendmail/sendmail-bin_8.14.1-11_amd64.deb
sendmail-cf_8.14.1-11_all.deb
  to pool/main/s/sendmail/sendmail-cf_8.14.1-11_all.deb
sendmail-doc_8.14.1-11_all.deb
  to pool/main/s/sendmail/sendmail-doc_8.14.1-11_all.deb
sendmail_8.14.1-11.diff.gz
  to pool/main/s/sendmail/sendmail_8.14.1-11.diff.gz
sendmail_8.14.1-11.dsc
  to pool/main/s/sendmail/sendmail_8.14.1-11.dsc
sendmail_8.14.1-11_all.deb
  to pool/main/s/sendmail/sendmail_8.14.1-11_all.deb
sensible-mda_8.14.1-11_amd64.deb
  to pool/main/s/sendmail/sensible-mda_8.14.1-11_amd64.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 375787@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Richard A Nelson (Rick) <cowboy@debian.org> (supplier of updated sendmail package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----

Format: 1.7
Date: Sat, 20 Oct 2007 23:35:30 -0000
Source: sendmail
Binary: libmilter-dev libmilter1-dbg rmail libmilter1 sendmail sendmail-doc sendmail-cf sensible-mda sendmail-base sendmail-bin
Architecture: source all amd64
Version: 8.14.1-11
Distribution: unstable
Urgency: low
Maintainer: Richard A Nelson (Rick) <cowboy@debian.org>
Changed-By: Richard A Nelson (Rick) <cowboy@debian.org>
Description: 
 libmilter-dev - Sendmail Mail Filter API (Milter)
 libmilter1 - Sendmail Mail Filter API (Milter)
 libmilter1-dbg - Sendmail Mail Filter API (Milter)
 rmail      - MTA->UUCP remote mail handler
 sendmail   - powerful, efficient, and scalable Mail Transport Agent
 sendmail-base - powerful, efficient, and scalable Mail Transport Agent
 sendmail-bin - powerful, efficient, and scalable Mail Transport Agent
 sendmail-cf - powerful, efficient, and scalable Mail Transport Agent
 sendmail-doc - powerful, efficient, and scalable Mail Transport Agent
 sensible-mda - Mail Delivery Agent wrapper
Closes: 375787 387799 433216 446415
Changes: 
 sendmail (8.14.1-11) unstable; urgency=low
 .
   * /etc/init.d/sendmail will now rebuild databases on
     start/reload/restart (like Redhat derived, various BSDs, etc)
 .
   * Finally nailed (fingers crossed) the elusive cause of
         ;; connection timed out; no servers could be reached
     There are still issues on what state to leave things in, but
     at least the file will be turned into garbage     closes: #375787
     + sendmail.conf::DAEMON_NETMODE now defaults to Static
     + /etc/{ppp,dhcp3,network,resolvconf}/*/sendmail pass an additional option
     + /usr/share/sendmail/dynamic is much more careful
 .
   * ARM is broken, disable -fstack-protector-all      closes: #446415
   * add FEATURE(use_cw_file) to default sendmail.mc   closes: #433216
   * remove /usr/share/bug/sendmail-doc -> sendmail    closes: #387799
Files: 
 f85f6759ed7db30d4aebe79dde717444 1157 mail extra sendmail_8.14.1-11.dsc
 9c74a2792c6d8205a651498917c5b7f3 355372 mail extra sendmail_8.14.1-11.diff.gz
 612efcc8ce570d44efebf68d5735ffc3 830062 doc extra sendmail-doc_8.14.1-11_all.deb
 b939e3786167e3659c9925fe09c00c36 202056 mail extra sendmail_8.14.1-11_all.deb
 6216885374c5a3c9af4216a9d037b665 354478 mail extra sendmail-base_8.14.1-11_all.deb
 0393f737bf1cf6169d2660eeefcd94e5 290690 mail extra sendmail-cf_8.14.1-11_all.deb
 274c4bdecf3f74ee59db1100182a72be 953598 mail extra sendmail-bin_8.14.1-11_amd64.deb
 02820aa92111e3311a0642a965692176 240732 mail extra rmail_8.14.1-11_amd64.deb
 7523976f8b545ef00e492963f257c993 210108 mail extra sensible-mda_8.14.1-11_amd64.deb
 973c5bed3f25f4bb259aecbe36f62591 229764 libs extra libmilter1_8.14.1-11_amd64.deb
 ac6f624bd8a1ad0c404a84f542d93add 250976 libs extra libmilter1-dbg_8.14.1-11_amd64.deb
 9b9e4bbdea6b193732888bc8935c9a27 318922 libdevel extra libmilter-dev_8.14.1-11_amd64.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iQCVAwUBRxqU3qVTksHk9ElFAQHCxwP/ZxAkPnViYhe51VtTHbRjHpYgQaXCCvVy
yf/tDX0PHolfcrR6MVMTvz+UXUKeO3tAz6kmJj8Ly/sd4yf+fJ32Jj9MvDW4RMKq
04edB/G6HwdosPWdY9bPk9pULI35DnDdjq4uHi+mvD6XeQ9L5KHx6/U3wwesdDUP
K+i8gbgZhUc=
=QTm/
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.