Slow outgoing email with sendmail

[xr200]

I have a laptop with PCLOS 2010 KDE installed on it.  I have sendmail
configured to use AT&T (my ISP) for outgoing SMTP email.  I sometimes
use mailx to send off quick emails when needed with no problems.

I read in a recent PCLOS magazine article about the ability to encrypt
filesystems on installation.  I created some new partitions and installed
LXDE 2011 with new encrypted home and root filesystem.  I set up sendmail
identically to what I have in my KDE partition, but outgoing emails often
take many minutes to complete.

I finally learned to look at /var/log/syslog, and started looking for
sendmail Sent OK messages, and in those messages, look for the value
of 'xdelay'.  On my KDE partition, they always take a few seconds.  When I
boot into my new LXDE partition, the xdelay is much greater, often
taking 3-9 minutes to complete.  This is a consistent problem.  Every so
often, I will get an email to go out quickly on the new LXDE partition, but
almost always, it takes several minutes.

I have the same boot services running in both KDE and LXDE.  I have a lot
more software packages installed on KDE, but that's just because I've used
it a lot longer.  The differences I know of are the desktop environment
(LXDE vs KDE) and the fact that my filesystems are encrypted in the LXDE
environment.  I haven't noticed any performance hit at all on the LXDE
partition (related to encrypted filesystems or otherwise).  I don't think
either of those points matter, but something obviously does.

Does anyone have any idea why my outgoing emails on from the LXDE/2011
partition take several minutes to be sent, but from KDE/2010, they go
out in a few seconds?

[AS]

Hi xr200,

I read in a recent PCLOS magazine article about the ability to encrypt
filesystems on installation.  I created some new partitions and installed
LXDE 2011 with new encrypted home and root filesystem.  I set up sendmail
identically to what I have in my KDE partition, but outgoing emails often
take many minutes to complete.

About encrypted partition: surely encrypting the root partition imply some CPU overhead, because of continuous encryption and decryption of each blocks read / written for each data or executable files. That said, this behavior apply system-wide. Did you noticed a general slowdown in your LXDE system ?

You can also try monitor the CPU usage, i.e. using top or htop or whatever you prefer, particularly when you use mailx.

I'm greatly interested about your feedback. 

Also, there are additional log files in /var/log/mail, take a look there for possible hints.

How large are the message you are sending out ? few Kb ? some Mb ?

Try to send an email to yourself, and watch at mail message source for each timestamp in the header, so that you can clearly identify at which step the delay has been introduced.

More, try to send 2 time the same message to yourself, the second time most probably each necessary block of data/executable will be already in the system cache, thus will not read/decrypted the second time. Should be interesting to compare the timing difference about the two messages, if any.

AS

[xr200]

I have not noticed any performance impact whatsoever using encrypted filesystems.

I tried running "top" as I sent some email using mailx, and I was getting over 95% CPU idle,
loads of free memory.

/var/log/mail/info.log seems to be completely contained in syslog, there's no additional information.

The messages I'm sending are all test message: just a few characters to my gmail account.

Looking at the headers, it appears the delay is between my laptop and the SMTP server.

If I send a series of separate messages in a row to my gmail account, it doesn't appear to matter
what the order is ; they all will arrive, but not necessarily in the same order they were sent,
and they take between 3-9 minutes, whereas using KDE/2010, it takes less than 1 minute.

On my desktop, I normally run Slackware.  I tried installing LXDE/2011 on the desktop as well,
using unencrypted partitions.  I had the same problem with very slow outgoing emails; 5 -9 minutes
each.  So I don't now think this has anything to do with encrypted filesystems.   I do have sendmail
working fine with KDE/2010 on the laptop and Slackware on the desktop.  But I can't figure out why
LXDE/2011 takes 5 minutes instead of 10 seconds.

My wife is taking the laptop over the weekend, so I won't be able to look at it again after tonight for
a while.  I'm considering installing KDE on my desktop just to see if there's any difference..
Maybe that's a waste of time, I dunno....

So in short, I don't think this is a problem with encrypted filesystems, sendmail itself, or the way
I'm configuring sendmail, or LXDE, or the kernel, etc.  I'm guessing I have configured something wrong
or I'm missing some package perhaps, and I'll be very happy to admit my error if someone can help
me figure out what the devil is causing sendmail to be so darned slow...

Thanks..... Walt

[AS]

I have not noticed any performance impact whatsoever using encrypted filesystems.

I tried running "top" as I sent some email using mailx, and I was getting over 95% CPU idle,
loads of free memory.

I was thinking so, .. asked just in case...

/var/log/mail/info.log seems to be completely contained in syslog, there's no additional information.

The messages I'm sending are all test message: just a few characters to my gmail account.

Looking at the headers, it appears the delay is between my laptop and the SMTP server.

If I send a series of separate messages in a row to my gmail account, it doesn't appear to matter
what the order is ; they all will arrive, but not necessarily in the same order they were sent,
and they take between 3-9 minutes, whereas using KDE/2010, it takes less than 1 minute.

On my desktop, I normally run Slackware.  I tried installing LXDE/2011 on the desktop as well,
using unencrypted partitions.  I had the same problem with very slow outgoing emails; 5 -9 minutes
each.  So I don't now think this has anything to do with encrypted filesystems.   I do have sendmail
working fine with KDE/2010 on the laptop and Slackware on the desktop.  But I can't figure out why
LXDE/2011 takes 5 minutes instead of 10 seconds.

It seems a starting point ...
Next check that come to my mind ... try some SSL connection ... the hypothesis is about some issue related to SSL protocol. For the same reason you should try to use a different email account that use plain SMTP, unencrypted.

My wife is taking the laptop over the weekend, so I won't be able to look at it again after tonight for
a while.  I'm considering installing KDE on my desktop just to see if there's any difference..
Maybe that's a waste of time, I dunno....

So in short, I don't think this is a problem with encrypted filesystems, sendmail itself, or the way
I'm configuring sendmail, or LXDE, or the kernel, etc.  I'm guessing I have configured something wrong
or I'm missing some package perhaps, and I'll be very happy to admit my error if someone can help
me figure out what the devil is causing sendmail to be so darned slow...

Thanks..... Walt

I agree with you, look like there is not direct correlation with encrypted partitions.

AS

[xr200]

I haven't gotten anywhere on figuring out the sendmail delays.

As a workaround, I've installed "msmtp" from the repository and am playing around with it.
It works slightly differently than sendmail: as far as I can tell, there's no queueing.
I've configured mutt to use msmtp, and when I send an outgoing email,  I don't get
my prompt back until the email has been sucessfully sent.   Might take 10-15 sends.
I actually like that: I can have gmail open, and send a test message via mutt/msmtp
to my gmail account, and as soon as I get my prompt back from mutt/msmtp, the message
appears in my gmail account.  With sendmail, because of the queuing, you get your
prompt back immediately, but you don't know how long it takes to send the email.

I've got some tinkering to do with my muttrc file to make it more like mailx, but overall,
I'm sure I'll be happy with the result.

I don't consider this a solution to the problem I have.  I'm just posting this in case someone
else is having sendmail delays and is also looking for a way around them.

Since msmtp works pretty darn fast, it makes my sendmail delays even more puzzling to me.

Walt.