Mailbox Corruption

Mailbox Corruption

Post by Scott Well » Tue, 25 May 1999 04:00:00



I've seen a couple threads regarding concurrent access mailbox corruption in
this newsgroup.  I will be implimenting an IMAP server (UW IMAPd) running on
OpenBSD that is going to be used by about 60-70 people, up to 10
concurrently, however none of them will be accessing the same mailbox.  I am
assuming that the mailbox corruption problem occurs when two users try to
access the same mailbox, when the mailbox format doesn't support concurrent
access?  (I'm currently using mbox format mailboxes, but will probalby
switch to mbx if performance dictates).

Thanks,

- Scott Wells
- Shadow Systems

 
 
 

Mailbox Corruption

Post by Mark Crispi » Tue, 25 May 1999 04:00:00


Mailbox corruption is not supposed to happen, ever.

Certain site mistakes, e.g. using NFS for mail files, can cause locking
failures that will lead to mailbox corruption.


> I've seen a couple threads regarding concurrent access mailbox corruption in
> this newsgroup.  I will be implimenting an IMAP server (UW IMAPd) running on
> OpenBSD that is going to be used by about 60-70 people, up to 10
> concurrently, however none of them will be accessing the same mailbox.  I am
> assuming that the mailbox corruption problem occurs when two users try to
> access the same mailbox, when the mailbox format doesn't support concurrent
> access?  (I'm currently using mbox format mailboxes, but will probalby
> switch to mbx if performance dictates).

-- Mark --

* RCW 19.190 notice: This email address is located in Washington State. *
* Unsolicited commercial email may be billed $500 per message.          *
Science does not emerge from voting, party politics, or public debate.

 
 
 

Mailbox Corruption

Post by Charles Hedri » Wed, 26 May 1999 04:00:00



>I've seen a couple threads regarding concurrent access mailbox corruption in
>this newsgroup.  I will be implimenting an IMAP server (UW IMAPd) running on
>OpenBSD that is going to be used by about 60-70 people, up to 10
>concurrently, however none of them will be accessing the same mailbox.  I am
>assuming that the mailbox corruption problem occurs when two users try to
>access the same mailbox, when the mailbox format doesn't support concurrent
>access?  (I'm currently using mbox format mailboxes, but will probalby
>switch to mbx if performance dictates).

I suspect the more common situation is when the same user tries to
access his mailbox more than once.  It can happen at least the following
ways:

 - a client program opens more than one mail connection (e.g. to check
        for new mail, or simply because of bad coding -- as in older
        versions of Netscape Communicator)
 - the same user is logged in from more than one computer
 - the mail delivery agent delivers mail to the mailbox at the same
        time the user is updating it

I believe the third is the most likely, but I've seen the first also.

I believe the UW imapd does synchronization correctly, as long as
everything is stored on a single machine.  I.e. as long as you're not
accessing mail over NFS or some other network file system. This
assumes that imapd is built to use the same locking mechanism as your
mail delivery agent.  

Synchronization across multiple machines is hard in Unix: NFS and
lockd aren't really up to the job, though I claim it can be made to
work well enough for practical purposes in most configurations.

 
 
 

Mailbox Corruption

Post by Carl S. Gutekun » Thu, 27 May 1999 04:00:00




> - the mail delivery agent delivers mail to the mailbox at the same
>    time the user is updating it

Definitely the third one in my experience, and it's obvious variant: two
delivery agents writing at the same time. (If you ever see two messages
totally intertwined, that's what happened.) Oddly, it seems like this is the
problem that gets the least attention from admins and developers.

There is a file-based locking algorithm called the "hitching post" that works
quite reliably over NFS.  The UofW code uses it, as does procmail, and most
software that has been influenced by c-client, like DtMail. But I don't think
anyone has done a mail.local or /bin/mail that uses that technique, and they
all have problems with timing windows over NFS that allow the delivery agent
and the client (or two delivery agents) to get write access to the file at
the same time.

I evade the issue (and many other evils of Berkeley mailboxes) by using UofW's
tmail delivery agent (when I'm not using SIMS ;-).

(Hi Chuck!)

<csg>

 
 
 

1. Mailbox corruption

Trying to track down an odd bug....

I am using Pine (4.33) to connect to Exchange (5.5.2653.13) through
IMAP.  I generally save my outgoing messages to a folder, and also
often take advantage of the pOstpone feature.

I've noticed, however, that outgoing messages (either saved to a
folder when sent, or saved to the postponed-msgs folder) are often
getting corrupted.  The nature of the corruption is that the end
of the message will appear before the beginning (including headers).

For example, this happened today:
-------------------------------- cut here ---------------------------
#=-
-=#| 1412C DCL, Workstation Services Group, CCSO Ofc:(217)244-3862 |#=-

Date: Tue, 18 Dec 2001 14:00:37 -0600 (CST)




Subject: Re: SGI surplus

Message-ID:

MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

It is *very* common.  I currently have four dead Indy power supplies on
my shelf.  I'm hoping to figure out how to fix them some day....

Thanks,

Damian Menscher
--

-=#| 488 LLP, 1110 W. Green St, Urbana, IL 61801 Ofc:(217)333-0038 |
-------------------------------- cut here ---------------------------
(space removed from after the sigdashes above to prevent issues
with people's newsreaders truncating the rest of my message)

I think it's somewhat suspicious that the first 1000 characters of
the message (including headers) are stored together (though moved
to the end.  I'm told that Exchange uses a database to store mail
files... perhaps this is related?

And yes, the messages are corrupt on the server.  Viewing them
using LookOut or the web interface shows the same corruption.

I'd be very interested to hear if anyone has experienced anything
similar.  Right now, I don't know whether the problem is with Pine
or with Exchange.  But it's something we've been seeing off-and-on
for about a year, so it would be really nice to track down!

Damian Menscher
--

-=#| 488 LLP, 1110 W. Green St, Urbana, IL 61801 Ofc:(217)333-0038 |#=-
-=#| 1412C DCL, Workstation Services Group, CCSO Ofc:(217)244-3862 |#=-

2. Win 2000 client to VPN, For a couple of seconds...

3. Eudora: continuous mailbox corruption

4. Q: Toolbuilder to set variables in NetView

5. Mailbox corruption

6. US/Canada Only One Digit Code?

7. Sendmail 8.6.8 mailbox corruption

8. FS: Indigo2video for IMPACT systems

9. mbx mailbox corruption and quotas?

10. menu&toolbar corruption

11. OE password corruption

12. Video Corruption