How many messages do you keep in a folder?
Published January 10th, 2006 in KiwiI’ve been working on stress testing Kiwi. I setup an e-mail account and signed up to all of Apple’s mailing lists and a bunch of Linux mailing lists. I’ve collected about 10,000 messages so far. We are aiming at handling 20,000 messages in an IMAP folder (not per account, per folder) as efficiently and quickly as possible. Right now we can handle 10,000 without too much trouble. Displaying the messages is instantaneous and downloading them is blazing fast. We can download the headers for 10,000 messages at the rate of about 400 messages per second. The only part that is somewhat slow is downloading the list of messages from the server, once we have that list, Kiwi flies. We’ll be working on improving this in the near feature as well.
So my question to everyone is, what is the maximum number of messages that you have in a folder?
I am reaching 20000 messages on some mailboxes.
I was told about 50000 messages by some sylpheed users.
wow… I generally keep it under 1000, because (at least on my computer) Mail gets slow after that.
No where near 20,000 in a single folder. Maybe 5,000 total across about 20 folders. Hope that helps.
My largest Apple Mail mailbox (IMAP) has about 5K messages in it now. I’ve deleted quite a bit of mailing list backlog since recently migrating from Mulberry, which could handle as many IMAP mailboxes/messages I could throw at it without any trouble. Mulberry never lost or damaged even a single message that I was aware of during maybe seven years of use on Solaris and OS X. That level of integrity made up for its ugly and obscure UI. Mail looks nicer but its fickle and finicky behavior demands more attention than I’d like to give it.
Well I am not a perfect expample, but in my largest mail box I have about 7K messages. My email volume is increasing exponetially and I am currently getting about 60 messages per day. But I delete the newslettes after consumption. I have a lame mail server at work and it pretty much crashes when a mailbox gets to 10GB. So lets also think about the size of the mail box independant of messages. I can easily see my mailbox on my imap growing to about 20GB by the summer. (I know that is pretty ridiculous, but I have 3GB inbox, 5GB outbox and many messages offloaded to local folders) In case your interested, I am using the Thunderbird Beta and it is having problems dealing with my old messages. It can only read the messages from two weeks prior to my install date, but the other messages only headers show up.
I keep under 500 in each mailbox. I don’t get much mail though, about 4 messages a day.
3700 is the largest folder I have (but it is a subfolder! Does that create difficulties? together , the folder *might* be over 10,000)
What I should not forget is: how fast and versatile is searching? I have been a long-time fan of PowerMail, but for two serious shortcomings: poor IMAP support and no Unicode. But searching is lighning fast, and what is more, customizable to the extreme.
I’m a network admin. One of my responsibilities is archiving peoples mailboxes. Some of our users have really high email volumes and all the messages need to be saved. The biggest mailboxes are usually sent folders (sometimes inboxes for people that don’t like folders) and they can have as many as 40k messages. If you wanted to be really ambitious you could shoot for a number like that or the 50k mentioned above. 20k is probably a fine target for optimization. Bigger mailboxes don’t have to be as quick or efficient, they just have to work.
Looks like 20,000 is going to be our target. Hoa, do you have any suggestions on how I can best utilize your IMAP library with such a high message count?
Obviously, you’ll have to cache the message headers so that you won’t need to download them again. flags may be synchronized in background (they don’t need to be correct immediately).
That will result in :
- When the user will choose the folder, the cached messages will be displayed immediately (or mostly based on the disk speed),
- the non-cached message will arrive a little bit later, and will be displayed,
- then flags will be synchronized and the list will be updated.
(you have to keep the nextuid information, uidvalidity and of course, a cache of flags and a cache of headers)
There will only be a performance hit at first time when downloading the whole message list: this will be the case where non-cached messages list is equal to the whole messages list.
To avoid user frustration, you can download headers by group of 50 for example and display them as soon as you have them.
In best case: when there are no new messages, the cached message list will be displayed, a request will be done on server to get the list of new messages (the answer will be empty), then, the flags will be synchronized in background.
sea sun frog kitchen are all white home stone