Mac OS 10.9's Mail App — Infinity Times Your Spam 158
An anonymous reader writes "Email service FastMail.fm has an blog post about an interesting bug they're dealing with related to the new Mail.app in Mac OS 10.9 Mavericks. After finding a user who had 71 messages in his Junk Mail folder that were somehow responsible for over a million entries in the index file, they decided to investigate. 'This morning I checked again, there were nearly a million messages again, so I enabled telemetry on the account ... [Mail.app] copying all the email from the Junk Folder back into the Junk Folder again!. This is legal IMAP, so our server proceeds to create a new copy of each message in the folder. It then expunges the old copies of the messages, but it's happening so often that the current UID on that folder is up to over 3 million. It was just over 2 million a few days ago when I first emailed the user to alert them to the situation, so it's grown by another million since. The only way I can think this escaped QA was that they used a server which (like gmail) automatically suppresses duplicates for all their testing, because this is a massively bad problem.' The actual emails added up to about 2MB of actual disk usage, but the bug generated an additional 2GB of data on top of that."
Apple Build Quality (Score:2, Funny)
Re: Apple Build Quality (Score:1, Troll)
Gosh you're right. There's no software that ships with critical bugs these days.
Re: Apple Build Quality (Score:4, Informative)
Well, software that generates a thousand copies of junk (seriously, the spam folder of all things...) isn't very typical.
Re: (Score:2)
Yes, because this one incident that got reported to slashdot and made the front page for some reason ...
Reason: Apple haters cruise the firehose and up-vote anything that makes Apple look bad.
Re: (Score:2, Interesting)
Ships? For the past 3 releases Mail.app will randomly choose not to display the body of a message. I have to quit and reopen Mail.app just to read my e-mail sometimes.
I thought it was just me and something screwy with my account, but the other day I asked a coworker and immediately heads began popping up from cubicles everywhere within earshot, with people admitting they've had precisely the same problem.
Apple's focus on iOS and cute little phone apps has, for whatever reasons, caused defect rates in their
Re: (Score:2)
I can beat that, whenever someone sends me a mail containing a PDF, it will crash, not only the mail client, but take down the userspace with it.
Re: (Score:3)
Definitely write a bug report at bugreport.apple.com with relevant crash logs in ~/Library/Logs/DiagnosticReports and /Library/Logs/DiagnosticReports.
Re: (Score:2)
Yes, Apple is *gasp* a company and is driven by profits. When something makes up over 80% of your revenue (and increasing) you focus on that and not the remaining (and shrinking) slice.
Apple's focus on iOS and cute little phone apps has, for whatever reasons
Because that's actually making them money.
caused defect rates in their core desktop code to serious balloon.
Macs and OS X are Apple's side business, they haven't been core for a long long time.
Re: (Score:1)
Re: (Score:1)
Then you probably have a corrupted mail database. I've experienced the same thing, fortunately, its IMAP so wiping out the mail configuration and recreating it was fairly painless and the problem went away. I've assumed thats due to the fact that I use the beta versions, which of course have bugs.
Re: (Score:3)
So you wrote a bug right?
bugreport.apple.com
Re: (Score:2, Interesting)
"There's no software that ships with critical bugs these days."
You'd better believe the stuff that ships with my food production systems is FLAWLESS.
When an entire country's population relies upon your tech to even stay alive, you make sure your shit works.
And I work on a budget a billionth Apple's worth.
What's their fucking excuse?
Re: Apple Build Quality (Score:5, Insightful)
If you think the 'stuff that ships with my food production systems is FLAWLESS' you are an idiot who doesn't understand the way the world works.
I've worked in food production and medical, nothing is flawless, you have to be an idiot to make such retarded statements.
You mitigate the risk, try to double/triple/quadrupal check for problems and build in fail-SAFE systems, but you are not flawless.
Your statement sounds more like an arrogant cluebie beginner who's going to get a nice spanking when reality finally hits.
Nothing is flawless, to imply you've created something flawless shows your ignorance.
Re: (Score:3)
10 PRINT "HELLO WORLD"
20 GOTO 10
Flawless.
Re: (Score:1)
Doesn't meet the design requirements.
Re: (Score:2)
Congrats! You managed to get stuck in an infinite loop in a self-contained system just two simple lines, instead of the dozens it took for Apple to do it in Mail against a remote server's junkmail system.
Re: (Score:2)
It's perfect. No unfreed memory. Since it's not a function call, there's no stack that can fill up. I can't think of ANY way to improve this!
Re: (Score:2)
Re: (Score:2)
Why would you want to exit if it's flawless?
Also, plenty of things are unstoppable withough killing them, notably, many deamons.
Re: (Score:2)
You are assuming it needs to finish. Infinite loops are common in embedded and control systems. They run until they are decommissioned and powered off. Stopping might be considered a bug, e.g. there should be no way to disable or stop the safety monitoring process in an industrial control system.
Software can be mathematically perfect, just not complex software written by human beings.
Re: (Score:2)
"If you think the 'stuff that ships with my food production systems is FLAWLESS' you are an idiot who doesn't understand the way the world works."
No, you're an idiot that doesn't understand how my system works. But that's okay, it's pretty much above your level in the first place because it all happens in pure hardware logic.
Re: (Score:2)
Flawless software exists. NASA can do it...
But not always, even when they try. The LEM had a couple of bugs [doneyles.com] that made the Apollo 11 landing even more dicey than it would have otherwise been. Mars Pathfinder had a priority inversion bug that caused software resets. And of course the Mars Climate Observer was lost because of its famous metric vs imperial unit mixup. [microsoft.com]
And those are just the ones that I know about.
Re: (Score:2)
When an entire country's population relies upon your tech to even stay alive, you make sure your shit works. [...] What's their fucking excuse?
Nobody relies on Mac Mail to even stay alive?
Re: (Score:2)
your budget is less than $500?
Re: (Score:1)
You'd better believe the stuff that ships with my food production systems is FLAWLESS.
I don't believe it for a moment. Send it to me and I'd certainly find flaws in it.
I believe you have a big ego though.
Re: (Score:3)
How long has imap been around now? The RFC was last updated in 2003 - thats only 10 years to get it right.
Re: (Score:2)
Oh, there are plenty of new RFCs coming out all the time. The most interesting bugs in mail clients tend to be trying to support new RFCs and not getting it right.
One infinity drive. (Score:5, Funny)
Not just an address anymore.
Re: (Score:2, Funny)
If you're making a joke about the address of Apple's corporate HQ, you got it wrong:
1 INFINITE LOOP
CUPERTINO CA 95014-2083
Re: (Score:3)
Maybe he just needs to drink a cup of very hot tea.
Re: (Score:2)
"You can never get a cup of tea large enough or a book long enough to suit me." - C.S. Lewis
http://www.booksatoz.com/witsend/tea/orwell.htm [booksatoz.com]
Re: (Score:2)
Maybe he just needs to drink a cup of very hot tea.
And something to stick in it.
Re: (Score:2)
Improbable.
All hail Apple's new storage technology! (Score:3, Funny)
Re: (Score:3, Informative)
From the summary: "The actual emails added up to about 2MB of actual disk usage, but the bug generated an additional 2GB of data on top of that." Which I assume means that it was really only 2MB of emails, but the duplication (ie millions of emails) used up 2GB.
Re:All hail Apple's new storage technology! (Score:5, Funny)
We de-duplicate on COPY, so there was only one copy of each email on disk. We don't de-dulplicate metadata though, because it's usually so small, and generally in the cache file of a different folder, where de-duplication isn't possible.
Re: (Score:2)
i've noticed weirdness when moving messages between mailboxes. drag and drop to move, then later i notice it is still there. gmail account.
Re: (Score:3)
Enable the "all mail" folder to be visible in IMAP, and remove [Gmail] as your IMAP path prefix. That fixed it for me.
Re: (Score:2)
Thanks
Re: (Score:2)
Actually, there is no COPY in this scenario; Mail.App sends a command to save a new message to the server, with the same content as the other one.
Re:All hail Apple's new storage technology! (Score:4, Informative)
Sorry, I'm not quite sure where you're getting your information about what Mail.App does. I'm getting mine from the server telemetry logs where the client first identifies itself as:
"name" "Mac OS X Mail" "version" "7.0 (1816)"
And then proceeds to issue a COPY command:
UID COPY 3360991:3361069 "INBOX.Junk Mail"
See the "COPY" in there. I am the author of the blog post, and I think my credentials in this particular case trump yours, even if you're the author of Mail.App.
Re: (Score:3)
Wow, Apple should be widely lauded for being able to store each email, including its header, in just one byte!
Even that is awfully wasteful. To store spam wisely you use a counter. If the size of the counter is 64 bits, each individual spam message occupies 64/2^64 bits, or 3.5e-18 bits.
Naturally, all spam will look alike to you, but doesn't it already?
Re: (Score:2)
Wow, Apple should be widely lauded for being able to store each email, including its header, in just one byte!
Even that is awfully wasteful. To store spam wisely you use a counter.
Store spam? I recommend a 1-bit counter set to zero.
Re: (Score:2)
They de-dupe disk storage, so 2 million copies of 71 emails take up no more room than the original 71. The index metafile *about* those duplicate messages, on the other hand...
Re: (Score:2)
"The actual emails added up to about 2MB of actual disk usage,"
So the 1,2, or 3 million emails occupied just 2MB of storage?
Wow, Apple should be widely lauded for being able to store each email, including its header, in just one byte!
It's 71 messages. The 1, 2, 3 million being mentioned is the UID being incremented, not more space being eaten.
Gave up on Mail.app years ago (Score:2)
Seems like Mail.app has been getting worse since about 2003. I finally gave up on it about 5 years ago - in favor of gmail's web interface. At first I was a little disgusted with myself - but I've never regretted it.
I still use mail on my iOS devices, though. Have not yet seen a better UI for those.
Re: (Score:2)
Seems like Mail.app has been getting worse since about 2003. I finally gave up on it about 5 years ago - in favor of gmail's web interface. At first I was a little disgusted with myself - but I've never regretted it.
I still use mail on my iOS devices, though. Have not yet seen a better UI for those.
I agree. I find myself using too many machines, in too many places to really care about a desktop version of the mail program, especially now that my mail storage is using about 15GB of data. The only "mail app" I use is the built in app on my iPhone, otherwise all web portals for me.
Re: Gave up on Mail.app years ago (Score:2)
I use mail on iOS and google gmail client for work mail. Both are decent but I think gmail is slightly better. You should try it. I don't actually like Gmail all that much but the iOS app is good.
As for desktop, mail.app is buggy but not as bad as trying to use thunderbird. All mail clients suck
But I thought it just worked! (Score:2, Troll)
Apple can't have bugs like this- it just works. Maybe they're sending it wrong?
Whar is wrong with programmers? (Score:3, Insightful)
Re: (Score:2)
I mean, it is a MAIL program, not a revolutionary new product. The protocols have been out there for years (esp. IMAP). Why is it still buggy? Even worse: why is it buggier than the previous version? If it worked before THERE IS NO F*ING EXCUSE FOR IT NOT TO WORK NOW. Very very very lame.
I would imagine they have uplifted the app and re-written a large portion of the application to work with new interfaces like outlook or whatever. Also perhaps they're trying to do something new with spam in specific to help reduce spam as a whole when using various services. This is a bad bug, and could cause a lot of problems for service providers if it's legitimate and not a "single case", but bugs happen. I'm surprised nobody caught it with the beta versions. Apple has been surprisingly good lately
Re:Whar is wrong with programmers? (Score:5, Interesting)
Clearly, whoever rewrote Mail to "better" support Gmail decided that as long as it worked okay with just the right settings, any deviation from that wasn't a bug but just user error. Despite the fact that those settings were both perfectly valid and *incredibly* common.
I think moving OS X to a yearly release schedule results in them pushing things out too fast. It's bad enough with iOS, and OS X is a more complex beast.
As someone once said... (Score:5, Funny)
Re: (Score:2)
..what? it works fine with "vanilla" settings but not on their specific hacks??
Re: (Score:2)
Re: (Score:2)
If the bugs were called duplicates, and the bugs are still open.. then clearly they weren't duplicates.
PLEASE write new bugs, with enough detail about the settings necessary, at bugreport.apple.com.
(I've been writing up VLC bugs for example, even though *they* close the bugs on us, and don't let the originator close the bug.. I wish all companies had public ways to officially file bugs, even though yes, products should have many many many fewer bugs when released. Zero is unfortunately probably impossible
Re: (Score:3)
There's a long list of answers to that - but the real question here is what is wrong with the testers that let something stupid from a rookie or slapdash programmer out into the wild. The answer to that is probably that they are at home looking though the job ads because proper testing was not considered important enough.
Email problems are fun when they happen to somebody else because of how much they can snowball into ridiculously huge numbers of messages. I've worked with
Blame Google. (Score:2)
Re: (Score:2)
Compare it to writing a book and making a grammatical error. The rules of grammar are nothing new, people have been writing books for centuries, and there are even tools that can automatically check a lot of it for you. How on earth could anybody publish a book with a grammatical error!? THERE IS NO FUCKING EXCUSE FOR IT!
A bug like this is the equivalent of a grammatical error when writing a book. The simple fact of the matter is when you've got humans writing millions of lines of code, some of them
Re: (Score:2)
Re: (Score:2)
It's incredibly common to update a book. Haven't you ever heard of different editions?
Yes, and then you make changes to it. By definition, that's what updating software is. Those changes can introduce new problems, just as rewriting a chapter of a book can introduce grammatical errors.
You are literally demanding
Re: (Score:2)
You are literally demanding perfection. Exactly how many professions are there where people don't make mistakes?
Well... programmers seems to be among the worst...
Re: (Score:2)
There was a time when the posters here knew the answer to that question because they were actually programmers.
what did you expect (Score:2, Flamebait)
This is the company that can't correctly implement daylight savings time in an alarm application. You expect them to handle something as complex as implementing an ages old industry standard correctly?
Apple is known for selling lots of shiny bobbles, not for writing solid code.
Nothing new (Score:1, Troll)
Apple has always been better at marketing than software engineering. The iSheep hipsters don't care. It's more important to be seen at the coffee shop using a Macbook than actual productivity.
iMail has a history of infinate recursion (Score:4, Interesting)
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
Neither of those is the correct name. There is an app called "Mail.app" (with the extension) in OS X (previously known as Mac OS X).
Re: (Score:2)
It's a defense against the NSA (Score:5, Funny)
Good job Oracle (Score:2)
Re: (Score:2, Funny)
Why doesn't fastmail also use servers that suppress duplicates?
The guy who approved it sent the approval via email on a Friday evening ... from his Mac. Since the recipient received millions of copies over the weekend he just figured it was spam.
Re: (Score:3)
Watch the Mail Activity section when receiving or sending mail. I have no idea what it's counting.
When I receive two emails and it says receiving 415 and 416 of 416 I kinda get concerned.
When sending one, sending 6 of 6... again, what is it counting?
And set up a smart folder that is all Unread mail. Set it up and watch that it's not very smart... Like it really can't tract unread mail at all...
Re: (Score:2)
I'm using Mail.app with Dovecot as the IMAP server - I upgraded to OS X 10.9 a few days ago, and haven't seen anything weird going on (yet). I sent myself a test email a few minutes ago while watching the Mail Activity window, and numbers appeared sensible. dovecot.index and dovecot.index.cache files on the server aren't ballooning - at 178KB and 11MB respectively.
The Fastmail article mentions Cyrus as the IMAP server. Is it Cyrus-specific, or have I simply not been bitten by this yet? (I get loads of spam,
Re:I guess I have to ask (Score:5, Informative)
It's also worth noting though that so far, there is only a single report of this, despite the author implying they have a huge number of users. Most likely this isn't something that happens on the average Mail install; it could be that Mail is hitting some error condition on this user's specific account and that is causing the bug to manifest.
Re: (Score:2)
It appears to only be that one user so far - I've seen a few isolated "copy into same folder and expunge the old ones" in the logs (now that I log that) but not enough to be a pattern.
Re: (Score:2)
BTW, the comment link on your page doesn't actually DO anything. You should write a bug with the details, even though it's hitting only one user. bugreport.apple.com
Re: (Score:2)
I did, and I've passed the user's contact details on to the engineers at Apple so they can talk to them directly. I've also re-enabled the account, and I'm just keeping an eye on the server that the user is on and moving some other users off so we can afford the disk space for a bit.
Re: (Score:2)
Why doesn't fastmail also use servers that suppress duplicates?
"This is legal IMAP, so our server proceeds to create a new copy of each message in the folder."
Because thay followed the standards, unlike Google that made this bastard imap like interface that has very poor "wake up" performance.
Re: (Score:3)
Actually part of the problem is they made Mail.app work better with Gmail, but all the hacks that used to be necessary really screw things up. Apple should've posted a FAQ about the changes rather than quietly make them.
One big change... you need to enable "All Mail" in IMAP now, since the latest Mail.app wants that as the Archive folder (which makes sense). But everyone has it disabled in IMAP since, up until now, it was problematic to do otherwise.
I found https://tidbits.com/article/14219 [tidbits.com] to be helpful.
Re: (Score:3)
Even enabling All Mail doesn't do the trick - from that tidbits article: (which has been doing the rounds quite a bit)
That is, I can read, move, delete, reply to, or otherwise operate on messages in my Inbox on the Gmail Web site, on my iPhone or iPad, or in another IMAP client, and they all sync up perfectly with each other — but even after several hours, my Inbox in the Mavericks version of Mail doesn’t reflect those changes. It seems not to matter how frequently I tell Mail to check for new m
Re: (Score:2)
Re: (Score:2)
Not to comment on the rest of of your note (I don't use GMail so have no idea what its support is like, on either end) but on your other complaint I have a hard time agreeing:
Not that having Mail cause problems is anything new; my personal favourite is the way Mail does embedded attachments, causing most other mail clients to struggle to handle his messages - usually, they end up with half an email, the attachment, and a second (and sometimes 3rd and 4th) set of attachments with the rest of the email message piecemeal. And then he complains that people can't read his bloody mail.
I have seen this but the garble I've experienced has only been with users with Outlook. (There may be other clients with problems reading them -- I don't know.) I've looked at those Apple Mail.app-generated messages and they appear to me to be completely RFC compliant. Very strictly so.
Now Apple Mail may or may not suck, but in this
Re:Apple done fucked up good (Score:4, Informative)
[...] I mention that because Apple now seems to be my Microsoft. iOS 7 is ugly as fuck.
My fucks are always beautiful. Or at least pretty (when I am more desperate).
Re: (Score:3, Interesting)
OS X has been going downhill (the autosave/versioning sucks for how I use software) and now with 10.9 mail.app regressions and iWorks losing features. I'm not upgrading to iOS 7. I'm not sure if I'll upgrade to 10.9 I need to buy a new computer in a couple months so I may switch to OpenIndiana. Maybe Linux for steam box, we'll see.
The last uphill version was 10.5. This current 10.9 is in big part back-pedalling the visuals of 10.7/8 without removing the functional crap they introduced. I decided not to go beyond 10.6 the moment I saw "Edge Resize" in 10.7 :-(( So... no, thanks - even being free (as beer) doesn't make it more appealing..
Re: (Score:2)
I decided not to go beyond 10.6 the moment I saw "Edge Resize" in 10.7 :-((
As OS X users finally get the power to unintentionally resize their terminal emulator windows when trying to select text that goes right up to the margin, just as other UN*X users have had for ages. :-)
(Yes, I've done that on a number of times on FreeBSD/Linux/Solaris/etc. inside $PICK_YOUR_TERMINAL_EMULATOR, but didn't feel the joy of doing that on OS X until Lion.)
Re: (Score:2)
This does not actually happen on OS X... But now I'm curious... how on earth does that happen on other UN*Xs?
Re: (Score:3)
Out of curiosity I've just tried, with every terminal application I have (xterm, lxterm, uxterm, gnome-termonal, konsole...). I can't do it. Perhaps I'm incompetent? Or perhaps it's a bug in some specific window manager? Or perhaps Guy Harris [slashdot.org] is special?
Re: (Score:2)
Re: (Score:1)
Running what window manager? In Unity on 12.04 theres a good 5-10 pixels worth of border to grab on to.
Re: (Score:1)
Re: (Score:2)
Out of curiosity I've just tried
If you're trying, you're probably doing it slowly and carefully enough that you don't hit the window border drag region. This happens (on occasion, not particularly often, but often enough to be a small irritation) when I'm doing a quick select-and-copy (or, when not on a Mac, a quick select-and-paste-current-selection).
Re: (Score:1)
Re: (Score:2)
If I ever did that, I'd stop working and have my eyes and brain checked for: 1.Not seeing the cursur icon
Sorry I didn't notice the cursor image change in the few milliseconds between hitting the region where selecting grabs the border and clicking the trackpad.
2.Using mouse when working with a terminal
In which desktop environments can I copy text from a terminal window without using a pointing device?
Re: (Score:1)
How are you unintentionally clicking your mouse on the window border while trying to select text? Just hovering over it doesn't do it, just dragging over it doesn't do it ... not sure what your problem is but you have to actually click the border to drag it. Clicking in the window then dragging over the border doesn't do anything.
Do you have pulsy or something?
Re: (Score:2)
How are you unintentionally clicking your mouse
I don't have a mouse, but presumably you meant "mouse or trackpad".
on the window border while trying to select text?
By having the pointer a slight bit to the left of the left-hand edge of the text in the window.
ust hovering over it doesn't do it, just dragging over it doesn't do it ... not sure what your problem is but you have to actually click the border to drag it.
Wrong. The cursor changes from the text cursor (meaning it'll select text if you drag it) to the horizontal resize cursor (meaning it'll drag the border) before you hit the border.
Do you have pulsy or something?
No.
Re:Apple done fucked up good (Score:4, Insightful)
iOS 7 is ugly as fuck.
Millions of people love it.
But you, personally, hate it.
So... it must suck.
Millions of people love the Kardashians, Honey BooBoo and Jersey Shore. Your point is irrelevant.
Re: (Score:2)
And of those, the vast majority didn't adopt it by choice, they just let their device install the update when it notified them that one was available. They use it now whether they like it or not.
I bought a new phone with iOS7 preinstalled so I never had a choice in the matter, but I've been perfectly fine with it. The icons are kind of bright but the applications themselves are clean and sterile in their design. The visual flair (animated backgrounds, parallax effect) are superfluous but easily disabled.
Re: (Score:1)
Well, if you're interested, I'm looking to sell mine. Mind you, as you can see in this post's header, my ID is so low, it's not even listed. Wrap your head around that one: It's so low, it does not need numbers! /. (almost on every article ever, in fact - usually multiple times). So, in order to protect my reputation, if we can agree upon a price, I will have to reset my karma so that you don't get that benefit.
Of course, I've been posting regularly here on
If that's acceptable to you, then please contact me