OS X 10.8 vs. Ubuntu On Apple Hardware, Benchmarked 130
An anonymous reader writes "OS X 10.8 has been benchmarked against Ubuntu Linux with some interesting results. From the tests on a Apple Mac Mini and Apple MacBook Pro, OS X Mountain Lion was clearly superior when it came to the graphics performance, but the rest of the time the operating systems performed quite closely with no clear winner. OS X also seems to have greater performance issues with solid-state drives than Linux."
Re:Why is Linux's SSD performance so terrible? (Score:5, Insightful)
That's Not what the article said. It said OS X had performance issues with solid state drives.
Also I'm kinda curious: Why would spend twice as much to buy an Intel Mac PC if they're just running linux? I'd buy a regular PC for 1/2 to 2/3rd the cost.
Worthless... (Score:5, Insightful)
Lots of graphs, worthless analysis (Score:4, Insightful)
The compilation benchmarks are not comparable as the compilers are different, not only in version number but in architecture! OS X ships with llvm-gcc, which is a different compiler from GCC. Think of it a LLVM pretending to be GCC (accepting GCC options, etc) for backward compatibility. This would explain the huge discrepancies between the results of the compilation benchmarks
Disk performance is another thorny issue. The Postmark benchmark shows Ubuntu 12.04 being 3x faster than OS X 10.8 (246 tps vs 80 tps), yet the postgresql database benchmark shows OS X to be 3x faster than Ubuntu. No explanation is even attempted. Why? Readers would like to know! How can OS X be faster at a database benchmark when a raw disk benchmark shows it to be a lot slower than Ubuntu?! Perhaps there's something screwy with the configuration of Postgres on Ubuntu? Does this mean that OS X is *THE* choice for hosting busy databases? My suspicion is that this is due to fsync (http://www.postgresql.org/docs/8.1/static/runtime-config-wal.html). If fsync is enabled, the database waits for the transaction log to be flushed to disk every time a transaction is committed. It's basically down to defaults, and who knows what the default values are for Postgresql on OS X vs Ubuntu?
The graphs raise far too many questions that are not addressed. Many of them should have raised warning flags, like the one about disk performance vs actual database performance. As such, the results are thoroughly suspect and no reasonable conclusions can be drawn. Pity, because they clearly have the kit just not the knowhow.
Re:Why is Linux's SSD performance so terrible? (Score:4, Insightful)
Footprint, noise (or lack thereof), the ability to run all major OS (OSX, Windows, Linux) on the same machine, low power usage and nice looking sturdy construction. If you're going to be putting it on a desk the Mini is a nice little package.
Comment removed (Score:2, Insightful)
Re:surprise surprise (Score:5, Insightful)
The graphics drivers are written in house at NVidia and AMD. Apple doesn't actually write their own drivers. And the GPUs are just bog standard AMD, NVidia, and Intel GPUs (expect for some of the graphics switching.). There is not reason Linux should be at a disadvantage.
What, you mean aside from the fact that Linux drivers for all those respective device manufacturers don't really get a whole lot of attention from the developers compared to Windows and OS X?
But really, that's not important. What people need to pay attention to is that this is something done by Phoronix. This means you need to consider a couple things:
* This benchmark is almost meaningless. Time and time again, I have seen them (falsely) correlate data with an assumption.
* The review was done by someone who doesn't really know what they're talking about.
* These are synthetics. Without context or understanding of what the benchmarks are doing (there is no explanation) or what may have led to the
* The discrepancies are, in most cases, severe enough that you have to assume (at least) one of the following: their benchmark suite was not properly/identically configured for all architectures, or there are drastic implementation discrepancies within the benchmark tool they're using (eg. it wasn't designed but with a specific use case in mind).
The reason there is "no clear winner" is because it's all rubbish. They're throwing 100 things at two different targets and comparing what misses and saying "no conclusion". Really? You'd have better consistency with an ink blot test of random participants, with ink blots generated by a true random number generator.
Some of the graphics benchmarks don't stand out; the ones that stand out the most are the computational ones involving (very) standard libraries or frameworks which then contradict later results.
For instance, CompileBench and Threaded I/O Tester: OS X falls flat on its face. The threaded I/O tester I believe, because I've seen the same with db and server performance. But earlier, they've got bgbench giving OS X four times the performance for postgresql as Linux. Is that even rational, given that even FileMaker has shyed away from OS X as a preferred platform due to threading and filesystem performance?
Then, they go on to fail to explain these things and why they're fundamentally inconsistent. Not just "this doesn't quite line up, we can write it off due to different library version overhead" but in line with "this car goes faster because its engine is smaller". What?
On a more personal level, I have used their suite of benchmark tools and come away fairly underwhelmed by the results. They're inconsistent and inexplicable, such as those seen in this review.
Here's a hint, benchmarkers:
* when you benchmark something, you must compare things and try to figure out why they are performing as they do.
* If there are gross discrepancies which belay a reasonable expectation or contradicts other information, investigate it, because it's probably important
* Be sure of what you're comparing. If you've got (more or less) identical binaries on different platforms and the hardware, you're just comparing the kernel. Is that what's happening here? Are their tools linked against native libraries (which would, you know, be an honest benchmark of said platform) or do they use their own stack?
Anyway, I could go on, but you get the idea. This benchmark is stupid on its face. The only benchmarks I'd trust from this roundup here are those that are straight up "measure something real" (frames per second in x, time to complete concrete task y). They make a very different picture than when the synthetics are thrown in to the whole: overall OS X performance is pretty abysmal, but is marginally better at graphical things than Linux. This fits pretty close with my (personal) observation that OS X is about 10-15% slower than linux on general things, markedly slower on threaded things, and a dog at file manipulations while having a firm grasp on display management/graphical stuff - so it might just be my "uneducated Apple-hating bias" speaking. :)