Apple Hardware VP Defends Benchmarks 1081
He said Veritest used gcc for both platforms, instead of Intel's compiler, simply because the benchmarks measure two things at the same time: compiler, and hardware. To test the hardware alone, you must normalize the compiler out of the equation -- using the same version and similar settings -- and, if anything, Joswiak said, gcc has been available on the Intel platform for a lot longer and is more optimized for Intel than for PowerPC.
He conceded readily that the Dell numbers would be higher with the Intel compiler, but that the Apple numbers could be higher with a different compiler too.
Joswiak added that in the Intel modifications for the tests, they chose the option that provided higher scores for the Intel machine, not lower. The scores were higher under Linux than under Windows, and in the rate test, the scores were higher with hyperthreading disabled than enabled. He also said they would be happy to do the tests on Windows and with hyperthreading enabled, if people wanted it, as it would only make the G5 look better.
In the G5 modifications, they were made because shipping systems will have those options available. For example, memory read bypass was turned on, for even though it is not on by default in the tested prototypes, it will be on by default for the shipping systems. Software-based prefetching was turned off and a high-performance malloc was used because those options will be available on the shipping systems (Joswiak did not know whether this malloc, which is faster but less memory efficient, will be the default in the shipping systems).
As to not using SSE2, Joswiak said they enabled the correct flags for it, as documented on the gcc web site, so that SSE2 was enabled (the Veritest report lists the options used for each test, which appears to include the appropriate flags).
Curious (Score:5, Interesting)
Did you recieve a phone call directly or something(Apple calling Slashdot)? If so did they act really aggressive wanting to make sure people don't become anti-G5 before it is even shipped?
Not too important you might say, but interests me.
Impressive turnaround speed (Score:5, Interesting)
Okay, so Apple needs the G5 to succeed in order to survive. Motorola just aren't sending out the chip upgrades fast enough. They (Motorola) have enough other problems in their wide range of markets that they're in that not having to worry about CPU competition is probably a good thing as far as they're concerned.
The fact that the (almost) top person at Apple has made this clarification shows how much importance they're putting against these claims. Given that nobody else has had a chance to verify yet, and people are making wild speculations based off of paper and a lack of understanding, it's probably just as well that they're putting a positive spin on things.
Maybe the documents should have been clearer, showing why these configuration decisions were taken.
The "we had to use GCC" argument is a little strange though; is there any other good compiler available for the PPC at the moment? if so, I'd like to know; I use macs myself!
Amazing how people trust some no-named net person (Score:5, Interesting)
Like, the switch -mfpmath=sse when used in a P4 *does* use SSE2, but this guy thought just cause the switch flag says sse that it must be SSE only.
Then someone else (can't find the post, on usenet, under the mac advocacy group) pointed out that Dell's SPEC tests also disabled hyperthreading [specbench.org].
Then, based on this person's web page who no one even knows who he is, they start drawing conclusions that if Apple faked these (based on his flawed analysis), that they also must have faked those Adobe, Mathmatica, and other demos -- despite the execs for those companies being on stage also confirming the results.
Gotta love the net...
As for me, I don't know what to believe. I'm just going to patiently wait until some reputable sites spend a lot of time and do an in depth analysis and their own benchmarks, like Tom's Hardware for example. Then I may start drawing my own conclusions.
As for me, all I want is to be able to encode mpeg video at something greater than real time. Show me *that* benchmark please!
2 GB RAM vs 1.5 GB for the G5 (Score:2, Interesting)
I guess my take is this... (Score:5, Interesting)
There are those people who want to know if those numbers are EXACT 101% of the time, so they go bust out their dynamometer and begin writing complaint letters when their engine only hits 195HP.
I think benchmarks these days are no longer a science that they used to be. There are far, far, FAR too many hardware and software variables to do an accurate cross-platform analysis and comparison.
I mean, is it really logical to compare Apples (har har har) to Oranges? I mean, most all applications that will be running on the G5 will be optimized for the G5. So does it matter how a 'comparable' application will run on x86? No, because the x86 Application might have a few more optimizations which would make the comparison pointless.
These days people should take benchmarks with a grain of salt. Just another selling point they'll put on the big list of bulleted marketing jargon on the back of the box to try and rope in first time buyers who are turned on by big acronyms and high-tech sounding words.
So yeah, I think people just need to cool their heels and take this for what it is, just marketing propaganda. Does QuantiSpeed really make your CDs burn faster? [enusbaum.com] No. Does the P4 make âthe internetâ(TM) faster? [intel.com] No. Just take it for what it is and let it go.
Re:Compiler's should be included (Score:4, Interesting)
This is the same when you do any sort of scientific test. Eliminate all variables except the ones that you're testing for. We're trying to test processors and processors only. All you get from the SPEC numbers where you use Intel's compiler on Intel and GCC on PPC970 is an indication that the GCC compiler is inferior. You can't draw any reasonable conclusion from the test at all.
Re:Honesty (Score:5, Interesting)
You mean the new chips from Intel that were announced the same day as the G5s?
Shit, some people you can never please.
and by the way (Score:4, Interesting)
Worst Case Tests (Score:2, Interesting)
Parity by using gcc (Score:4, Interesting)
Re:Honesty (Score:3, Interesting)
And, as I mentioned below, the most egregious benchmark, the Quake one, wasn't mentioned at all. Yet independent tests had P4's running that benchmark at nearly double the quoted speed.
I can understand focusing on the benchmarks quoted on Apple's site. However there was a lot more done during the keynote that he conspicuously didn't mention.
Re:Even if Apple is faster (Score:4, Interesting)
Pricing on the lower end models are not as aggressive, but for what you get, it's still reasonable.
Re:Even if Apple is faster (Score:5, Interesting)
Apple 2x2 G5: $3000
Dell (2x3.06 Xenon): $4000
Mac speed in "real world" application tests, about 2x as fast as the Dell.
Dell = $4000/work unit
Mac G5 = $1500/work unit
The Mac G5 is a much better value on cost on a price/performance basis.
Or were you thinking of something else?
Re:Even if Apple is faster (Score:5, Interesting)
I think this is the one time where Apple hardware hasn't been "horribly" priced. A 64bit dual-2Ghz workstation with SATA HDD, DVD-R, PCI-X, a 1 GHz FSB and a max of 8GB of DDR-RAM for under $3000.
As for software, that comment is just pure ignorance. 50% of the stuff they make is free, and the other software is all competitively priced. What software from Apple is overpriced?
Re:Compiler's should be included (Score:3, Interesting)
Certainly Apple has put more effort into GCC. IBM perhaps, but almost certainly their GCC contributions have primarily been on the x86 and IA64 side of things. In terms of PPC backends, I suspect IBM at the very least has put in much more work into their own compiler.
The real point I'm sure you are trying to get at though is that you suspect that GCC has been optimised much more for PowerPC than x86. Let me assure you that you could not be more wrong. If you recall the days of egcs and pgcc you'll realize that x86 optimisation with gcc has been a strong area of interest for gcc maintainers. Indeed, until VC++7 came out, gcc was one of the fastest x86 compilers out there. The PowerPC backend on the other hand was essentially useless until Apple started working on it. By comparison, IBM's PPC compiler has been used by Big Blue for RS/6000 SPEC benchmarks for over a decade. It also doesn't have gcc's ancient architecture and cross-platform needs, so it is literally light years ahead of gcc.
Saying: "IBM PPC Compiler : gcc PPC backend
Re:Who cares? (Score:2, Interesting)
How much marketing is Spec?! (Score:2, Interesting)
A processor company could write "Spec compilers" with special cases for the Spec code -- and that was half-OK for anything else. Then the code could be run on special test machines. Also, the processor company could be a member of the spec organization so they could influence what is part of the Spec suite -- and also know what will be in the next version, so they could update the specialised compilers in advance...
Intel seemed to be doing all of the above 6-7 years ago.
The main problem was that the Spec suite wasn't available for everyone to test on real systems with compilers that don't unroll loops exactly right for the Spec code, etc...
Some or most of the previous points might not be applicable anymore (e.g. the building of special "spec test machines", probably.) But I really doubt the value of specialised compilers for a test suite -- especially when the test suite isn't free!!
Re:Curious (Score:-1, Interesting)
Figures, given that Slashdot in general panders to Apple. Apple has 10 topics [slashdot.org] devoted to Apple, and Apple products, compared to one for IBM, one for HP, one for SUN, one for Compaq, and one for SGI.
Seem skewed? It probably is. But then, so is the sentiment: "Who cares if they fudge their benchmark reporting-- at least they're not an illegal Monopoly like Microsoft"
So lying and cheating their customers is okay, because they're not Microsoft. Awesome logic.
Amazing, this hatred (Score:5, Interesting)
What mostly surprises me is that so many people feel this desperate burning need to flame computers that are not the same as the ones they have, and operating systems they do not use. Is there a genuine need to diss the PPC 970, when it seems that it is truly -at the very least- in the same performance area as Intels modern CPU's? Why? No one is forcing you, as a x86 Linux, *BSD or Windows user to buy a Mac. Yet you feel the need, now that the CPU is in the same region performance wise to complain about the prices. And again, no one is forcing you to pay those prices or to buy a Mac if you prefer x86 machines.
The x86 machines I have, in one case -the Dell laptop- outperforms my Mac by a healthy margin, yet I find the Dell to have pretty poor workmanship and although I actually find WindowsXP the best Windows version I have ever used, and quite stable to boot, I don't like the way the OS seems to lack a sense of continuity.
I paid more for this Mac than I would ever have paid for a PC laptop of the same performance, but the look, feel and feeling of "good design" is what made me buy this Mac. I don't regret that money at all.
Would I diss x86 if it were slower and more pricy than a similar PPC? No. There are the advantages of larger choice and lower prices that still count and shouldn't be laughed at.
Each to his own.
But envy seems to be a common sin here.
Re:Removed one of the processors for the SPEC CPU (Score:4, Interesting)
I think he mistook "The Xeon" with "our buggy intel implementation"
"reputable" + "Tom's Hardware" = funny (Score:2, Interesting)
Re:But he didn't refute the most damning claim! (Score:3, Interesting)
Are you saying that a computer that costs $2999 isn't cheaper than a $3000 computer? Saying taht tehy're the same price, now that's absolutely ridiculous.
The funniest part to me is that the VP's name is Joswiak. What, did someone genetically engineer an amalgamation of the Steves? (Jobs + Wozniak)/2 = Joswiak??
Re:Other Benchmarks? (Score:5, Interesting)
- Tom's was using Q3A 1.16 instead of 1.32 (PunkBuster code is thought to be a little slower)
- Tom's used set s_initsound 0 to disable sound while Apple noted default settings, which would imply sound was left on.
- Tom's used demo_001 while Apple used demo_4
So I guess it's up to you to decide in the end if the benchmark was fair or no. I don't know enough about the details of Q3A to say whether the differences above are enough to justify a 30% decrease in framerate. It is worth pointing out that Apple's G5 matched Tom's reported framerates for the fastest Athlon XP.The Xeon isn't the competition (Score:5, Interesting)
Re:Who cares? (Score:4, Interesting)
Computers, software, and programming languages are tools, I'll give you that. But they are not single purpose tools like a hammer or screwdriver. A computer can do a multitude of tasks. It's malleable and can do just about anything. Since programming languages drive the computer they also fall into the same category. No matter what computer or what programming language, you have a all powerful system (well, as far as any electronic piece of equipment can be).
Picking the "right tool for the right job" when you're talking computers isn't like deciding whether to use a pair of pliers or axe to cut down a tree. It's like having a box of super tools and each one can do just about anything. Which one do you pick? Well, that answer isn't so easy when just about any of them can do the same tasks just maybe in a different fashion.
I also believe because of the flexibility of computers and specifically programming languages that it is in fact possible to create a more perfect language than anything currently existing. There is no perfect programming language, but there could be.
Sorry if that came out confusing. This only just now hit me. I'll have to organise my thoughts as I think about this some more.
Counterpoint: I switched before switching was cool (Score:5, Interesting)
Since my existing computer was then almost six years old and showing its age, my parents opted to loosen the purse strings and buy me a Macintosh LC. Within fifteen minutes of getting it out of the box and up and running, I knew I was gonna be a Mac user for life (sorry, Apple-haters, but there was no consumption of Kool-Aid involved). Not long after that I got my first look at Windows 3.1, and I couldn't believe what a half-assed Mac knockoff it was. Microsoft has made great strides with Windows over the years, but they still can't touch the synergy between hardware and software that Apple achieves. That synergy means much more to me than raw speed, and I'm more than happy to pay for it.
Therein is the basis for the holy wars, IMHO: The Mac people don't understand why the Windows users are eating dogfood when they could be having filet mignon, and the Windows people don't understand why anyone would choose to pay more for a computer that they perceive as working the pretty much the same as a much cheaper Windows box.
These days, I make my living as a system integrator. I support Windows and Macs, but specialize in Macs-- slightly difficult because my Mac clients seldom need me. I own several Macs and a couple PCs, but my main machines are a G4 and an iBook-- after a long day dealing with Windows (which "just stops working" from time to time), it's damned nice to come home and use my Macs (which "just work"). In my experience, more often than not, people who have really used both OSes for an appreciable amount of time prefer the Mac.
~Philly
Re:Why won't Apple just use the AIX C compiler? (Score:2, Interesting)
Re:Who cares? (Score:5, Interesting)
For some people, e.g. physicists who do numerical "experiments", the benchmarks are crucial, or, at least, a large factor when considering which machine to buy.
Sure, one could buy 10 Linare boxes and Beowulf them together, but if you're a lone physicist with relatively little funding--Beowulf clusters take lots of time, money, and space to feed and maintain--you might care about being able to run floating-point intensive jobs quickly while being able to use MS Word or PowerPoint or some such.
In fact, I already know one astrophysicist who will be getting a G5 in the fall when her new research grant begins. She also happens to be one of the 3 physicists I managed to convince to switch to Mac and get a PowerBook.
Re:Who cares? (Score:3, Interesting)
Re:Everyone should benchmark with GCC (Score:3, Interesting)
Furthermore as a general comment not directly related to mozumber's post, I'd like to point out that this bit about using GCC on both platforms to normalize configuration is just bullshit. Spec reports two numbers, base and peak. The base numbers are what you get with no super-special flags, just the basic simple ones any dodo developer would use. The peak numbers are what you get when the architecture and compiler mad scientists get through with tweaking the build to squeeze every last iota out of the system. Thus using gcc might be analagous to the "base" numbers for the Intel boxes tested on. For the Mac, it may be a different story as others have already pointed out that Apple has put considerable resources into the PowerPC branch of gcc.
It is interesting to note that for most recent Spec submissions on Intel (IA32) platforms, the base and peak numbers only differ by a couple of percent, at the most. Compare that to other platforms and the implication is that getting the utmost performance out of the Intel compiler on Intel hardware is not rocket science, that it defaults to some pretty good settings. Of course, one might take it to mean that Intel has hardcoded Spec into their compiler ala Nvidia's recent fiasco with that 3D benchmark. But, just like Nvidia was caught, Intel would also be caught - competing vendors have people that regularly go over the other guy's results looking for those kinds of games.
Re:Honesty (Score:1, Interesting)
Not anytime soon [macworld.com].
And I quote:
Sorry.As usual, Apple handles this big fuss in style (Score:5, Interesting)
The whole benchmark industry has been created by the like of Intel and Dell for marketing purpose only. I expect the average
Re:Everyone should benchmark with GCC (Score:3, Interesting)
Errmm, but GCC does generate SSE2 instructions. There is a switch -msse2 to enable it. There was no good reason for Apple to have SSE2 disabled, other than to cripple the competition. Notice that they did use G5-specific switches on GCC on their own system.
Re:Who cares? (Score:3, Interesting)
And LaTeX, at least amongst my peers, is on the decline. There are a horrific number of PhD students here typesetting thesis in Word. *shudder*
Re:Who cares? (Score:3, Interesting)
You can say that again... there were 319 PowerMac G5 articles aggregated by Google News yesterday... and the G5 was the 3rd headline on CNN.com on Monday.
http://news.google.com/news?num=30&hl=en&edition=
Re:Who cares? (Score:2, Interesting)
Re:This is really simple (Score:2, Interesting)
Apple has been lagging behind the PCs for years in the performance field. This made Apple's user base frustrated, angry and/or anxious.
Could you please elaborate on this lag because the statement that you gave is very vague. First of all, what is performance? If you think that performance is defined solely by the number of MHz that a chip can crunch out then you better read up on computer architecture. Apple succeed in adopting a new architecture (PowerPC) while Intel and AMD still try to patch x86 fix it inherent flaws from scratch. Also, what about vector processing, low power consumption, vector permute functions that allow rearranging data in the registers? Right from the beginning PowerPC chips outperformed x86 family in terms of floating-point computation and video (d)encoding. What about the Velocity Engine on G4 that allowed data processing in 128-bit chunks? These (and many other) examples show that Apple is not quite *behind* PCs in terms of performance. Thank you.
Re:Even if Apple is faster (Score:1, Interesting)
Sure the minor update (not SP) rate is faster.
9.66 months apart on average (assuming an august release of 10.3), vs. 2+ years for Windows. But the cost over that span is roughly the same. The numbers are as follows:
Mac OS X:
10.0: included w/ Mac (early 2001)
10.1: $20 (upgrade) (late 2001)
10.2: $130 (full) (mid 2002)
10.3: $130 (full) (mid 2003)
10.4: $130 (full) (mid 2004?)
10.5: $130 (full) (mid 2005?)
Total: $540
Windows:
Win ME: included w/ PC (early 2001)
2K Pro: $200 (upgrade) (equivalency upgrade)
XP Pro: $200 (upgrade) (2002)
Longhorn: $200 (upgrade) (2005?)
Total: $600
Notes:
I extrapolated out to OS 10.5 and 'Longhorn' based on estimated release dates for Longhorn and a projected yearly 10.x update (see 10.1 -> 10.2 -> 10.3). So in the course of 5 years, if you keep both machines at the most recent releases, you pay $60 less for OS X updates (despite buying full versions of it and upgrade versions of Windows). If you skip one of the updates, you end up paying $10 more for OS X, but if the wait between 10.3 and 10.4 is longer than the projected year, you might save an additional $130 (total of $190).
Let me also point out that you're more likely to have to buy a new machine to run Longhorn effectively than to run 10.5 effectively.
Re:Apple is a Well Run Business? (Score:3, Interesting)
I have formed the opinion over the years that no company can compete in x86 space while Microsoft has the power it currently has over PC manufacturers. I believe that that's the reason why BeOS, OS/2, PC Geos and DR-DOS died (by the way, I've used all of the former).
"Dell's equipment may not be as slick as Apple's, but it works really well, and they've been selling the hell out of them for years. And like it or not, MS has made good progress with its OSs. I'm no MS fan, but XP is "good enough" for the buying public, just like PCs are good enough."
By crap hardware, I don't mean Dell. I mean $5 network cards from a chain store, $30 taiwanese motherboards etc. I don't give a shit about the aesthetics of the computer I use as long as it has a querty keyboard and the X and Y axis of the mouse isn't inverted, to be honest.
"Your argument about other doomed OSs has no relevance. BeOS died from lack of apps, not because it was based on Intel platform."
I'm not saying there's anything wrong with the x86 platform, in fact I like Dell servers running a Unixy OS, I'm saying that no company can compete on the x86 platform because Microsoft can force them out of the business instantly by using strongarm tactics against the PC manufacturers. I base this opinion on the fact that they have done it every other time a commercial OS competitor has come along trying to swipe the crown jewels from them.
"You sound like yet-another-rabid-Machead. If Apple didn't want to support every peripheral in the world, they could produce an approved hardware list. Microsoft did that for NT, and other OS vendors do that for Linux. And anyway, nowdays it's largely up to the hardware vendors to write the drivers for the OS. All Apple would have to do is review/approve/certify them."
If it was left up to the hardware vendors to supply drivers for linux, you'd have a choice of Framebuffer, nVidia or ATI video cards, 3com network cards, no mice as far as I know, no sound cards, no USB devices etc etc etc. There never was a scanner driver made for BeOS as far as I know...
Hardware manufacturers might support Apple, but their primary focus would always be on windows. It most likely wouldn't improve driver support, but it would increase the number of people who put crap hardware in their computers, and then expect it to work in OS X because it works in Windows.
As it is, Apple get Apple specialised hardware manufacturers because the platform is sufficiently different electronically to prompt the companies into action. I'm convinced that companies that produce PC hardware don't support Linux because they've already done a driver for x86 and don't see why they should do it again.
"Regarding reliability, despite my dislike for MS, I have to say my office development machine (running XP Pro on a Dell P4 workstation) runs flawlessly. I never reboot it, at least not often enough to even remember. It's up for at least 30+ days at a time. Of course, my RedHat 9.0 running on the Dell P4 workstation beside it stays up even longer, but it doesn't see as much day to day use."
That's nice. I have three machines here running windows, used by the rest of the household, two laptops, and one machine slapped together out of parts bought from a nearby computer supplier. The two laptops have hardly any problems, but I did have all manner of problems with expansion cards in the white box.
This is what I'm talking about. The laptops have wireless cards in them, apart from the fuss of getting them set up with WEP (had to resort to 56bit in the end to get it running) they work pretty flawlessly.
I'm arguing this point to you. When you buy a computer from Dell or Gateway or Compaq/HP, you get something that someone has taken a reasonable amount of time to test and make sure the parts all work together well. When you buy a box and slap it together yourself, you don't know whether the problems come from the hardware/drivers or the
Re:It's money that matters. (Score:3, Interesting)
I do agree with almost everything you say, but felt the usual geek urge to correct every micro-error. :-) In particular, I think you're falling into the "I can build a bottom of the barrel system for way cheap!" error.
The eMac does begin at $799, but that version of the eMac has only a CD-ROM (rather than a far superior combo drive), only an 800 MHz G4 (rather than a 1 GHz G4) and an undersized hard disk (40 GB; and I can't believe I just called that undersized but in this day and age...). Yeah, it exists, but the $999 version is clearly the cheapest "feature complete" one. You also really, truly want the swivel/tilt stand, and that's $59 if you have to pay full price. This is not to say that the eMac is a bad machine; I think it's a *superb* value for the price. But that price is more like $1000 than $800.
In a similar vein, the cheapest ($1999) G5 box actually has some subtle differences in specs from the wildly cool and high tech 1.8GHz and dual 2.0GHz box that might be worth mentioning. In particular, the low-end model lacks PCI-X slots, and has a slightly different memory arrangement (both slower, and less expandable). This might still be a good buy for many people, but it isn't just the 1.8GHz model with a slower processor, less memory and a smaller hard disk.
On the other hand, the pricing of these is exquisite marketing. Yes, you could buy the cheap eMac, but you really want the $1000 eMac...and then wouldn't it be nice to have a superdrive, too? Yes, you could buy the $2000 G5 system, but the $2400 model is really where the coolest high tech starts, and then for just $600 more, you can get a machine that is much more than 25% faster...
I think the marketing niches Apple is setting up now are pretty clear. On the desktop, for $1000, you get an eMac. For $2000, you'll get a maxed out iMac (I'm betting these will soon get faster processors). And for $3000, you'll get a G5. Way more rational than some of their older model line-ups.
Re:Why won't Apple just use the AIX C compiler? (Score:4, Interesting)
You can't say hire a linux support person because their salary will be more than made up by the money you save on licensing and hardware over Mac. It just isn't. Even if you could get the chief accountant into a Linux class to take on an OS, is it really a wise use of his time and talents? No.
Macs are about as user friendly to administer as you can get and with 10.3 giving Active Directory integration with a dead simple GUI interface, it's a good choice for companies like this.