Forgot your password?
typodupeerror
Technology (Apple) Businesses Apple Technology

Apple Hardware VP Defends Benchmarks 1081

Posted by pudge
from the i-blame-florida dept.
Greg Joswiak, vice president of hardware product marketing at Apple, in a phone interview today, defended Apple's performance claims for its upcoming Power Mac G5, after they came under fire in the wake of yesterday's announcement. Read on for the details.
Joswiak went over the points in turn, but first said that they set out from the beginning to do a fair and even comparison, which is why they used an independent lab and provided full disclosure of the methods used in the tests, which would be "a silly way to do things" if Apple were intending to be deceptive.

He said Veritest used gcc for both platforms, instead of Intel's compiler, simply because the benchmarks measure two things at the same time: compiler, and hardware. To test the hardware alone, you must normalize the compiler out of the equation -- using the same version and similar settings -- and, if anything, Joswiak said, gcc has been available on the Intel platform for a lot longer and is more optimized for Intel than for PowerPC.

He conceded readily that the Dell numbers would be higher with the Intel compiler, but that the Apple numbers could be higher with a different compiler too.

Joswiak added that in the Intel modifications for the tests, they chose the option that provided higher scores for the Intel machine, not lower. The scores were higher under Linux than under Windows, and in the rate test, the scores were higher with hyperthreading disabled than enabled. He also said they would be happy to do the tests on Windows and with hyperthreading enabled, if people wanted it, as it would only make the G5 look better.

In the G5 modifications, they were made because shipping systems will have those options available. For example, memory read bypass was turned on, for even though it is not on by default in the tested prototypes, it will be on by default for the shipping systems. Software-based prefetching was turned off and a high-performance malloc was used because those options will be available on the shipping systems (Joswiak did not know whether this malloc, which is faster but less memory efficient, will be the default in the shipping systems).

As to not using SSE2, Joswiak said they enabled the correct flags for it, as documented on the gcc web site, so that SSE2 was enabled (the Veritest report lists the options used for each test, which appears to include the appropriate flags).

This discussion has been archived. No new comments can be posted.

Apple Hardware VP Defends Benchmarks

Comments Filter:
  • Who cares? (Score:5, Insightful)

    by ObviousGuy (578567) <ObviousGuy@hotmail.com> on Tuesday June 24, 2003 @07:14PM (#6289536) Homepage Journal
    Really?

    If you want OSX, you'll need to get the PPC.

    If you want Windows, you'll get the x86.

    If you want Linux, you can pick up 10 [slashdot.org] and build yourself a cluster for the price of one of these new machines.
    • Curious (Score:5, Interesting)

      by igabe (594295) on Tuesday June 24, 2003 @07:23PM (#6289628) Homepage
      This is the first time I think _I_ have seen slashdot with an article they wrote compltely on their own.

      Did you recieve a phone call directly or something(Apple calling Slashdot)? If so did they act really aggressive wanting to make sure people don't become anti-G5 before it is even shipped?

      Not too important you might say, but interests me.
      • Re:Curious (Score:5, Informative)

        by pudge (3605) * <.slashdot. .at. .pudge.net.> on Tuesday June 24, 2003 @07:30PM (#6289698) Homepage Journal
        Eh, we do this sometimes, when it is appropriate. In this case, I have a PR contact at Apple who asked me last week if I wanted to talk to someone about WWDC, and we set up a call last weekend, for this afternoon. It just happened to coincide with the benchmark discussion, which Greg was eager to set straight (he had read the arguments and already compiled his responses :-). We also talked a bit about some other topics, but nothing of interest that you haven't read elsewhere.
        • Re:Curious (Score:5, Insightful)

          by Graff (532189) on Tuesday June 24, 2003 @07:42PM (#6289800)
          we do this sometimes, when it is appropriate. In this case, I have a PR contact at Apple who asked me last week if I wanted to talk to someone about WWDC

          You know, I always thought that this would be a good idea for Slashdot. I mean, you guys must have some pretty interesting contacts by now, use some of them to do a "news" article or two on your own. I'd still keep the old Slashdot question/answer interview around because they are interesting and good for the people who don't have time to do a traditional interview.
        • Re:Curious (Score:5, Funny)

          by tbmaddux (145207) * on Tuesday June 24, 2003 @09:20PM (#6290448) Homepage Journal
          (he had read the arguments and already compiled his responses :-)
          Cheater! Dirty cheater, I say!!

          What, did he use GCC to compile them?! Filth!!! DIE!

    • Re:Who cares? (Score:5, Insightful)

      by Xerithane (13482) <(xerithane) (at) (nerdfarm.org)> on Tuesday June 24, 2003 @07:24PM (#6289632) Homepage Journal
      The people who care are the zealots who don't understand, "Use the best tool for the job."

      This means 3 things:
      • Use a tool that is made for the task.
      • Use a tool that you are comfortable with.
      • The other tools don't suck.


      People just have a hard time dealing with this whole "choice" thing.
      • Re:Who cares? (Score:5, Insightful)

        by Pootie Tang (414915) on Tuesday June 24, 2003 @07:46PM (#6289828)
        Personally I think the speed of the G5 and the validity of the benchmarks are both valid questions.

        Does either of those questions alone determine whether you should get a G5 based system or not? No, but that doesn't mean the question isn't worth discussing.

        I'm curious how fast the G5 is at certain kinds of tasks. Not because it helps me make a purchasing decision, but because I'm a geek and I'm interested in that kind of thing. This being slashdot, I'm sure I'm not the only one. Does superior floating point performance mean "better for photoshop"? Maybe not, but I'm more intersted in FP performance that PS performance.

        I thought the original article was worth a read. I thought some of the comments are interesting. I thought this follow up was interesting. People like me are the ones who care. People who just want to know what kind of computer to buy, well yes, they are totally missing the point.
        • Re:Who cares? (Score:4, Insightful)

          by ObviousGuy (578567) <ObviousGuy@hotmail.com> on Tuesday June 24, 2003 @07:55PM (#6289890) Homepage Journal
          The thing I wonder is what the purpose of this kind of company-sponsored benchmark is supposed to prove? Especially in a case like this where the results of the benchmark do not point to a clear winner (what with the questions surrounding the tests).

          Apple may be a hardware company, but it isn't the hardware that is attracting customers. It's the software, stupid. If anything, Apple should be talking up the benefits of the OS and the "Apple System" (where everything works seamlessly) rather than the raw speed of the processor and leaving the benchmarking to review sites.

          Apple's core competence is in making systems that are easy to set up and easy to administer and easy to use. It has never been in making "the fastest machine".
          • Re:Who cares? (Score:5, Insightful)

            by bursch-X (458146) on Tuesday June 24, 2003 @08:39PM (#6290172)
            >Apple's core competence is in making systems that are easy to set up and easy to administer and easy to use.

            And that's not only thanks to the software, but also due to the great integration of software and hardware.

            This integration ("it just works") is why people buy Apple. And therefore it's really hardware and software that attract customers (ey, and don't tell me I didn't buy my 17" PowerBook just for the software, I could have gotten an iBook if I only wanted to run OS X!)
          • Re:Who cares? (Score:5, Informative)

            by RestiffBard (110729) on Tuesday June 24, 2003 @09:23PM (#6290463) Homepage
            you know this and I know this but many trolls don't know this. I think Apple just got tired of hearing how PCs are faster and what not. Personally I was blown away by the keynote. Also, for anyone wondering I'm using the developer preview now and if the release of Panther is anything like the preview, holy crap. It is nice. There are a ton of tiny improvements here and there that really make it nice, even nicer than Jaguar. These are little things that weren't mentioned in the keynote.
        • by DataPath (1111) on Tuesday June 24, 2003 @09:03PM (#6290339)
          There are 3 kinds of lies - lies, damned lies, and benchmarks.

          I think Apple will have validity (in the performance arena) when AMD or Intel start publishing benchmarks against APPLE's systems.
        • by emil (695) on Tuesday June 24, 2003 @10:52PM (#6291007) Homepage

          Let's face it: in their own way, Apple is being quite fair. Everybody in the free software community uses gcc, and publishing SPEC scores on x86 gcc is valid and useful.

          However, IBM probably has C compilers for the POWER architecture that produce far more optimized code than gcc. Why hasn't Apple licensed and ported this technology?

          Apple needn't resell such a C compiler, but critical system binaries (i.e. the kernel) could be recompiled for much better performance. Granted also that IBM is unlikely to support Objective C anytime soon, so such a compiler is only marginally useful.

          However, Apple positively wastes these POWER chips without a vendor-optimized C compiler.

          • by dbrutus (71639) on Tuesday June 24, 2003 @11:44PM (#6291342) Homepage
            Apple's pulling off a miracle every day of the week by staying competitive and often moving ahead of the pack when it has such a small market share. When Apple has 10% of the market, they'll likely have the money to support such a project. But then again, why not just pour the same effort into gcc PPC optimizations? You get the same result (more hardware sales due to faster software) and you get kudos for contributing code.
            • by kriegsman (55737) on Wednesday June 25, 2003 @09:03AM (#6292991) Homepage
              I don't know if they still do, but for a while at the beginning of the PowerPC era Adobe was using the AIX compiler to generate its PowerPC binaries for Photoshop.

              Admittedly, this was when the PowerPC was pretty new, and the choices were the IBM/AIX compiler which was robust and produced fast code but required an AIX box in addition to a Power Mac, or the nacent Metrowerks CodeWarrior compiler which run natively on the Power Mac, but generated poorly optimized code.

              If I recall my history timeline correctly, after CodeWarrior came
              • the Apple MPW "MrC" compiler (better code than CodeWarrior 1.0, but with a wacky command-line "IDE"), then
              • gcc for PowerPC (cruddy code back then), then
              • the Motorola PowerPC compiler (better code than Apple's compiler, with NO IDE - it plugged into the CodeWarrior or MPW IDE).
              • Then Motorola inexplicably stopped selling their compiler.
              • Later Motorola bought Metrowerks.
              • Somewhere along the line, gcc learned to generate better PowerPC code.
              • Eventually, Apple pretty much shelved their "MrC" compiler, and settled on using gcc for Mac OS X
              • Monday, Apple released their "Xcode" environment -- still using gcc, I believe.
              Apple's MPW tools are still available (free) here [apple.com] for Mac OS 7/8/9. The new Mac OS X tools including Xcode are available here [apple.com].

              As a side note, it's really nice to see Apple giving away a full development suite for free, and continuing to put development time and effort into improving it.

              -Mark
          • by Fefe (6964) on Wednesday June 25, 2003 @04:15AM (#6292210) Homepage
            It is quite difficult to produce better code than gcc, and my tests on powerpc (granted, those were a few years back using xlc on RS6000 with AIX 4) showed that xlc produced code of about the same quality -- sometimes worse, sometimes better.

            The gcc "Haifa" scheduler was donated by IBM Haifa, by the way, so I think it's not surprising that gcc produces good code on powerpc.

            On Intel it's quite the same, except that gcc does not vectorize code. From what I have seen, however, icc's vectorizer is not very useful either. I recently tested ogg-vorbis (which is a plain C floating point intensive benchmark) with icc 7 and gcc 3.3 and the gcc version was actually faster than the icc version (on my Athlon XP, target CPU pentium3) despite icc having vectorized several loops.

            So all this "vendor-optimized C compiler" stuff is really besides the point. No C compiler will ever be able to match the quality of hand optimized assembler code, and the most important code (ffmpeg MPEG-2 decoder and MPEG-4 codec) has already been hand-optimized. You might be able to squeeze anoter 5 percent out of your code by using a vendor C compiler with insane optimizer settings, but what good is that if the end user is only going to use gcc anyway. I know I am, so I find the numbers for gcc actually more useful for comparison purposes than some vendor C compiler comparison.

            Also, we don't want to encourage vendors to produce super vendor optimizing compilers, we want them to optimize gcc (so that everyone benefits, not just their users). So the more benchmarks are done using gcc, the better!
            • by Hast (24833) on Wednesday June 25, 2003 @07:55AM (#6292706)
              Interesting, when I took a course in Optimizing Compilers last year the concensus was that GCC is pretty awful when it comes to optimizations. Even general non-architecture dependent optimizations. The lecturers reason behind it was twofold.

              First most research on compilers are being done at big corprorations. IBM being the single largest as I understood it. Naturally they put their optimizations in their own compilers first, the rest of the world have to implement them from their papers. (If they are lucky and the algorithms are not patented.)

              Second if you were to put a good optimization in GCC it wouldn't take long before all other compilers had that optimization as well. GCC is OSS afterall.

              We did comparisons between GCC and SunCC on UltraSPARC. SunCC minimal optimizations (O1) beat GCC with maximum optimizations (O4).

              I'm just finished a course on vectorizing/parallelising compilers. There the situation is that even the best commercial compilers are pretty much equivalent to junk. Implementing the vector algorithms is a lot harder though. Even compared to complex SSA-form optimizations.
        • Re:Who cares? (Score:4, Insightful)

          by ball-lightning (594495) <spi131313@yahoo.com> on Wednesday June 25, 2003 @02:06AM (#6291850)
          This is what makes me dislike benchmarks, they're treated too much like sports (everything has to be fair). We're talking about computers here. We should use the optimized version, of everything. Compile with the Intel compiler, compile with the ibm compiler, I don't care which one wins 'on even ground' I want to see what can go the fastest, period. If the G5 wins with gcc, but if you use inte's compiler and the P4 completely blows it away (or the other way around) then I want to know that, as opposed to thinking something else because 'the benchmark had to be fair'
      • Re:Who cares? (Score:4, Interesting)

        by Cthefuture (665326) on Tuesday June 24, 2003 @10:51PM (#6290996)
        I'm sick and tired of people saying this crap about "use the right tool for the right job". It's like a geek mantra or something. I'm a geek but I don't subscribe to this theory that a computer or software or a programming language is a regular tool to be confused with a hammer or something.

        Computers, software, and programming languages are tools, I'll give you that. But they are not single purpose tools like a hammer or screwdriver. A computer can do a multitude of tasks. It's malleable and can do just about anything. Since programming languages drive the computer they also fall into the same category. No matter what computer or what programming language, you have a all powerful system (well, as far as any electronic piece of equipment can be).

        Picking the "right tool for the right job" when you're talking computers isn't like deciding whether to use a pair of pliers or axe to cut down a tree. It's like having a box of super tools and each one can do just about anything. Which one do you pick? Well, that answer isn't so easy when just about any of them can do the same tasks just maybe in a different fashion.

        I also believe because of the flexibility of computers and specifically programming languages that it is in fact possible to create a more perfect language than anything currently existing. There is no perfect programming language, but there could be.

        Sorry if that came out confusing. This only just now hit me. I'll have to organise my thoughts as I think about this some more.
        • Re:Who cares? (Score:4, Insightful)

          by Daniel Dvorkin (106857) on Tuesday June 24, 2003 @11:37PM (#6291300) Homepage Journal
          I think the key lies in your second-to-last paragraph: "There is no perfect programming language." Yes, exactly -- and moreover, some languages are more nearly perfect (not that I think any existing language is anywhere near perfect) than others, and how close a given language comes to perfection very often depends on the task you're trying to accomplish.

          I used to write image-processing software. I wrote it in C, because writing it in a higher-level language would have been absurd. These days, I write database and Web interfaces, and I use the "P" languages (PHP, Perl, and Python) because writing it in C, while certainly possible, would be a huge pain in the ass. I like all of these languages, but it's indisputable that each of them is the right tool for some tasks but not for others.

          The same is true of computers in general -- processors, architectures, OS's, etc. It would be great if you could set up one system that was clearly better than all others, or even equally good, for all tasks you might want to use it for. But you can't. The difference might not be quite as dramatic as that between pliers and an axe, but it's real.

          I'm very happy with my iBook. It does many things I want to do very very well, and everything else I want to do at least passably. But I'm well aware of its limitations, and chafe at them fairly often. And this would be true of any system -- laptop, desktop, handheld, whatever -- I could possibly buy. I chose it because overall it offered the best fit for what I want to do. If my requirements change, well, then, so will my computer.
    • by robvs68 (560549) on Tuesday June 24, 2003 @11:23PM (#6291215)
      I'm becomming really dissapointed with what seems to be the majority of posts on /. (and especially in this thread). I was anticipating a lot of /.ers going on and on about how sweet the tech specs are on the PowerMac G5 hardware. It shouldn't matter what religion you are (M$, Sun, *NIX, Mac, IdogAppleToSoundSmart...), the hardware freeking rocks! Just like my attitude towards BeOS - I don't necessarily care whether the thing will gain 82% market share, its just cool shit.
    • Re:Who cares? (Score:5, Interesting)

      by drauh (524358) on Wednesday June 25, 2003 @12:26AM (#6291519) Journal

      For some people, e.g. physicists who do numerical "experiments", the benchmarks are crucial, or, at least, a large factor when considering which machine to buy.

      Sure, one could buy 10 Linare boxes and Beowulf them together, but if you're a lone physicist with relatively little funding--Beowulf clusters take lots of time, money, and space to feed and maintain--you might care about being able to run floating-point intensive jobs quickly while being able to use MS Word or PowerPoint or some such.

      In fact, I already know one astrophysicist who will be getting a G5 in the fall when her new research grant begins. She also happens to be one of the 3 physicists I managed to convince to switch to Mac and get a PowerBook.

      • Re:Who cares? (Score:4, Insightful)

        by ziriyab (549710) on Wednesday June 25, 2003 @01:47AM (#6291798)
        Most of the physicists I knew run *nix for their simulations, use LaTeX (not word) to do word processing and DTP, and use pdf files for presentations.

        Not a flame, just a note on how things were when I knew physicists. Now I'm stuck with bio types :) Maybe things have changed and physicists are moving toward macs; I don't claim expertise in this area.

  • Honesty (Score:5, Insightful)

    by dioxn (640015) on Tuesday June 24, 2003 @07:15PM (#6289543)
    At least everything that they did seemed to be amply documented.
    I found that to be refresing especially in light of all the recent benchmark tests that have not been so forthright with all their methods and procedures.
  • by RalphBNumbers (655475) on Tuesday June 24, 2003 @07:18PM (#6289570)
    If everyone benchmarked with open source compilers, there would be none of the shady benchmark-specific optimizations you'd expect to see in proprietary compilers. Everything would be above the table.

    And that's not to mention the benefits for OSS compilers. Imagine the kind of resources and funding processor companies would dump into open source compiler projects if they were going to be the basis for their benchmark scores instead of their closed source proprietary compilers.
    • by mrseigen (518390) on Tuesday June 24, 2003 @07:28PM (#6289672) Homepage Journal
      And that's not to mention the benefits for OSS compilers. Imagine the kind of resources and funding processor companies would dump into open source compiler projects if they were going to be the basis for their benchmark scores instead of their closed source proprietary compilers.

      "So, Bob's been looking at those Intel diagrams for quite some time now."

      "Yeah, I wonder if it's anything to do with his new assistants and Porsche."

      Even the OSS community has a price. ;)
    • by AHumbleOpinion (546848) on Tuesday June 24, 2003 @07:38PM (#6289753) Homepage
      All this talk of gcc removing a variable is naive at best, misinformation if the speaker is knowledgable on the subject. Gcc is not a constant, the quality of it's code optimization varies from platform to platform. To be more specific, gcc is used by Apple to build MacOS X and Apple has been improving gcc PPC code generation. Apple provides gcc to Mac developers. Apple is also IBM's partner in the development of the PPC970. Gcc is the developer optimized compiler for the chip in many ways and is more comparable to Intel's compiler in this respect.

      If everyone benchmarked with open source compilers, there would be none of the shady benchmark-specific optimizations you'd expect to see in proprietary compilers. Everything would be above the table.


      No. Benchmarks would become less realistic. There is nothing wrong with proprietary compilers. If they use proprietary techniques not available to gcc, so what. The only consideration is whether the compiler is available to other developers. The Intel compiler is available under Windows and Linux so it would be completely fair to try it and gcc and pick the faster of the two.
  • by NSParadox (135116) on Tuesday June 24, 2003 @07:18PM (#6289578)
    Why on earth would you want to separate the software from the hardware? This isn't a IBM vs Intel comparison. This is an Apple vs Dell comparison. Apple is selling a platform, not a bunch of PCB boards. I sure as heck won't use GCC to compile SAS or Oracle just before I put up a mission-critical database server...

    • No, in this case, it actually is an IBM vs. Intel comparison. The spec benchmarks only test the performance of the processor.

      However, the IMPORTANT benchmarks are the ones that test the whole system. The stuff up on stage during the keynote is the proof of that, I think. The architecture of the G5 gives it a big win. Getting data to the processor is almost as important as having a fast processor itself.
  • by Anonymous Coward on Tuesday June 24, 2003 @07:19PM (#6289581)
    He's a Corporate Drone(tm) justifying Marketing Speak and Glossy Lit numbers.

    Doesn't everyone realize that this is a black and white issue?

    Corporate Drones == Lies
    Populist Raving == Truth

    Always always always. Doesn't matter what the numbers mean. They threw in that one graph with the single processor machine slower than the Intel just to throw off the hounds. But it didn't work.
  • by SuperKendall (25149) * on Tuesday June 24, 2003 @07:20PM (#6289587)
    I hadn't looked through the detailed report before - one interesting thing was that they physically removed one of the processors for at least one test (SPEC CPU 2000). I seem to remember some people claiming some of the spec tests were unfair when run on a DP system... well there you go.

    It really seems like they tried to do a pretty even evaluation. And again, if the benchmarks were so off then why was the performance on the G5 apps so good? And that was without G5 tuning most likely.
    • by kwerle (39371) <kurt@CircleW.org> on Tuesday June 24, 2003 @07:33PM (#6289713) Homepage Journal
      It really seems like they tried to do a pretty even evaluation. And again, if the benchmarks were so off then why was the performance on the G5 apps so good? And that was without G5 tuning most likely.

      Oh, yeah. Steve probably said "hey, vendors, come on over and do a little demo. Yeah, it'll be a duel, but don't worry about recompiling for the G5 (which is supposed to be trivial). We'll just see what happens."

      Look -- they spent every last minute they could optimizing the builds they used for the demo - don't doubt it for a minute. On the other hand, every last minute probably wasn't all that long, and the demos did kick ass.

      But let's call an Apple an Apple. This was a DEMO. Smoke and mirrors were involved. But I drank the cool-aid; I believe it's faster. Dunno how much, but I don't really care. Mostly I'm just happy it kicks the crap outta the systems they're shipping now.
  • by Capital_Z (682911) on Tuesday June 24, 2003 @07:20PM (#6289590)
    /. had better get with it! We're talking about G5s now and the G4 chip icon is still up in the story post.

    The G4 is so last month.

  • Does this mean.. (Score:4, Insightful)

    by tarquin_fim_bim (649994) on Tuesday June 24, 2003 @07:23PM (#6289622)
    that all software vendors have to be honest now, or just Apple?
  • Other Benchmarks? (Score:5, Insightful)

    by WatertonMan (550706) on Tuesday June 24, 2003 @07:24PM (#6289634)
    Unfortunately the more egregious benchmark was the Quake benchmark. I'd have liked to have heard about that one. Th

    Further I notice he didn't mention the problem of not doing comparisions to AMD.

    While I can understand his reasoning, the fact is that most software on the PC runs under VC or Intel's compiler. It doesn't run under gcc. The benchmark might be a fair Linux/OSX comparison but implies something about Windows/OSX that is incorrect.

    I'd also like to see the tests done under Mathematica and Photoshop discussed more. Apple's had a history with photoshop so there is prima facie reasons to distrust it. But the Mathematica test, which seemed the most exciting to me, is what I'd really like to see.

    Realistically though the tools for Apple, including graphics drivers, are all very beta. So we should see improvements with time. And realistically benchmarks are typically kind of deceiving as an indicator of real world performance.

    So any word on these other questions?

    PS - I love OSX and would love to make a Mac my primary machine. If only Project Builder was up to the task so I could abandon Visual Studio. But I am excited about the G5, but I think Apple's "questionable" tactics have brought a lot of unfavorable press that more honesty would have avoided. Personally I think being within 10% - 15% of the top end PC would have been fine.

    • Re:Other Benchmarks? (Score:5, Interesting)

      by tbmaddux (145207) * on Tuesday June 24, 2003 @09:17PM (#6290425) Homepage Journal
      Unfortunately the more egregious benchmark was the Quake benchmark.
      Are you talking about this one, [apple.com] where Apple posts 337fps at 1024x768/32bpp for the G5 and 275fps for a P4? I asked about that on another forum, [xlr8yourmac.com] noting that Tom's Hardware gets over 400fps [tomshardware.com] from a P4/3GHz, and one respondent noted that
      1. Tom's was using Q3A 1.16 instead of 1.32 (PunkBuster code is thought to be a little slower)
      2. Tom's used set s_initsound 0 to disable sound while Apple noted default settings, which would imply sound was left on.
      3. Tom's used demo_001 while Apple used demo_4
      So I guess it's up to you to decide in the end if the benchmark was fair or no. I don't know enough about the details of Q3A to say whether the differences above are enough to justify a 30% decrease in framerate. It is worth pointing out that Apple's G5 matched Tom's reported framerates for the fastest Athlon XP.
      • Re:Other Benchmarks? (Score:4, Informative)

        by Anonymous Coward on Tuesday June 24, 2003 @11:31PM (#6291261)
        The Apple Quake 3 benchmarks disabled hardware acceleration. They were solely testing the CPU, or trying to at least. The guy from... oh, damn, what's that guy's name, the guy who did the OpenGL demo yesterday. He did the same thing. They did all the rendering they could in the CPU.

        That's why the numbers were low compared to other tests that used accelerated graphics.
  • Honesty (Score:4, Insightful)

    by r84x (650348) <r84xNO@SPAMyahoo.com> on Tuesday June 24, 2003 @07:25PM (#6289639) Homepage Journal
    It seems to me that this rep from Apple, unless I am very naive, is being very candid and honest with us. It seems that, by showing us the complete specs on the benchmarking, they are doing what they claim to be doing. Thinking differently, and giving us (for 3 grand) an honestly faster machine. I appreciate the prompt frank response from Apple on this controversy. I am typing this on a PC, simply because I could build it myself for less money than I could buy a nice Apple. Ah, the life of a poor student...
  • by General_Corto (152906) on Tuesday June 24, 2003 @07:25PM (#6289644)
    There have been a few things that I haven't seen Apple pull out of the hat in the last few years:
    • a revolution in hardware platform (not since G4 launch);
    • SPEC benchmark results;
    • a fast response to potentially damaging remarks

    Okay, so Apple needs the G5 to succeed in order to survive. Motorola just aren't sending out the chip upgrades fast enough. They (Motorola) have enough other problems in their wide range of markets that they're in that not having to worry about CPU competition is probably a good thing as far as they're concerned.

    The fact that the (almost) top person at Apple has made this clarification shows how much importance they're putting against these claims. Given that nobody else has had a chance to verify yet, and people are making wild speculations based off of paper and a lack of understanding, it's probably just as well that they're putting a positive spin on things.

    Maybe the documents should have been clearer, showing why these configuration decisions were taken.

    The "we had to use GCC" argument is a little strange though; is there any other good compiler available for the PPC at the moment? if so, I'd like to know; I use macs myself! :)
  • by digital photo (635872) on Tuesday June 24, 2003 @07:27PM (#6289661) Homepage Journal

    I have to say, this puts things in an interesting light.

    Does a company, in trying to be fair as it seems in this case, get penalized for choosing the best optimization and not testing with the worst optimizations(as per their views)?

    In looking at other sites like Tom's Hardware and Anantech, I think the answer is simple: Show all of the results, both the good and the bad. That way, it removes the spectre of doubt in peoples' minds that fairness wasn't present during testing.

    Personally, I don't have the funds to get a G5 based system. It just isn't in the budget. But then again, the only reason I would buy a G5 system over an x86(Opteron or P4) would be to run Mac OSX. :)

    I'm guessing that tests will be conducted by various groups over the next few days to either validate or invalidate the tests. Sounds alike like that whole MS/cost analysis/web server speed fiasco all over again.

    Despite the tests, for Mac users who wish to stick with Mac OS X, the G5s are as fast as they come.

  • by weave (48069) on Tuesday June 24, 2003 @07:29PM (#6289686) Journal
    He made so many errors in his "debunking" yet so many people took it for gospel.

    Like, the switch -mfpmath=sse when used in a P4 *does* use SSE2, but this guy thought just cause the switch flag says sse that it must be SSE only.

    Then someone else (can't find the post, on usenet, under the mac advocacy group) pointed out that Dell's SPEC tests also disabled hyperthreading [specbench.org].

    Then, based on this person's web page who no one even knows who he is, they start drawing conclusions that if Apple faked these (based on his flawed analysis), that they also must have faked those Adobe, Mathmatica, and other demos -- despite the execs for those companies being on stage also confirming the results.

    Gotta love the net...

    As for me, I don't know what to believe. I'm just going to patiently wait until some reputable sites spend a lot of time and do an in depth analysis and their own benchmarks, like Tom's Hardware for example. Then I may start drawing my own conclusions.

    As for me, all I want is to be able to encode mpeg video at something greater than real time. Show me *that* benchmark please!

    • by TrekkieGod (627867) on Tuesday June 24, 2003 @08:49PM (#6290246) Homepage Journal
      I'm gonna be modded as a troll on this one, but I gotta do it, because it will happen to me :)

      ::bangs head on keyboard for giving moderator ideas::

      ::Slashdot story: Tom's Hardware benchmarks the G5, and compares it to dual Xeons, dual Opterons, and (I guess), the P4::

      Me: "Woohoo...I'll finally found out which is better" ::clicks link::

      "Page 1: We have tested all these systems, and you will soon see our results." ::scroll down through ads, click next::

      "Page 2: These tables show the systems we have tested on" ::scroll down, next::

      "Page 3: Tables, cont.." ::yells out profanities, looks on table of contents, chooses "benchmark results"::

      "Page 45: And now, let us take a look how the G5 stands against the current x86 and AMD64 processors" ::AAAAAAAAHHHHHH...can't stand it anymore, clicks on conclusion::

      "Page 666: And thus, we conclude that the G5 is better in some ways and worse in others" ::NOOOOOOO...Now I'll never know!!!::

    • by sakusha (441986) on Tuesday June 24, 2003 @10:34PM (#6290908)
      As for me, all I want is to be able to encode mpeg video at something greater than real time. Show me *that* benchmark please.

      Man are you way behind the times. I can do that even with my dual 1Ghz G4.
  • by WndrBr3d (219963) on Tuesday June 24, 2003 @07:31PM (#6289704) Homepage Journal
    If my car has 200HP at 6,800RPM on the sticker, I usually donâ(TM)t take the stickers word for it, but trust that I would get around those numbers on average.

    There are those people who want to know if those numbers are EXACT 101% of the time, so they go bust out their dynamometer and begin writing complaint letters when their engine only hits 195HP.

    I think benchmarks these days are no longer a science that they used to be. There are far, far, FAR too many hardware and software variables to do an accurate cross-platform analysis and comparison.

    I mean, is it really logical to compare Apples (har har har) to Oranges? I mean, most all applications that will be running on the G5 will be optimized for the G5. So does it matter how a 'comparable' application will run on x86? No, because the x86 Application might have a few more optimizations which would make the comparison pointless.

    These days people should take benchmarks with a grain of salt. Just another selling point they'll put on the big list of bulleted marketing jargon on the back of the box to try and rope in first time buyers who are turned on by big acronyms and high-tech sounding words.

    So yeah, I think people just need to cool their heels and take this for what it is, just marketing propaganda. Does QuantiSpeed really make your CDs burn faster? [enusbaum.com] No. Does the P4 make âthe internetâ(TM) faster? [intel.com] No. Just take it for what it is and let it go.
  • Benchmarking (Score:5, Insightful)

    by bm_luethke (253362) <luethkeb@comcastOPENBSD.net minus bsd> on Tuesday June 24, 2003 @07:33PM (#6289716)
    The Apple guy is both correct and wrong.

    Correct in the sense that he wasn't necessarily being unfair. I don't think Apple was raelly cooking the books here. OTOH benchamarking is quite difficult.

    No, it would not be fair to compare intel compilers to gcc compilers. But what about, say, another non-hardware tied compiler? Look at it this way - 3dmark scores on graphics cards. Theoretically it should give a good impression on thier relative hardware - but we all know that it doesn't necessarily. It may do something bad on one system, great on another, one system may cheat and have special code to work better with that particular test.

    Same here. Ideally you would find many benchmarks, not just gcc, but both with all optimizations on, with all off, both with the best compilers, worst compilers, and middle of the road. You also need memory intensive, processor intensive, grpahics intensive, floating point, integer, and many others to get the full idea and compare it to what you need to do with the computer. For many of the crowd that worries over this stuff overclocking can become an issue also.

    This is why benchmarking is as much art as science. I care about all those numbers - I have code compiled specifically for my athlon-mp's, some generic, and some optimized for p4's for the consumer tasks. On our computation cluster we use specialised compilers. I care how it runs on all of it for real world use. But no hardware manufacturer does those extensive of a tests - they pick the best of the ones they can claim "fair" on usually.

    And lastly, in the end, who cares? Unless you are regularly running 4 hour jobs from a console it is irrelevent. It is more important that you are productive with the interface and that is personal choice. Few consumer tasks (and even programming tasks) require that power - and the stuff that does is generally handled by specialised hardware. Then if they have the fastests today they won't tomorrow.
  • by Chad E Dirks (681955) on Tuesday June 24, 2003 @07:33PM (#6289720)
    Is there good reason to believe that the same compiler will produce relatively as well-performing of code for one chip it supports as it does for another? I don't think so.

    In this case, performance will in part be a function of how mature and optimized the generation of code for the advantages of that particular chip is.

    Because there is no guaruntee at all of fairness by using gcc for both processors, except of course if we had the expert opinion of someone intimately familiar with gcc's code generation for both processors, using gcc for both processors would seem to be little more than a marketing tactic to give the appearance of fairness and credibility.

    It seems to me that a better test is to take the best compiler widely available for each chip, and then run your tests with the produced code. Now, this isn't necessarily real world application testing, but that isn't what we are necessarily looking for here.

    How well the processor performs with code generated by the best generally available compiler, is, apart from extraordinary measures, the best prediction we have of how generally the processors will compare for any given well-written, production quality code.
  • by coolmacdude (640605) on Tuesday June 24, 2003 @07:36PM (#6289739) Homepage Journal
    As someone submitted in the last story, hyperthreading and those other options does not always mean a performance increase in every situation. I am glad to see that Apple responded to clear up the confusion. I had suspected something like this was really the case when the trolls came out looking to bash [slashdot.org].

    What seems to be missing in all of this is the big picture. Whether or not the G5 is 1.2% faster or slower than the Xeon/P4/Opteron is not a uniform answer. Different apps are going to perform differently on different platforms. Not only that, but there are a million possible variations of benchmarks that could make both sides the winner. Like Greg said in the interview, if Apple was looking to cheat they wouldn't have hired an independent company and provided full disclosure.

    Processor speed notwithstanding, most Mac users are so because of Apple's OS not their hardware. Windows would slow me down much more than 6 extra cycles of processor speed. For my circumstances, the fact that Apple now has hardware fast enough that it can even attempt to make the 'fastest' claim is far more important.
  • by trouser (149900) on Tuesday June 24, 2003 @07:38PM (#6289747) Journal
    .....can be trusted 100%. Only Apple would exaggerate for marketing purposes.

    I have this theory. A 2Ghz twin G5 system is really fast. And if you have some money to spend and you want a really fast system and you'd like to run OSX then you could do worse than buy one.

  • Joswiak (Score:5, Funny)

    by Jon Abbott (723) on Tuesday June 24, 2003 @07:40PM (#6289785) Homepage
    Every time I hear the name "Joswiak", I keep thinking the guy is some hybrid between Steve Jobs and Steve Wozniak -- some kind of mutant creature straight out of the R&D Labs at Apple. :^)
  • Parity by using gcc (Score:4, Interesting)

    by exp(pi*sqrt(163)) (613870) on Tuesday June 24, 2003 @07:48PM (#6289847) Journal
    He gives the illusion of parity by using the same compiler on both platforms. But the back ends to these compilers are different pieces of code written by different people. There is no parity.
  • by faust2097 (137829) on Tuesday June 24, 2003 @07:54PM (#6289879)
    Somehow this shifty Apple exec ignored the boldest claim of the bunch:
    Misleading Prices

    Both Apple and Dell are guilty of using misleading prices. For example, Apple gives the price of the low-end G5 as "$1999", and the high-end G5 as "$2999". In other words, they have subtracted $1 from a $3000 computer to make it seem cheaper, which is absolutely ridiculous. This demonstrates that both Apple and Dell are willing to mislead people when stating their prices.

    What do you have to say now mister Joswiak if that is in fact your real name?
    • by doce (31638) on Tuesday June 24, 2003 @08:01PM (#6289933) Homepage
      this is the funniest claim i've seen in a while. not only does apple do this, so does dell, and so does virtually every consumer-oriented company on the planet. gas companies shave a TENTH of a PENNY off gas prices to make them seem cheaper.

      a department store (was it macy's?) started this practice. the funny part is that the aim wasn't really to fool consumers into thinking it was less expensive. alas, the real purpose was to force cashiers to open the register, since the customer was almost always going to be due some change.
  • by vought (160908) on Tuesday June 24, 2003 @08:01PM (#6289930)
    The "status quo" crowd that jumped all over Apple this morning for the "fake" benchmarks and "dishonest" wording will still find lots of reasons and ways to disparage the fruit company, simply because Apple isn't doing what they want - building the best, fastest, and most cutting-edge computers for $400.00.

    Forget about Serial ATA - (Apple is the first top-tier manufacturer to make this interface stardard across their high-end machines.)
    Forget about the new motherboard featuring HyperTransport, PCI-X, and the IBM-fabbed 1GHz northbridge chip. Oh, and 802.11G, USB 2.0, FireWire 400 and 800, and Bluetooth, too.
    Forget about the imagination and creativity that goes into making a project like this go from concept to reality in eighteen months.

    Why support a company like that? Bunch of dirty liars - there's no way a 2GHz chip could be faster than my Intel/AMD/whatever86!

    Maybe it's not ultimately faster (although Greg's comments seem to indicate that the playing field was pretty equal). I don't buy "fast". I buy well-integrated tools that help me get work done, and in turn, bill clients. So I (still) use a Mac.

    Jeez - to hear people around here, you'd think that innovation, style, performance, and the courage to move forward agressively and definitively with new technologies doesn't come at a price.

    What other comapny would develop all these technologies to hardware and software maturity as part of a new hardware platform, then bring it all to market with system software already written (by the same vendor, I might add) to take advantage of new hardware features?

    Those things DO come at a price. The price begins at $1999.00 for the 1.6GHz G5, or $799.00 for an eMac.

    As long as there are people who just want to get work done on their computers without hiring an IT department or worrying about who is responsible for which component of the system, Apple will still be around.

    I bill around eight hours a day with my Macintosh - the $400.00 price premium over PC hardware at the time I bought my G4/800 simply isn't an issue - over the lifetime of the machine, I'll probably bill at least two hundred times that amount for work made possible by its existence.

    That $400.00 up-front cost means that I don't have to spend my time - my extremely expensive and finite time - having to deal with at least two vendors just to get a system with competitive hardware, a competitive OS, and support for them both. If your time isn't valuable, by all means cheap out and build your oft-touted (and perfectly capable) PC from parts you buy at Frys. $400.00 means nothing to professionals - it's cheap support insurance.

    I hope Apple sells a TON of these machines - because they're practically the only personal computer company willing to take the initiative and responsibility for supporting hardware and operating system on equal terms.

    Perhaps if Apple stressed the cost of ownership point to more people, they'd have higher sales. Our small business has nearly thirty Macs. I'm the lone IT person, spending an entire hour a week on supporting a bunch of artists and their Macs. What similarly-sized Windows-based business can make that claim?
    • by Anonymous Coward on Tuesday June 24, 2003 @08:31PM (#6290118)
      Hell, most of the trolls on slashdot want it for free, or it's crap.

      For example:
      Troll: "If I had a way to download a MP3 for a very small payment, I wouldn't steal them from KaZaA anymore"

      www.emusic.com, the Apple Itunes Music Store...

      Troll: "Emusic won't let me mirror the whole site, so I won't use it! Itunes doesn't use OGG, so I won't use it! So I'll still use KaZaA"

      Troll: "When they make OS X that runs on Intel/AMD hardware, then I'll buy it!"

      But you didn't buy your Windows XP either...

      Troll: "Software should be Free!"

      Same shit, different pile.
  • by slyfox (100931) on Tuesday June 24, 2003 @08:02PM (#6289943)
    Apples claims seem quite reasonable to me. Why? Look at the other reported SPEC scores for Power4+ (the G5/970 is based directly on IBM's Power4+ processor core). Right now the Power4 ranks well on both SPECint2000 and SPECfp2000. See the SPECfp2000 [ideasinternational.com] and SPECint2000 [ideasinternational.com] benchmark report summaries.

    SPECfp: The Power4+ at 1.7 Ghz has the highest SPECfp score (1699 @ 1.7Ghz); higher than Itanium (1431 @ 1Ghz), the most recent Alpha (1482 @ 1.15Ghz), and the Pentium 4 (1229 @ 3.0Ghz).

    SPECint: As far as SPECint, the Power4 is not in the lead (1113 @ 1.7Ghz), but is still respectable when compared to Pentium4's (1200 @ 3.0Ghz).

    The G5/970 should do similarly or better than the G5/970 (since the G5/970 is running at 2.0Ghz vs Power4+ 1.7Ghz). One caveat is that the G5/970 has a smaller on-chip second-level cache (512kB vs 1.5MB), which will hurt its performance on some codes.

    Certainly Apple's test uses a drastically different compiler than the reported SPEC results. This results in absolute numbers that are lower, but Apple's relative comparison is still reasonable, IMHO. I think it is safe to claim that Apple has really closed the gap in processor speed and now has processors with comparable performance to the fastest chips money can buy. About damn time. :)

  • While surprising and most certainly refreshing to see that apple is serious about their claims, serious enough to publicly rebuke the claims almost certainly first brought to the big light by /.'s earlier article, this may only be leading into a circle of prooving and disprooving.

    I believe it would be best for apple to answer with a full fury of tests to truly show the full range of operating prowness of the G5's vs the P4's, etc.... at least initialy, and to from there LEAVE IT ALONE. Cause no matter how many tests they do, no matter how much proof ... there will always be people out there ready to bring flames over nothing.... For instance this guys claims that FP isn't all that important, and that int tests are basicly all that matters for the majority of users.

    He and others will stick too their guns even if they have only a couple benchmarks to cite as being supirior (kinda like the G4's and their altivec/photoshop optimizations of yester-year).

    Apple needs to make sure that they have a clean image of being flatly open on their claims, and then to move on without being bogged down in an obvious quagmire of platform evangalism. The truth is, their strongest advantage remains the OS and not their hardware's direct horse-power. Of course the G5 along with all the goodies they come with are incredibly great, but this isn't apple's mainstay... it's simply another selling point.

    If they become entrapped in having to proove themselves through benchmarking every new release, it won't be long before their entire image would have to live up to being ahead at all costs.... and guess what... they ARE going to fall behind again.... and then they'll leap ahead again.... and then they'll fall behind... etc.... And every down cycle will be worse, since the specs will be much more associated with their image.

    keep your strong point in innovation apple, and if youve got the great hardware... great.... but don't get stuck in the mind-less mhz/spec race that has stagnated computer innovation for the love of ego's.

    just my 2 cents.... I develop ASP, and love win 2k adv srv, ill never use anything but unix/linux for my networking gear, and OSX keeps me damn happy when i want to do anything not mind-numbing. Cisco IOS is arcane but makes me feel good. I am biased towards all platforms.

  • by enderwig (261458) on Tuesday June 24, 2003 @08:31PM (#6290117)
    So many people are slamming Apple for posting biased benchmarks. Yet, I found it very interesting that Apple posted one benchmark which showed the G5 being the slowest machine: the SPECin_base2000, single processor [apple.com] mark. For someone posting completely untrue and biased benchmarks, showing a last place finish shows that not everything was biased in favor of the G5.

    Is the PowerMac G5 the "world's fastest personal computer"? Probably not, but it may be the first 64-bit personal computer to ship to the masses (ie. bought in a store like CompUSA). I wonder if AMD will move up the Opteron release now or if Intel will drop the price on their Itanium. If so, then people who want 64-bit x86-compat CPU's should thank Apple for bringing them their CPU's faster. =)

    Anthony
  • Amazing, this hatred (Score:5, Interesting)

    by theolein (316044) on Tuesday June 24, 2003 @08:46PM (#6290226) Journal
    I use a 667MHz Powerbook with a G4 that is nowhere near as fast as modern P4's and the PPC 970, but I love this machine and I love this OS. I also have a Dell laptop with a 2GHz P4 and WinXP and an old 450MHz K6-2 that runs Debian. I use all of them but mostly prefer this Mac and OSX.

    What mostly surprises me is that so many people feel this desperate burning need to flame computers that are not the same as the ones they have, and operating systems they do not use. Is there a genuine need to diss the PPC 970, when it seems that it is truly -at the very least- in the same performance area as Intels modern CPU's? Why? No one is forcing you, as a x86 Linux, *BSD or Windows user to buy a Mac. Yet you feel the need, now that the CPU is in the same region performance wise to complain about the prices. And again, no one is forcing you to pay those prices or to buy a Mac if you prefer x86 machines.

    The x86 machines I have, in one case -the Dell laptop- outperforms my Mac by a healthy margin, yet I find the Dell to have pretty poor workmanship and although I actually find WindowsXP the best Windows version I have ever used, and quite stable to boot, I don't like the way the OS seems to lack a sense of continuity.

    I paid more for this Mac than I would ever have paid for a PC laptop of the same performance, but the look, feel and feeling of "good design" is what made me buy this Mac. I don't regret that money at all.

    Would I diss x86 if it were slower and more pricy than a similar PPC? No. There are the advantages of larger choice and lower prices that still count and shouldn't be laughed at.

    Each to his own.

    But envy seems to be a common sin here.
  • My turn to bitch! (Score:5, Insightful)

    by presearch (214913) * on Tuesday June 24, 2003 @09:08PM (#6290372)
    I seems that some people just like to bitch.
    I know I do, and that's why I'm writing this.

    I can't figure out why so many people post to threads like this and
    bash Apple, while saying that they would never buy a machine from them
    anyway. What's the point in that? Would the industry be better off if Apple
    didn't exist? Would you finally be happy if everyone went out of business
    except for Dell, only selling boxes pre-loaded with Linux, for $299?
    If that was true, Lindows should be ./'s favorite vendor.

    And those that say that they could build a machine themselves for way
    less than a Mac, if Apple had a build it yourself, parts in a bag option for
    $500 less, then people would still bitch that for that price, it should come
    fully assembled.

    Although yes, I am a "Mac guy" (but I've got Windows, Linux, Solaris, IRIX,
    NeXT and a few other boxes on my home network), regardless of my
    prejudice for the platform, you have to acknowledge what a beautiful
    $3000 machine the G5 is. Clean inside and out, plenty, plenty fast for
    the years that you'll have it in service, arguably a better OS than any
    Linux variant and absolutely better planned out and cleanly feature
    rich (and economical) than any Windows release. I was doing some
    admin work on Win 2000 server today, what a disorganized, steaming
    plie that thing is. Some say it's superior, I think it might be the absolutely
    worst collection of software ever crammed into one box. Pheeeewwww!

    But I digress. I have come here to praise the Power Mac G5....

    One of my favorite things about the G5 (and I know that non-Mac users
    think than Apple just makes pretty boxes), is indeed, the pretty box.

    J. Ive did such a restrained design. So clean and minimal.
    There's a guy with rare discipline and insight.

    The new design language, aluminum and circular hole accents, also
    seen in the iSight and hints of it in the line of new aluminum PowerBooks,
    in my opinion is the best we've seen in the 2nd Jobs era at Apple.

    I liked the clean white, crystal and chrome designs of the G4 iMac and the
    iPod but this new design language is going to make for some other very
    exciting products. The new display line will be beautiful, wrapped in a
    thin sheath of aluminum. Will a future iPod have the look of a large-ish
    Zippo lighter? What would an all-aluminum G5 iMac look like?

    I'm just glad that Apple's still here, still thinking different, and still making
    insanely great products.

    Dell? HPQ? Gateway? Lindows? Sony? (Well, Sony's trying).
    The parts bin at Frys? That little shop in the strip mall that sells cases and
    motherboards? For the most part, all of that is commodity crap. Even if
    you throw on your free homemade Linux on it, it's half-assed at best,
    even after hours of effort.

    Apple is the only computer company left that's doing anything that really matters.
    Like it or not.
  • by DA_MAN_DA_MYTH (182037) on Tuesday June 24, 2003 @09:14PM (#6290404) Homepage Journal
    Apple G5 running Mac OS X:

    ###(My little bench mark bar graph)###

    Dell Intel Xeon running Mac OS X

    #

    Mac OS X runs infinitely faster on the G5 than the Dell Intel XEON. Focus on that.

    The G5 blows the G4 outta the water, so I really don't care how it performs to the Intel XEON.

    Processor speeds aren't going to make people 'switch'. It's the User Experience / WTF can I do with this computer now? (Meaning does it run the apps I need it to run?)

    I think it was Panther that stole the show for Apple, not the G5. That is an awesome OS, just the fast user switching alone sells it for me.

  • by Performer Guy (69820) on Tuesday June 24, 2003 @09:14PM (#6290412)
    This seems reasonable of Apple now. There are many applications compiled on Windows that don't use Intel's optimizing compiler. Indeed that's the norm, since most Windows developers use Microsoft's compilers that ship with Visual Studio and other x86 environments like Linux are dominated by gcc development. You have to buy Intel's compiler separately and add it to your development environment in most circumstances and it ain't cheap despite the obvious benefit getting better x86 optimized apps released has for Intel. The biggest difference AFAIK is Intel's good work in optimizing for their SIMD style instructions like SSE2, where their compiler does a much better job at parallelizing multiple serial operations into a single SSE op. The difference this makes to some code when comparing Intel's compiler to Microsoft's compiler on the same CPU can be dramatic, even 2X or more on specific benchmarks.

    All in all I think this was a fair test of these CPUs, it was a level playing field. OTOH we know Intel can do much better with their compiler, but only some developers use their compiler. It would be interesting to see just how much of a benefit Apple could squeeze out of non gcc compilers, probably not as much as Intel, perhaps not anything, it depends on the work they or IBM et.al. have done on their compilers. You just know if it was to Apples advantage they'd have compiled with their best compiler and dont teh comparrison with those numbers vs Intel's so this situation has been contrived to an extent.

    With Intel charging what they do for their compiler developers can be reluctant to pay extra for it, I expect almost everyone (on Windows) would use it if it were free. I know I would, but I can get by without it. I don't really have much sympathy for Intel here, they make billions of chipe, make significant performance claims based on their own compiler, yet charge for it to the point where many developers simply stick with Microsoft's compiler that they've already spend a fair bit on. Now Intel is upset that Apple used gcc, well more people might use Intel's compiler if it were easier to aquire, and clearly it would benefit Intel. If they want to run there business where everything is a profit center and they don't have to be smart enough to evaluate obvious but intangible benefits that's their business, but this is part of the price you pay for charging an arm and a leg for your compiler when you should be in the hardware business and giving your compiler away to help your customer gain the benefit of faster code from the applications they purchase. In the meantime specbench numbers for Intel are simply bogus for many applications.
  • by mnemonic_ (164550) <jamec AT umich DOT edu> on Tuesday June 24, 2003 @09:30PM (#6290510) Homepage Journal
    The G5 will be available on September 1st. The Athlon64 will be available in the same month. With both processors purported to bring 64 bit to the desktop, it would seem the Athlon 64 would be more appropriate to compete with rather than the Xeon.
  • by shylock0 (561559) on Tuesday June 24, 2003 @10:47PM (#6290975)
    Regardless of this hype about the SPECint and SPECfp score, I'm much more interested in real-world performance -- specifically the Photoshop tests given closer to the bottom of the page. It's real application support that counts.

    I'm the head of a mid-sized consulting company that deals almost exclusively with digital media and digital arts firms. We have a few G5s on order, and because we're a solutions provider, we'll probably get them pretty early. I'm going to wait and see exactly how fast they are, not just in Photoshop, but also in Final Cut Pro -- which in my experience has a history of outrunning similar applications on faster hardware. It's going to be real-world performance that matters. Not SPECfp scores. And we won't know the real-world performance until people start getting their hands on some production units. End of story.

  • by Lulu of the Lotus-Ea (3441) <mertz@gnosis.cx> on Tuesday June 24, 2003 @10:47PM (#6290978) Homepage
    Several other posters have noted that GCC/970 is really not the same compiler as GCC/Xeon. Sure there may be a bit of code in common between the versions, but the job of a compiler is to produce object code... and by definition, the object code for 970 is different from that for Xeon.

    What matters to a purchaser is "How much performance can *I* get out of this machine". If I am performing CPU-intensive scientific calculation that require the fastest CPU I can find (at least for a given number of kilodollars), I'll almost certainly spring a few hundred extra for the compiler that produces the fasted object code on that platform (if needed, there's nothing ruling out GCC automatically because it's free).

    It happens that for a Xeon or P4 (or Opteron, for that matter), the compiler that produces the fastest object code is ICC. Intel has done an amazingly good job with their compiler.

    Now, sure, I *could* get a similarly optimized 970 compiler for comparison.... if one existed, that is. It looks like right now, GCC is the best you can get on a 970. It doesn't do a buyer any good to know that IN PRINCIPLE a more optimized compiler could be written.

    All that said, the 970 looks like a very respectable chip. And Apple is selling their new machines at a very competitive price; and Macs have extremely friendly and stable OSs. All that means that it is probably well worth buying a PowerMac even if it will crunch big computations a few percent slower than a more expensive Xeon. But still... the "GCC is the common element stuff is pretty darn bogus."
  • by failedlogic (627314) on Wednesday June 25, 2003 @12:04AM (#6291442)
    I don't see why people are even debating this so early on. Come August, when home and commercial users get their hands on these systems, I'm sure we'll see more than enough benchmarks: Photoshop, 3d animation programs, Quake and other games from so many sources our heads will be spinning. We can then all witness which system comes out on top overall.
  • More on benchmarking (Score:4, Informative)

    by kajod_kaka (682015) on Wednesday June 25, 2003 @12:52AM (#6291613)
    The 30th International Symposium on Computer Architecture had an interesting panel discussion on benchmarking in industry and academia, with people like John Hennessy, Dave Patterson and Gurinder Sohi on stage. The conclusions: most benchmarking in industry, especially SPEC, is a pack of lies. And benchmark results published by academic researchers aren't much better. So, not really much point in losing a lot of sleep at least over their SPEC numbers.
  • by afantee (562443) on Wednesday June 25, 2003 @04:23AM (#6292231)
    They have used a third party with full disclosure, selected options (SSE2, no hyper threading, Linux instead of Windows) to give higher scores to the Dell. How fair is that?

    The whole benchmark industry has been created by the like of Intel and Dell for marketing purpose only. I expect the average /. readers are more sophisticated than just focusing on the manufactured numbers. There are far more important factors to consider, and the G5 is 200 - 700% faster than the Dell in running real-world apps like PhotoShop, Logic, Mathematica, BLAST, HMMer, etc.
  • by amichalo (132545) on Wednesday June 25, 2003 @08:06AM (#6292731)
    Forget the debate over if restricitng GCC is fair, let's think about what we are trying to measure!

    What it comes down to is the speed of the system, not the chip. The 1000+hp dragster is a useless vehicle to me because I don't need to go in a straight line at over 200 mph. What I do need is to be able to accelerate in traffic, handle corners, etc.

    To me, a computer is a system. So I don't really care if the G5 is cranking out power I will never need. I feel like the G4 and the P3 are plenty powerful chips if the OS is built to be efficient and the supporting components to the system are configured correctly.

    These days, my biggest reason to upgrade to a new computer is desire for faster system components. I wish my P3 had firewire (might go buy a card), I wish my PowerBook G4 (400mhz mind you) had BlueTooth (not 1600 extra mhz).

    Benchmark the user experience, give a review that is more like Automobile or Road and Track. Tell us the zero to sixty and then move on the how the G5 handles in the turns. Tell us if the wind noise is less than a Xeon. Tell me if it has power windows. But don't spend 90% of your marketing materials telling me about the engine, that is only 10% of my buying criteria.

    You just gotta TRY OS X!

When Dexter's on the Internet, can Hell be far behind?"

Working...