Dell $38m Supercomputer [not] More Costly than VT's G5s 578
An anonymous reader writes "According to the Austin Business Journal, Dell's 3-teraflop, 600 server supercomputer cluster cost the University of Texas $38 million. As The Apple Turns has pointed out that this is 7 times the cost (and a quarter of the power) of Apple's cluster at Virginia Tech! " Update: 10/14 17:56 GMT by M : worm eater writes "The Register has posted a correction to the widely-reported story that a 3.7 terraflop Dell cluster cost the University of Texas $38 million. As it turns out, the computer cost $3 million, vs. $5.2 million for the 17.6 terraflop Mac G5 cluster at Virginia Tech."
Here's a copy of the apple turns web page (Score:5, Informative)
Monday, 5:57 PM: Virginia Tech's G5-based supercomputer is (sort of) running-- with 17.6 teraflops of theoretical performance. Meanwhile, Dell tries to build something (sort of) similar, but it winds up with a quarter of the power and seven times the price, and Apple (sort of) announces Xgrid, a product for "parallel and distributed high performance computing"...
Monday, 5:57 PM: Today's holiday episode is now broadcasting. Don't forget to take your shot for a free AtAT shirt (tee or turtleneck) by entering the Q4/03 Beat The Analysts contest; guess closest to Apple's final reported quarterly profit or loss, and you get the garment-- or your choice of creaky old software from the Baffling Vault of Antiquity(TM). You've only got until Wednesday at 4 PM, and in the likely event of a tie the earliest entry wins, so why wait? Enter now!
Up, Running, & Kicking Tail (10/13/03) Fun fact: believe it or not, folks, AtAT's wild success isn't confined to these here United States. No, seriously, it's true! The show actually has semi-regular viewers holed up in such far-flung corners of the world as Iceland, the Dominican Republic, and Delaware-- and for the benefit of those fans, we thought we'd explain that, here in the U.S., today we celebrate a holiday called Columbus Day. Columbus Day, for the uninitiated, is one of our most sacred occasions: a day on which we reflect on the many cultural and technical achievements of the city of Columbus, Ohio. We celebrate Greater Columbus's world-famous quilts, its shrubberies recreating Pointillist masterpieces, and (most importantly) its commitment to the preservation of really old TV sets by wondering why the bank is closed and our mail never came. A good time is had by all.
So if this is such a major holiday, why are we broadcasting, you ask? Well, normally we wouldn't, but faithful viewer Nathaniel Madura pointed out that Slashdot just referenced a BBC World report on that G5 supercomputer down at Virginia Tech, and we're just a little giddy about the existence of a Mac-based cluster than can chew through 17.6 trillion floating point operations per second. Yes, the thing is up and running (at least enough to run performance testing), and reportedly it pumps out 17.6 teraflops of raw perforated aluminum muscle while sucking down enough juice to power 3,000 average homes. Wow, is it getting warm in here, or is it just us? (It's just us-- the G5s are cooled by means of 1,500 gallons of chilled water pumped through every minute. Ooooo, frosty.)
Kudos to the Virginia Tech team who pulled this off, because frankly, this is the sort of technological triumph we'd normally only expect to come out of, say, Columbus, Ohio. Now, what's interesting about that 17.6 teraflop figure is that if you scope out the last compiled list of the world's top 500 supercomputers (from last June), you'll notice that, if 17.6 teraflops is Virginia Tech's "theoretical peak performance" score, it'll probably slot in nicely at number three. (Scores are ranked by "Maximal LINPACK performance achieved," so it's just guesswork so far.) The top-ranked Intel-based cluster is currently ranked at number three, with 2,304 2.4 GHz Xeons and a theoretical peak performance of 11 teraflops. Gee, more processors, a higher clock speed per processor, and 63% of the performance. Now that's efficiency, baby!
We'll have to wait until the next top 500 list comes out in November to see if "Big Mac" (as the VA Techies apparently call it) really takes third place, or if the real-life LINPACK scores push it down lower-- but we figure a top five placement is a safe bet. One of the world's bestest supercomputers made of Macs and running Mac OS X? Why, it's a Columbus Day miracle!
4x The Bang, 1/7 The Buck (10/13/03) Meanwhile, we know that the G5 supercomputer is delivering more pluck per processor than any other supercomputer out there, but what about bang for the b
Re:Here's a copy of the apple turns web page (Score:4, Informative)
She said: "The $38M total you refer to was NOT for a single supercomputer. It was announced in February for a total package that included:
"The establishment of the new Institute for Computational Engineering & Sciences (ICES) at UT, including:
four new endowed faculty chairs in ICES at UT
additional funding for the research endowment and the visiting scholars endowment in ICES
the completion of construction of the ACES building (the 4th floor) for use by ICES and TACC
"and the establishment of a terascale distributed computing infrastructure at UT, hosted by TACC, including:
two supercomputers at TACC (the cluster you refer to, and the other IBM system
two massive storage systems at TACC
three leading-edge components to increase UT's networking infrastructure
increases in operations funding over five years for ICES and TACC".
She adds: "There are many more things that were needed to create ICES and establish a terascale distributed computing architecture at TACC. This point was made by TACC Director, Jay Boisseau, during the Lonestar dedication ceremony. The value of the specific computer referred to was approximately $3.0 million. And, no tuition funds were used in this process. Most of the money did not even come from UT. The package included $8M in discounts and donations from about 10 leading technology vendors, and over $15M from a generous foundation." And, she continued: "The VaTech number ONLY includes the actual computer, not the cost of the building, power, cooling, people, or anything else needed to actually operate it."
Re:Here's a copy of the apple turns web page (Score:5, Interesting)
To the parent, the focus in high performance computing nowdays is on "portable perforamnce"; for the most part, libraries are written so that the performance critical components are either very small, or self-optimizing (or both). For most applications, the most you might need to do is optimize some of the kernels (eg BLAS) for the architecure.
Re:Here's a copy of the apple turns web page (Score:3, Informative)
Re:Here's a copy of the apple turns web page (Score:3, Informative)
Re:Here's a copy of the apple turns web page (Score:4, Informative)
Here is the link:
VA Tech Cluster [vt.edu]
Look for the "Slideshow Presentation" link in the bottom left.
apple delivers, finally! (Score:3, Insightful)
second, i have never heard labor factored into the cost of a machine (tho to call either of these beasts, just a machine is a shame, lol) but it would be reasonable for someon to calculate the TCO based on computing power and the cost of cooling, interconnect, etc.
third, the macs are using SATA which is today's technology,
Re:Here's a copy of the apple turns web page (Score:3, Interesting)
This just in from Apple. (Score:2)
Common sense (Score:2)
See?! (Score:2, Insightful)
Seriously, though: How?
From the article: (Score:2)
If Dell had to contract Cray to help them build the cluster, I'm sure that aspect wasn't cheap. I'd imagine that most of the actual costs associated with getting the Dells into an efficient cluster were more mundane things than buying the servers themselv
Re:See?! (Score:4, Insightful)
Then I hope they're waterproof too, since the $38M also included the cost of a building to put them in...
Re:See?! (Score:5, Insightful)
Re:See?! (Score:2)
> large volumes of PCs? I'm intrigued by this
Bundled software (i.e. Windows) costs Dell something whether they install it or not. Since Apple owns the OS, they can lower the price of the machine below the price point of OS X and make a profit on the machine but not the software.
Costs (Score:4, Interesting)
Re:Costs (Score:5, Funny)
Wanna really go for cool? Guard the Apples using girls with guns!
Re:Costs (Score:2, Funny)
And, because this is Apple we're talking about, you make a movie about it and call it Kill Bill, Volume 3.
Sorry about that.
Re:Costs (Score:2)
Re:OMFG LOL!!.. Mod parent up plz!!!.. :) (Score:2, Funny)
Re:OMFG LOL!!.. Mod parent up plz!!!.. :) (Score:3, Funny)
What happens when it attains self-conciousness, and kills it's operators?
Re:Costs (Score:2)
More things that I'm sure people will talk about: The Dells are 1U and 2U boxes designed for rack enclosures meaning they'll be more heat and power effecient not to mention they take up about 1/3 the physical space as the enormous PowerMac G5.
If VT had waited for the XServer with G5s in it they would have a better cluster. But I realize the desire to be in a big computing list with other big dogs.
Re:Costs (Score:5, Informative)
Re:Costs (Score:2)
If you put a processor into a large box, it does not output any more heat than if you put it into a small box. If anything, the large box will have a more efficient means of heat exchange. Of course, this also means that the building will need the same cooling requirements too - since the same amount of heat is generated, the same amount of cooling is required.
It might be more space efficient, yes, but space mostly means money, and obvi
Re:Costs (Score:5, Interesting)
Re:Costs (Score:2)
moreover, apple wouldn't be able to provide these computers consistently at that price point to companies. making the whole comparision of "they should have bought apples" quite a stupid argument in these other, seperate bids(if they're not bidding how you choose them? you don't).
the g5's ain't bad price/perf/usability, but the vt cluster really yells 'marketing' a
Re:Costs (Score:2)
Re:Costs (Score:3, Insightful)
Offering discounts in compensation for marketing isn't "dirty cheap dealing", it's good business, particularly if the value of that marketing exceeds the discount amount.. Dell could have done the same...
Re:Math (Score:2)
Re: (Score:2)
Re:Costs (Score:3, Insightful)
If they are running the same code multiple times and comparing the results, shouldn't that be taken in to account when computing how fast it is?
Hmmm, let's see ... (Score:5, Insightful)
Can you say "sweetheart deal," boys and girls? I knew you could.
Re:Hmmm, let's see ... (Score:2)
Re:Hmmm, let's see ... (Score:2)
Re:Hmmm, let's see ... (Score:2)
This kind of thing happens all the time in small businesses, of course, and I have a hard time arguing with it; often, it's the only way a small business can survive those first
Re:Hmmm, let's see ... (Score:2, Informative)
Wasteful? Maybe, but... (Score:2)
Re:Wasteful? Maybe, but... (Score:2, Informative)
Shocking! (Score:5, Funny)
Re:Shocking! (Score:4, Informative)
Except that the nodes are not 5 years old. The PowerEdge 1750s are new models as of this summer. The 2650s are certainly not more than 1-2 years old, either.
Re:Shocking! (Score:2)
HH
Paying $126k for each server -- idonfinkso (Score:5, Interesting)
Methinks the price tag includes a lot more than the hardware costs.
The comparison with the VT supercomputer is almost certainly not apples to apples (so to speak)
Re:Paying $126k for each server -- idonfinkso (Score:3, Insightful)
The VT use Mellanox InfiniBand (InfiniScale 96-port 10Gb/sec switches and InfiniHost dual port host channel adapters ).
On the other hand UT probably paid many milions for the Cray solution. Faster(?) and with lower latency, but with a worse price/performance ratio.
Re:Paying $126k for each server -- idonfinkso (Score:2)
A direct comparison is way out there in left field. Somebody didn't bother to do the basic math. The actual article is full of grand assumptions. They may have a point that the apple cluster is a better value than the dell cluster, but you'ld have to look at several factors before you can realyl say that....
What was UT using previously to perform their calculations/sumulations, and what did it cost to operate? How much does it cost to customize/code the apps that they will be running on the clu
Re:Paying $126k for each server -- idonfinkso (Score:2)
Yes Indeedy - Read The Story, People (Score:3, Informative)
The article makes it plain that this is the just the beginning of a five-year project that will eventually spend $38mil, and which will end up with a lot more than 300 systems (200 to be added next year alone, for example) too. Comparing this to another project without knowing all the details of both is pointless.
Yes, I didn't read the article yet (Score:4, Insightful)
38E+6/6E+2 $ is about 60000 $ per machine. Seems to be a little much for a cluster of "cheap" machines, right?
Isn't there more to it?
Ok, off, reading the article.
Re:Yes, I didn't read the article yet (Score:2)
1,100 G5 machines, 1/7 the price of the Dell cluster = 4,545 per machine.
I cal shenanigans, wish someone would look into these biased articles before posting them.
~Berj
Re:Yes, I didn't read the article yet (Score:2)
How is it fair to compare five-year cost (which includes all kinda of things like upkeep, all the supporting hardware, etc) to the base machine cost of the G5 cluster?
~Berj
Re:Yes, I didn't read the article yet (Score:2)
Re:Yes, I didn't read the article yet (Score:2)
Additionally, as one can read at
http://www.tacc.utexas.edu/resources/hpcsyste m s/
TACC seems to have invested quite a bit into the network, interconnect and storage technology, while by the calculations of your parent poster, there shouldn't be much financial leeway for stuff like that in the apple cluster (and no, gigabit
Re:Yes, I didn't read the article yet (Score:2)
the extra 30 million (Score:5, Funny)
Re:the extra 30 million (Score:2)
In other news... (Score:2)
One UT suppporter was quoted as saying "We didn't get lousy clusters....we got lousy cluster planners!"
Re:In other news... (Score:2)
Someone sure got bent over a table... (Score:2)
PC users rationale (Score:2)
(/me happy new mac user)
Re:PC users rationale (Score:2)
My mouse has 5 times as many buttons! I scoff at your 2-button DELL mouse.
Price reversal (Score:2)
Re:Price reversal (Score:2)
Re:Price reversal (Score:2)
A better comparison would also consider what you get for the money. There is little price difference between similarly configured Dell and Apple machines -- and Apple usually comes out cheaper by a bit.
Too bad Austin BJ didn't know.. (Score:2)
Lets just hope Apple can keep up the good work and keep the G5 line updated and in pace with the x86 lines from Intel and AMD
Apples and Oranges (Score:2, Informative)
Speccing up a dual G5 at the Apple store comes out at over $5,000. They also need to pay for the power and cooling hardware to run the thing.
Looks like they are getting a very good price from all their suppliers/contractors...
The retail price for the processing hardware for the UT cluster is very similar, a dual PowerEdge 2650 with 4Gb of RAM is also about $5,000. If they had taken the workstati
a TENTH of the power, and they'd still use Dell... (Score:3, Interesting)
And don't try to tell me that the Company-Formerly-Known-as-Compaq had a shot even though they're based in Houston...well not really anymore anyway.
Just remember (Score:2)
Dude, (Score:2)
Not to mention the next site visit by NSF - that should be a hoot.
This should be on the Apple Hot News page.
Yeah but.... (Score:2)
Just because you have a cluster with mega TFlops doesn't mean it'll do more real work that something more expensive with less mythical peak performance.
$38M wasn't only for the cluster (Score:5, Informative)
So I hit Dell's website and at educational pricing the servers they bought run around $4k apiece. Which means that this solution should be very price/performance competitive with the VT cluster.
I hit the UT page and found that the $38M number came from a press release about their investment in quite a bit of stuff, including the "Institute for Computational Engineering and Sciences (ICES), a new center for interdisciplinary research and graduate study in the computational sciences." I.e. at least one new building.
Re:$38M wasn't only for the cluster (Score:2)
Not really. Dell's server hardware is great, even better than Apple's workstation/server hardware. Plus those servers come with at least Gold support contracts.
Project Cost vs. Equipment Cost (Score:2)
Apples got nothing on this (Score:2, Interesting)
Story location: http://www.wired.com/news/technology/0,1282,60791 , 00.html
02:00 AM Oct. 14, 2003 PT
A small chip-design firm will unveil a new processor Tuesday it says will transform ordinary desktop PCs and laptops into supercomputers.
At the Microprocessor Forum in San Jose, California, startup ClearSpeed Technologies will detail its CS301, a new high-performance, low-power floating-point processor.
The new chip is a parallel processor capable of perf
weird (Score:3, Informative)
they quote a dell spokeswoman saying that a configuration like that costs about 3 million with installation. it also states that UT gets an educational discount, but doesnt say how much they got off the $3million.
if the 38 million were correct, theyd be spending on the order of 120,000 per machine....a 2650 with highest processors and max ram only comes out to $13,500 on dells site...yeesh
Re:weird (Score:2)
The '1/7 cost' for the Apple machine is for the hardware only.
Someone should wake up and smell the coffee. In these days of commodity hardware, differences like a factor of 7 (overall a factor 28 in price/performance!) is at best highly improbable.
let's see... (Score:2)
Also, "flops" is a pretty meaningless measure of performance. If you want to use benchmarks, at least use SPEC, and on those, the Macs are not particularly outstanding in terms of price/performance.
Finally, the Macs aren't rack mounts and they require manua [vt.edu]
Cost of Operation (Score:3, Interesting)
Stupid question: Are these really supercomputers or superclusters? I always think of a computer as one unit not a collection of units.
I hope the NSA is not reading this ... (Score:2)
Not wanting to get picky (Score:2)
More to it (Score:2)
Of course, there's also this concept of "theoretical FLOPS", which is open to interpretation... we'll have to wait and see what the LINPACK (and other such benchmark) numbers report.
Ignoring the fact that 38m is PROJECT cost. (Score:2)
Very soon we will have that power.. (Score:2)
According to this Wired article [wired.com] a small firm in CA called Clear Speed [clearspeed.com] will soon revolutionize the PC space with Super computing power.
I know we will all believe it when we can find these chips on Bestbuy aisle no:4, but still currently from where I am sitting (I am sitting on a Microsoft biztalk server 2004 training session, boring as hell, being inundated by claims of innovation by a clueless trainer who programmed in Visual bas
What noone seemed to notice... (Score:3, Informative)
Oh, sure it's a quarter of the power... (Score:2)
Top500.org (Score:2)
On a supercomputer, communications overhead between the nodes is a big deal. Fast processors that can't be adequately fed don't cut it.
Cray... (Score:2)
It's interesting that his comes up. This month's Linux Journal had an article with a guy with no cluster experience building a large cluster using...drumroll...Dell systems. Cheap. And he did it with out expensive consulting services from big names. He's now on one of those international comparison lists for top-performing clusters.
The Apple cluster is also 64 bit. (Score:2)
Man.... (Score:2)
Not to mention ECC ram on the Dell's (Score:4, Insightful)
So you have fast incorrect data.
Please correct me if I am wrong.
Too much Spin (Score:3, Informative)
Hey, I'm a Mac zealot, I know. I love to make Apple look good, for any reason. But this story has too many inconsistencies.
The Slashdot blurb and the article don't even match. And As The Apple Turns is being quoted as a technically informed Mac news source?
Anyways, yes, $5 million for the G5s. Now let's add in the price of those racks they sit on. Let's add in the price of the cooling system, the network equipment, cables, power supply. Contractors. $38 million doesnt' sound so far off the mark when you think about what all that stuff.
Having a tough time accepting the numbers? (Score:5, Informative)
If this were an OS to OS comparison between Windows and OS X, perhaps we wouldn't be getting so frothed up. But this is hardware, and dammit, PC hardware is supposed to *always* be cheaper than Mac hardware!
To summarize what others have said:
1) Dell gave UT a sweetheart deal
2) Apple gave VT a sweetheart deal
3) Nobody has dredged up any information to indicate that the $38M UT spent includes the cost of a building. As csoto pointed out:"A "Center" at UT is a special term for a particular type of organized unit, often a research unit. It does not necessarily mean this place gets its own building. In fact, at UT, space is such a premium that most "Centers" don't have their own (yeah the place is huge, but has lots of people). In fact, I'd venture to guess that NO center has its own building."
4) Hardware is only a portion of the total cost, obviously. UT and VT have set up their supercomputing projects differently. This again is obvious.
5) The really important point of all this is that VT manage to put together a very powerful supercomputing cluster using Macs at a cost that in no way can be considered more expensive than if they'd used PC hardware.
You can argue that costs would have been cheaper had they built their own, or used PCs from some source cheaper than Dell. But they still would have had to deal with labor costs in assembling the PCs, or higher maintenance costs associated with keeping all of those commodity PCs running properly.
UCLA [ucla.edu] is already using OS X to run Beowulf-style clusters. Tokyo University [asahi.com] is replacing over 1,100 Linux PCs with OS X boxes.
Even the total cost of installing, operating, and maintaining large numbers of Macs running OS X is cheaper than either PCs running Windows or PCs running Linux, people often seem incapable of absorbing that information.
You can talk all you want about the Reality Distortion Field, but the truth is that Apple is always working against an incredibly strong bias that says Apple is always more expensive.
That's simply no longer true.
Price per Teraflop (Score:3, Informative)
3.7 tflops / $3M = 1.23 tflops/$1M
17.6 tflops / $5.2M = 3.38 tflops/$1M
So with the Apples you get 2.75x more computing power for the same $$.
Sounds like a an easy argument to get by the Dean's office...
Re: (Score:2)
Re:Apple is more expenssive (Score:2)
"Cost per Tflop: Dell - $10.3 million; Apple - $295,000. It's almost shameful."
In truth, Apple machines seem more expensive only because they don't sell $400 machines. Thus, if you want a Mac at all, you have to pay at least $1000. But that grand will still get you more computing power (and other advantages) than if you'd bought a PC.
Re:Apple is more expenssive (Score:2)
Re:Making it even more meaningless (Score:2, Funny)
One positive note though, Both the Apple and Intel clusters will be faster then the Microsloth cluster
Re:Making it even more meaningless (Score:2)
Since when is poor math "conclusive evidence"?
Hint: (3TFlops/300 machines = 10GFlops/machine)
Re:Misleading anyone (Score:2)
Even w/o the student discount, each dualie G5 sells for $3K roughly, 1100 machines in the cluster: Voila! $3.3 Million....
See math ain't that hard. They may have got a break in the memory since their machines come with 4GB per, and I believe the base dual G5 comes with 512MB.
Re:My e-penis... (Score:2)
In the real world you want the best performance for your budget. 38 million is a waste, as the comparison shows.
Re:Stereotypes (Score:2)
Re:WARNING; Reality Distortion Field Engaged (Score:2)
The savings come from the fact that it won't cost as much to maintain the G5 cluster.