Apple Wins VT in Cost. vs. Performance 105
danigiri writes "Detailed notes about a presentation at Virginia Tech are posted by by an attending student. copied most of the slides of the facts presentation and wrote down their comments. He wrote some insightful notes and info snippets, like the fact that Apple gave the cheapest deal of machines with chassis, beating Dell, IBM, HP. They are definitely going to use some in-house fault-tolerance software to prevent the odd memory-bit error on such a bunch of non-error-tolerant RAM and any other hard or soft glitches. The G5 cluster will be accepting first apps around-November."
mfago adds, "Apple beat Dell, IBM and others based on Cost vs. Performance alone, and it will run Mac OS X because 'there is not enough support for Linux.'"
water cooled laptops as blades (Score:4, Interesting)
A few years ago I asked apple if they would be willing to sell me 200 laptops without the screens, disks, video cards, and keyboards. They were interested helping me build my cluster but, the the engineers said it would actually cost them more to have a special manufacturing run than woul dbe saved by deleting the hardware.
my plan was stack these things on water cooled chill plates. Basically this would be like a blade.
In my circumstances, adding a well ventilated computer room to the building I was in would have been probibitively expensive. but water cooling and a high density configuration made this very appealing. And if I could have gotten the costs down and reliability up by deleting the screens, keyboard, video, and disks I'd have an affordable system with low sys-admin costs.
I still think its a good idea. Cooling/power costs (including building retrofits) and sys admin costs can dominate the differential purchase price of vairous cluster configurations. In my building the space alone was >120/sq foot, so even the footprint mattered.
Re:water cooled laptops as blades (Score:5, Informative)
The viriginia folks must have one huge room with some massive air handlers to circulate the air that will be trapped behind the towering walls of 1000 4U boxes.
I don't know any more than what's publicly availble, but the VT follks in the know have said that they've designed a specialized, liquid based cooling system precisely because of the issues wrt cooling this many units. The FA makes reference to this many units generating windspeeds of 60mph from fans alone.
I am gonna guess that behind each G5 rack will be a radiator type arrangement, with cooled pipes flowing with a liquid that will carry the heat away from the internal airspace, much like a large car radiator. I don't know if that would be cost-effective, or what it would take to move that much liquid, or if the radiator could be made to transfer enough heat fast enough. Maybe the liquid cooling units actually replace the internal fans directly. Who knows--I think we'll get some more details on this this week as the G5s start to come out of their boxes. They've apparently received about 10% of them already.
Free printer and Ipod case (Score:5, Funny)
Interesting (Score:3, Interesting)
Who could have guessed?
Re:Interesting (Score:1)
Here's my question, and no, I'm not trying to start a flame war...if they were going purely on a price/performance ratio, why didn't they let Dell "explore pricing options"? Presumably, Dell would have given them a better deal due to educational and/or prestige factors, if Dell really wanted the deal. If that was the case, the uni may have been better served with the
Re:Interesting (Score:3, Insightful)
At least, that's the way I've been parsing it.
-Ster
Re:Interesting (Score:2, Interesting)
Re:Interesting (Score:2)
Oh, really? Does Dell have a competitive 64-bit solution? I don't think so. Even the 32-bit dual Xeon Dell is more expensive the dual G5 Power Mac. Don't bother mention Itanium2, because it's too hot and expensive, and there is hardly any native apps, which might be why people are not buying t
Apple Outshines Dell on Ethics (Score:4, Interesting)
As an American company, Dell is a huge disgrace. Please read the "Environmental Report Card [svtc.org]" produced by the Silicon Valley Toxics Coalition [svtc.org]. Dell received a failing grade and is little better than Taiwanese companies, which are notorious for destroying the environment and the health of workers. Dell even resorted to prison labor [svtc.org] to implement its pathetic recycling program.
Re:Apple Outshines Dell on Ethics (Score:1)
software to solve memory problems? (Score:1, Interesting)
IMHO the lack of ECC RAM is the only flaw in an otherwise perfect machine (well that, and the massive HEAT).
Re:software to solve memory problems? (Score:1)
In my humble ignorance, I can devise a simple stratagem (surely far simpler, very inneficient and dumber than the one used by VT). Just duplicate all calculations (effectively halving processing power) on different machines, chances the same error hitting both machines would be vanishingly small.. If a discrepancy in results is found, just recalculate.
Re:software to solve memory problems? (Score:2)
You have to look a little further, and don't let the number fool you. The reason for the huge heat sink and 9 individually controlled fans in the G5 is reduce noise level.
The G5 consumes about 40W at 1.8 GHz, which is much more efficient than both the 1.5 GHz Itanium 2 (130 W) or the 3 GHz P4 (75W ?).
Re:Power concerns (Score:4, Funny)
Power consumption would have only made Apple look better.
Re:Power concerns (Score:5, Informative)
# 3 MW power, double redundant with backups - UPS and diesel * 1.5 MW reserved for the TCF
# 2+ million BTUs of cooling capacity using Liebert's extreme density cooling (rack mounted cooling via liquid refrigerant) * traditional methods [fans] would have produced windspeeds of 60+ MPH
Seems that they did talk about both.
whew! (Score:4, Informative)
1.5 MW reserved for the TCF
2+ million BTUs of cooling capacity using Liebert's extreme density cooling (rack mounted cooling via liquid refrigerant)
traditional methods [fans] would have produced windspeeds of 60+ MPH
bug free computing...the other kind (Score:2)
Yeah, but NO BUGS! [terminix.com]
Clueless Sysadmins... (Score:1, Informative)
Yes, incidentally, it does. The units came with high end graphics cards
Aside from games, when is a high end graphics card needed for rendering and not just displaying a rendering.
Re:Clueless Sysadmins... (Score:3, Insightful)
IANAS, but:
Re:Clueless Sysadmins... (Score:5, Insightful)
So they could equip one G5 with a radeon9800 and let that one display the results. No need to buy another 1099 Radeons.
Re:Clueless Sysadmins... (Score:3, Interesting)
Re:Clueless Sysadmins... (Score:3, Interesting)
Re:Clueless Sysadmins... (Score:2)
Re:Clueless Sysadmins... (Score:3, Interesting)
Re:Clueless Sysadmins... (Score:1)
Johann
Re:Clueless Sysadmins... (Score:1)
Imagine a display made of 1000 monitors in a 40x20 grid. That would be pretty freaking cool.
Or better yet, a display made of 1000 monitors in a 50x20 grid, so that it would make sense.
Re:Clueless Sysadmins... (Score:1)
graphics in science (Score:3, Interesting)
The system doesn't have to be chaotic, just complex:
Watching protein folding simulations.
Watching full 3-D seismic waves propagate through the Earth.
Watching, in general, any kind of 3-D model or simulation of a complex process evolving over time.
A couple links:
The Scripps Institute of Oceanography Visualization Center:
http://siovizcenter.ucsd
Re:graphics in science (Score:1)
Re:Clueless Sysadmins... (Score:4, Interesting)
Now that "consumer" graphics cards run in floating point and have comparitively complex shader engines, it's quite possible to start working on rendering movies etc. with the substantial quantity of hardware acceleration possible on these things. You don't have to hit 60fps, and you can have as many passes as you like.
Mind you, with 1100 nodes if you can render a frame in 45 seconds
Dave
Re:Clueless Sysadmins... (Score:2)
Re:VT? (Score:1)
Infiniband insured latency? (Score:5, Interesting)
From most of my reading with Infiniband, it was designed from the ground up as a NAS style solution, than for large multi-node cluster computing. I'm curious as to if they have any issues with cluster latency.
http://www.nwfusion.com/news/2002/1211sandia.ht
The primary timings and white papers I've seen published for Infiniband have been for small clustered filesystem access. Although it's burst rate is much higher than Myranet, it's hard to find any raw retails for their multiple node latency normalization.
I hope it scales, since Intel's solution appears to be less cost prohibitive than some of the other solutions offered on the market, and would really open up the market even for smaller clusters (16-36 node) for business use.
I love Google (Score:3, Informative)
For those in the know (Score:4, Interesting)
I wonder if by "lack of support in linux," that they're refering to the fact that the fans are controlled by the operating system in the powermac? Or the fact that there are relatively few support companies for ppc linux?
Any insiders care to comment?
Re:For those in the know (Score:3, Informative)
Re:For those in the know (Score:5, Insightful)
Re:For those in the know (Score:2)
Fine, fine. But you didn't really answer my question. I waned to know why they said that linux was too unsupported. The fans are an issue that I know about. The relative dearth of support companies for ppc linux is another. VT might have another rational.
You can get into flame wars all day about linux vs. osx. I use both daily. I am a scientific programmer.
For their sake, I really hope that they're not planning on using HFS, tho. mpich (which I know they're not using, but I'm going to cite as an ex
Gentoo Linux Runs On The G5 ... from Mac/ (Score:2)
on macslash this story [macslash.org] talks about the crazy speeds they are claiming on the G5 running Gentoo Linux. Says they can not go superfast because of fan control issues still unresolved, but yikes! too good to be true?
Re:Gentoo Linux Runs On The G5 ... from Mac/ (Score:3, Interesting)
Why hell, i get blazing speeds with gentoo on my Athlon, i'd sure hope that you'd get them on the g5 as well
Re:Gentoo Linux Runs On The G5 ... from Mac/ (Score:2)
Re:Gentoo Linux Runs On The G5 ... from Mac/ (Score:2)
ECC FUD (Score:5, Informative)
1. The majority if not all of the bit errors that ECC corrects are caused by thermal noise. Thermal noise is an issue in a cluster of rack mounted 1U units due to the difficulty of cooling such tightly spaced units generating so much heat in so small a space. It is not an issue in a cluster of DESKTOP machines utilizing a Liebert system with way more cooling capacity than is needed.
2. Even if somehow a none-thermal bit error occurs, each node has 4GB RAM. The probability of it being in an OS or application critical (especially given the converging nature of many long running calculations) piece of RAM as opposed to an empty piece of RAM is small.
How many of you are reading this from a desktop without ECC RAM that has an obnoxiously huge uptime? ECC is a non-issue in a well-cooled cluster of desktop cased machines.
Re:ECC FUD (Score:1, Interesting)
Errr, what is the point of putting 4+ GB into your cluster nodes if you're not going to use it? This isn't a SETI@home cluster. Seems to me that "long running converging apps" tend to have large datasets associated with them. The higher the data density per node the less network bandwidth needed except for "em
Re:ECC FUD (Score:2)
Re:ECC FUD (Score:1)
1. The majority if not all of the bit errors that ECC corrects are caused by thermal noise. Thermal noise is an issue in a cluster of rack mounted 1U units due to the difficulty of cooling such tightly spaced units generating so much heat in so small a space. It is not an issue in a cluster of DESKTOP machines utilizing a Liebert system with way more cooling capacity than is needed.
Why is it necessary then to jam 1U units stacked on each other? If you can get the same performance and storage c
Re:ECC FUD (Score:4, Interesting)
Think before you post. The failure rate is constant in each memory chip (actually it goes up a bit with higher capacity due to higher density). Unless you setup the memory to be redundant (which the G5 can't do either...) you will experience MORE errors since a good OS tries to use the empty memory for things like file buffers.
How many of you are reading this from a desktop without ECC RAM that has an obnoxiously huge uptime? ECC is a non-issue in a well-cooled cluster of desktop cased machines.
Sigh... this is a 2200-cpu *cluster*. Here's a primer on statistics. Assume the probabiliy of a memory error is 0.01% for some time interval (say a week or month). The likelyhood for a perfect run is then 99.99% on your single CPU, which is just fine. Running on 2200 CPUs, the probability of not having any errors is 0.9999^2200=0.8, or 20% probability of getting memory-related errors somewhere in the cluster.
The actual numbers aren't important - it might very well be 0.01% probablility for an error per year, but the point is that when you run things in parallel the chance of getting a memory error *somewhere* is suddenly far from negligible.
ECC is a cheap and effective solution that almost eliminates the problem. Incidentally, one of the challenges for IBM with "Blue Gene" is that with their super-high memory density even normal single-bit ECC might not be enough.
But, what do I know - I've only got a PhD from Stanford and not VT....
FUD Back At You (Score:2)
There is nothing worse than having a computer without ECC or parity memory, and trying to detect and diagnose subtle pattern sensitivity memory problems.
Besides thermal noise, you also have to co
neat. (Score:4, Interesting)
Re:neat. (Score:1, Informative)
ah. (Score:2)
Dont forget.... (Score:2)
And never forget the costs of installing these puppies. Cooling systems, power busses, cable harnesses, UPS, Diesel backups, Air filtering, locks, redundant parts.
and what about the disk servers....
definitely. (Score:2)
As for the disk 'servers', I figured they were just sharing all of the 160GB HDs over the network, seeing as how 160GB x 1100 ~= 176TB (ok, it's more like 172TB, but who's counting...)
Re:Dont forget.... (Score:1)
Re:neat. (Score:5, Informative)
Only 4GB RAM (Score:2)
-Waldo Jaquith
Dude... (Score:5, Funny)
Certain things are easy to imagine in large quantities, but dude.
Just....dude....
Re:Dude... (Score:2)
Re:Dude... (Score:4, Funny)
Re:Dude... (Score:2)
Re:Dude... (Score:2)
Re:Dude... (Score:2)
An interesting tidbit (Score:5, Interesting)
Re:An interesting tidbit (Score:2)
I caught that too. Use of Macs in 2006 no doubt depends on 2 factors: 1) how well the 2003 cluster works out, and 2) how the Mac compares to competitors in 2006. Could be a nice win for Apple, again, if they manage to keep both 1 and 2 competitive. Which remains to be seen, and I'm holding my breath.
Re:An interesting tidbit (Score:4, Funny)
I don't know. Holding your breath until 2006 sounds... dangerous.
Re:hope this doesn't mean... (Score:3, Informative)
Nice rack! (Score:2, Interesting)
(Especially seeing as a G5 XServe will probably be at least several months away -- at least until most of the desktop orders can be filled.)
-Alex
Why was bidding secret? (Score:3, Interesting)
Re:Why was bidding secret? (Score:3, Insightful)
Re:Why was bidding secret? (Score:3, Interesting)
"dealt with vendors individually because bidding wars do not drive the prices down in this case."
I don't think they've even dealt with Apple until Apple's G5 announcement but they did deal with other vendors. I'm interested why VU dealt with all of them individually and why do prices not come down when you deal with them in this way. This is why I was alluding to collusion.
Re:Why was bidding secret? (Score:2)
Silent bidding with all the potential vendors knowing that you are getting bids from other vendors means they don't just fudge numbers to come in lower than the other guy.
And before you start - 3 fixed rounds of bidding just means round after round of f
G5's cheaper than VTs? (Score:5, Funny)
And they scroll MUCH more smoothly than OS X.
Imagine a beowulf cluster of those (Score:2)
Cost Analysis (Score:1)
From what I've heard, VT ordered the G5 the day they came out, or shortly after. But if one were to perform a cost vs performance, they would need background data. Also, they should have been hesitant to accept Apple's specs on the machine, and hoped for some real world test, or maybe some in-house testing of a few machines.
I find it hard to believe that VT was able to truly compare the G5 to competitor products, with out prior data of the ma
Re:Cost Analysis (Score:5, Insightful)
Just enough to demonstrate that Apple *would* have a solution, and enough that VT could narrow down the decision to a possible, pending the actual production and purchase of a single machine... then, the contract being 99% complete, they just had to sign a couple papers and purchase, overnight, 1,100 dual G5s.
On the flip side I bet they had a similar contract in the wings with other vendors, all pending on 'simple' bottlenecks.
I finally make it to slashdot... (Score:1)
Re:I finally make it to slashdot... (Score:1)
Re:They aint won VT!! (Score:2)
Re:They aint won VT!! (Score:1)
Yes, but the -1 (Stupid) moderation option hasn't been implemented yet by Taco. ;)
-T