Virginia Tech Announces Supercomputer Plans

CousinVinnie writes "Previously noted in this Slashdot story, the administration of Virginia Tech has announced they're puchasing 1100 G5's (another story) in hopes to build a top-10 supercomputer by October 1. Tech will be spending $5.2 million over five years on the project, which should help it pull in more research money." Maybe VT can use the new computers to beef up their web site.

This is quite cool but...

[WinterSolstice]

Does anyone know who else was considered for this contract? I'd love to see the arguments for the different platforms!

-WS

Yikes..

[knghtrider]

1100 G5's...that should corner the market for about a week...and give Apple a small boost to it's bottom line..

Anyone have any real specs?

[anzha]

So far we've seen that it's a cluster and what the building blocks are. What's the interconnect? What's the OS? What are the nodes using for a network filesystem? Are they at all? Is this intended for parallel jobs or for embarassingly parallel work?

Re:This is quite cool but...

[Kalak]

Dell and HP were considered, and Apple won based to a large degree on delivery date. There are more issues to computing than benchmarks, and in the issue of deliverability, Apple won. If you RTFA on the CT, they say it was on the speed and memory of the G5, but the geek grape vine, and hints from the Roanoke times article said availability to get it up in time to make the next top 500 comuter listing we big factors.

Both Dell and HP have recently announced large clusters, so that may be why they were unable to deliver in time.

PowerMac G5s?

[foo fighter]

Why aren't they waiting for the Xserve update? Rhetorical question, but still...

I haven't seen one, but it looks like the PowerMac G5s are about 4U wide. 1100 x 4U = 4400U / 42 per rack ~= 105 racks.

Not only is this going to take up an enormous amount of room, but the power and cooling requirements are going to be crazy as well. And they don't have rails so getting them in the racks, and working on them once in the rack, is going to be a PITA.

1100 G5 Xserves would need only about 25 racks. Many fewer UPSes and A/C units to power in each rack. Much easier to install and work on.

I know Apple is gung-ho about this validating their "Fastest PC Ever" claims. But it seems a little poorly thought out on the University's part even if they got a sweet up-front price on the machines. Remember: the system price is a small part of TCO.

Their Website

[rabbit994]

Pudge thinks their website isn't good enough. What does he want? Some flash? Maybe some pop up ads to spice it up. Whatever happen to simple being good and fancy being woooo pretty but useless. Oh wait, that still hasn't changed.

Re:Overpriced G5s

[Anonymous Coward]

Well, there is the small matter of thermal dissipation--heat to polysyllabically-challenged types like yourself. Those AMDs you're babbling about would cook themselves; the traditional advantage of PowerPC processors for clustering is in the fact that they run MUCH cooler.

Even if you could make the AMD-based cluster run, you'd have spent so much on cooling that the G5s would end up being cheaper. Just ask DOD...

Re:Anyone have any real specs?

[mfago]

The interconnect is Infiniband by Mellanox [mellanox.com]. These things get 10Gbps bandwidth with 6us latency under MPI. Very decent stuff. There is more information at the site above.

Note that 1100*$3000 = $3M. This doesn't include the 4GB RAM, but also doesn't include any volume discounts. Thus the interconnect may cost about $2M.

Oh, and to the guy who said "4 Athlons + Myranet is the same price as one G5" -- can I have some of what you're smoking?

Not really a dupe

[Kalak]

This isn't really a dupe, as this is a mention of the first official words form the school on the subject. Officials are finally speaking (and in some cases backing off) of the cluster in public.

Re:This is quite cool but...

[BWJones]

Does anyone know who else was considered for this contract? I'd love to see the arguments for the different platforms!

Well, considering that the G5 has many of the architectural features of those $40k SGI Octanes that I purchased a few years ago, I would consider that pretty impressive. In short, Apple designed the G5 machines with completely independent busses, so that saturating say an I/O bus will not have any effect on the throughput of say memory to CPU. They are pretty impressive and I can see why many folks who are currently using the Octanes etc... would want new G5's.

So, you have a UNIX box with true plug and play, 64-bit, nice GUI, full CLI access, Firewire, USB, REALLY nice archetecture etc...etc...etc... All that makes for a pretty convincing argument for clusters moving to the G5's

Argument for G5 here.

[eddy]

I'd love to see the arguments for the different platforms!

I think the argument for G5 came from here [mac.com].

Re:I keep overestimating slashdot...

[Lally Singh]

(and of course, after all that intellectual trash-talk, I posted without preview. here it is again, with the right (semi-) formatting). Lots of "WHY?" questions, with lots of pointless trolling on the G5; but none of them actually look for answers. Mostly just more idiots who can't understand that a good vendor is important; that their own time is important; that ease of use is even more important now than it ever has been before. Luckily, these same idiots spend all their time setting up sendmail over their 14.4 modem. As for the G5, here are some strongpoints for it: - A fast memory pipe (1GHz) - Good heat management (9 fans but it's quieter than its predecessor) - Damn good FP performance (To get comparable FP performance on intel, you have to use the -fviolate-ieee flag on gcc, think about that) - Vendor-installed, vendor-supported Unix, with the vendor employing the entire OS's development team. - Fast system interconnects with network & I/O - Easy system setup (this matters a lot when you've got 1100 of them) - Proven apple reliability (and if you're going to fight this one, have something better than "is not!") (again, very important when you've got 1100 of them) Oh yeah, and OS X. Mach microkernel, Rondezvous, and distributed builds in the default toolset. Again, the idiots I mentioned above wouldn't have a clue about this stuff. As for _why_ VT getting this, VT's one of the largest engineering schools in the country. We've gotta simulate airflow over wings, heat propogation over materials, and other stufff this CS major doesn't understand. And we've got big development in bioinformatics. All kinds of CPU to crunch. AFAIK, the cluster's being paid for by federal grants or something like that. And now fools, flame me. Prove me right.

They are very quiet

[Iowaguy]

Below is a link to show the noise of g5. (a movie). Apple did something called engineering (imagine that in a pc!), to put in many noise reducing features. So, the boxes may be bigger, but you get less power consumption and less noise. It is almost as if you pay extra money, and get extra features. Weird, I know....

http://homepage.mac.com/aaronsteele/iMovieTheater2 0.html

Re:Apple ... supercomputer...?

[alfredo]

Apple has won other contracts by being able to act quickly. When the Postal Service needed to set up an intranet, everyone said one to two years. Apple using WebObjects was able to do it in 6 months. At the time the Postal Intranet was the largest in the world.

Re:This is quite cool but...

[Anonymous Coward]

Just curious. Do you log your memory errors, and if so what is the error frequency?

As an alumn...

[Simkin1]

As an alumn, I am irritated with the decision. As a cluster developer of 5 years, I'm highly irritated with the rational of the G5. It's one thing to develop a system for the intention of doing research, it's another to base a decision on "..delivered by Oct. 1..". The question you should be asking is, which is more important - getting on the list? or doing the research? Seems to me that there is a more cost effective solution, that provides higher capacity, greater throughput, and more overall compute capability at lower cost... I'd personally suggest VT slow down, rethink the cluster, and buy something that fits the needs of the school and research programs-therein.

Side Note: While Tech has a great football team, the football program is (other than special discounts to students, and using the VT name) completely independent of the school. The football program is a business venture that does not interact with or require school permission, nor is it governed by the school boards that Steger answers to.

Use Linux!

The G5 really is all that.

[GPS Pilot]

Read why here [macedition.com].

Re:I keep overestimating slashdot...

[Erich]

There has got to be some sort of random apple troll comment generator.

Just follow these easy steps:

One: Baseless flame against everyone who disagrees with you:

Mostly just more idiots who can't understand that a good vendor is important; that their own time is important; that ease of use is even more important now than it ever has been before. Luckily, these same idiots spend all their time setting up sendmail over their 14.4 modem.

Two: Copy random specs from Apple's web page:

A fast memory pipe (1GHz) - Good heat management (9 fans but it's quieter than its predecessor) - Damn good FP performance

Three: Straight-out lies and made up stuff:

To get comparable FP performance on intel, you have to use the -fviolate-ieee flag on gcc, think about that [...] Proven apple reliability

Four: more flaming everyone else:

Again, the idiots I mentioned above wouldn't have a clue about this stuff.

Five: Claim to be superior:

We've gotta simulate airflow over wings, heat propogation over materials, and other stufff this CS major doesn't understand. [...] All kinds of CPU to crunch.

That's all it takes!

Sigh.

I haven't done real system administration for quite a while, but it's still blatantly obvious that you've never really had to deal with the administration of a compute cluster.

Apple doesn't have a proven reliability record. At least, not in the enterprise server arena. Sun Enterprise Servers do. HP does. Apple? No.

How can you seriously consider something like this without ECC memory? Do you really think that running 1100 copies of MacOSX on 1100 hard drives is a reasonable way to run a compute cluster? Do you have any idea how unreliable that would be? Do you really believe that Apple's Rondezvous will get everything setup perfectly for Infiniband?

I see. You really do. How unfortunate. If you really are from VT, you represent it poorly.

Rumors getting closer

[Kalak]

In talking to the person who is recruiting me to help lug the computers around when they arrive, the OS is to be OS X 10.2.7 on arrival, with plans to upgrade to Panther upon it's release. Straight out-of the box releases, with NetBoot planned to be used to distribute the images to each computer. This contradicts the rumors I've heard before, but is closer to a source who is on the planning team, who is too damn busy to talk to a luser like me.

Those who are possible volunteer recruits, there is an info session in Andrews ISB in the Corp. Research Center at 7:30 tonight and tomonrrow night (same presentation both nights). You *cannot* be on wage for VT to be elegible. I'm not sure if GAs count as this, since I'm not one, I didn't check.

Re:Apple marketroid

[BWJones]

Wonderful: three pointers to Apple's web site, pointing to pages with slick corporate "interviews". Do you actually work for Apple or are you just insanely zealous?

There are an awful lot of scientists using Macs for their research and work. I use them almost exclusively now after retiring my SGI's in favor of the OS X boxes and judging from the meetings I attend, I would say Macs have anywhere from 10-40% penetrance in science depending upon the subfield. For instance the last vision meeting I attended (ARVO, the big one for the vision research community), there were Powerbooks and iBooks everywhere. Probably a good 33% of the laptops I saw.

dual use for these machines?

[esome]

would it be possible to use the machines both as lab machines for students AND as a cluster? I mean does the gear that networks them necessarily prevent them form being used as individual machines? sorry for the n00b question but it sure seems like VT would get a lot more for their money that way.