Apple Discontinues ZFS Project 329
Zaurus writes "Apple has replaced its ZFS project page with a notice that 'The ZFS project has been discontinued. The mailing list and repository will also be removed shortly.' Apple originally touted ZFS as a feature that would be available in Snow Leopard Server. A few months before release, all mention of ZFS was removed from the Apple web site and literature, and ZFS was notably absent from Snow Leopard Server at launch. Despite repeated attempts to get clarification about their plans from ZFS, Apple has not made any official statement regarding the matter. A zfs-macos Google group has been set up for members of Apple's zfs-discuss mailing list to migrate to, as many people had started using the unfinished ZFS port already. The call is out for developers who can continue the forked project."
Daring Fireball suggests that Apple's decision could have been motivated by NetApp's patent lawsuit over ZFS.
The straight dope (Score:5, Interesting)
Posting anon, lest someone guess who my sources are.
The long and short of it was, Apple and Sun couldn't come to terms on the licensing. Sun wanted a lot of money for giving it to Apple under different terms and the amount they wanted was in the range of "hell, we could do it ourselves for that".
Add to that, the Oracle buyout and Sun going into management paralysis, and Apple decided to go it alone.
Apple's CoreOS team includes several of the lead engineers from the ZFS project (who fled the remnants of Sun in the Schwartz melt-down), and the architect of the BeFS. I'm expecting Apple to do their own next-generation file system, probably in the 10.7 timeframe.
The Reason is Probably Technical (Score:5, Interesting)
However, they still desperately need a next generation filesystem and according to the linked article they're hiring filesystem engineers. I don't see any evidence that this was anything other than a technical avenue that they've explored that has fallen by the wayside as so many have before.
Comment removed (Score:5, Interesting)
Re:The straight dope (Score:5, Interesting)
That doesn't make any sense. I fail to see why Apple should agree licensing terms for a CDDL licensed open source project or how Sun could demand money for the privilege. Sun were positively overflowing with love towards Apple (as they usually are) when they heard that anyone would actually be interested in their uber new filesystem.
Too bad, ZFS has some nice features (Score:4, Interesting)
Leaving aside all the crazy storage pool stuff (great for servers, not necessarily that useful for desktops), there are some interesting features in ZFS that I hope make their way into Mac OS X in some filesystem.
Snapshots and Copy-On-Write filesystem clones seem like a great way to improve the Time Machine backup feature, and would make it easy for applications to provide backup-on-save very efficiently.
The compression and encryption features would likely be useful for some people. I don't think the increased filesystem limits (number of files, size of files) would matter for most folks.
Re:The straight dope (Score:5, Interesting)
(emphasis mine)
Unfortunately, btrfs isn't "going" anywhere. Guess who their development was funded by? That's right, Oracle! Notice that they haven't released anything new since BEFORE Sun's shareholders approved the acquisition? (Latest release on the btrfs wiki is v .19, released in June 2009) It's not exactly improving at a breakneck pace... If btrfs is going to go anywhere, they need some real development money.
Dang Oracle.
You can still undevastate yourself (Score:2, Interesting)
I suggest you drop MacOS like a hot potato, send a nastygram to Apple giving them a piece of your mind, and check out both OpenSolaris and FreeBSD. They both support ZFS, OpenSolaris because Sun invented ZFS, and FreeBSD because they have competent management AND engineering. Unlike certain others (and I'm not pointing the finger at linux).
Once again, FreeBSD has shown the fools in Cupertino how it's done.
Re:The Reason is Probably Technical (Score:5, Interesting)
I'm sure it will, but I'm afraid that doesn't mean that made it practical for Apple to integrate into OS X or that it fitted the use cases they needed for many desktop scenarios. The FreeBSD people still haven't been able to run and integrate it reliably.
The ZFS advocates trot those lines out every time and they're total nonsense. Ultimately, the only way to deal with silent data corruption or 'bit rot' is to have multiple levels of redundancy several times over for your data - which ZFS has and deals with. No desktop Mac can ever have that. Anyone who thinks that is anywhere near being practical to deal with on a desktop system is an idiot, and no, I'm afraid booting OpenSolaris with ZFS on your desktop system at home and not having it crash and burn does not even approach the kind of issues and corner cases that Apple's engineers will have to deal with, especially in a desktop system like OS X.
By no stretch of the imagination does ZFS handle this 'magically'. There is a severe price to be paid. If you don't have redudancy then you will simply risk losing your ZFS pool if there is corruption.
I'm afraid that hardware, bad sector and disk issues are far, far more prevalent problems than data corruption at an OS level. Many apparent corruption issues at the OS level are usually down to hardware issues somewhere down the line. It might be a problem for operating systems with fairly shitty and poorly maintained disk and controller device drivers with a poor history on x86 and widely used hardware (hello Solaris!) but I'm afraid it's just not a primary concern for everyone else or for those developing desktop operating systems.
Re:The straight dope (Score:4, Interesting)
Probably because Apple hadn't felt like supporting an FS on its own?
Mac OS X already includes pile of licensed technologies. The sole purpose of that is to offload work from R&D so that they can do something more useful than reinventing a wheel.
Licensing deal likely would have been needed so that if Sun/whatever goes tits up, Apple would retain all rights to the code so that they can develop and maintain it further on their own - without being in mercy of whoever buys Sun after that.
Re:With SSDs, who needs it? (Score:1, Interesting)
You're saying CRT's never had tearing? My LCD has a 85Hz refresh rate, and it was cheap. You seem to be remembering CRT's as having infinite refresh speed for some reason. Hell, the fact that CRT's actually *blank* and LCD's don't makes CRT's far worse as that particular family of artifact goes.
As for color gamut, yeah the cheap one isn't quite up to it. But LED TV's are out now tho and they have a gamut that's bigger than any CRT ever was. That'll be coming to monitors real soon now.
Re:With SSDs, who needs it? (Score:3, Interesting)
Grab a small SSD for apps/games and a 5400RPM terabyte disk for all your music/movies/series/home video/whatever. 64/80GB disks seems to be the sweetspot now, which even leaves some space for apps after installing Win7 ;)
Comment removed (Score:4, Interesting)
Re:Yes. (Score:3, Interesting)
For the reason I stated: That using Sun's ZFS left them without control of development, and tracking an outside codebase has reputational risks to which Apple in particular is averse. Having ex-Sun people work on a new filesystem is great, but they still need to navigate the patent minefield that Sun has sown around ZFS.
Interesting that Sun non-competes did not stop their engineers walking down the street to work on directly competitive technology... (First I heard that engineers left Sun for Apple, actually. I thought the ZFS team was quite small, and it is obvious from the list that the key people remain at Sun.)
Re:The straight dope (Score:5, Interesting)
It doesn't make sense to you. But it does to every other business.
Free, open source software != costs nothing. (*)
Sun wanted Apple to share development and maintenance costs. Apple wanted some long-term guarantees that Sun wouldn't stop development and would also help Apple to solve problems of ZFS under Mac OS X.
Similar deals happen all the time.
It's just this time the companies couldn't agree on price and/or terms. Obviously acquisition by Oracle contributed to the volatility of situation.
(*) File system as file system is a rather trivial thing with well defined interface. ZFS is not only a file system, but also volume manager and network service. And long list of management tools for all that. Those are big money involved in development, maintenance and support of all that stuff.
Re:With SSDs, who needs it? (Score:4, Interesting)
Yeah, but a lot of people don't need terabytes worth of storage. I just built a machine for my wife with a 64 Gb SSD, which cost $180. She's currently only using about 25% of the space. The good thing is that the machine is really fast on some tasks she does that require a lot of hard-disk access.
Re:With SSDs, who needs it? (Score:5, Interesting)
SATA attach SSD has achieved price parity with enterprise SAS, the density is almost there, and the performance completely blows it away. We're not at the end of spinning disc, but you can see it from here.
The new performance tier of storage is PCIe attach SSD [memory4less.com]. At two terabytes of storage and 1.5GB/s per slot, we're getting close to what we used to get from Ramdisk in performance and adequate density at 3TB per rack unit including server (HP DL785 G5 or equivalent). Yes, this is expensive right now, but the performance tier always has been. This is for trading platforms, HPC and such. These are approaching 2M IOPS and 40TB per 7U server.
The second tier is 2.5" 256GB SATA SSDs. You get 3TB per rack unit including the server. About the same cost as SAS for 10x the performance. Software options enable you to scale this to infinity in both bulk and performance. Great for databases, VMDK files and iSCSI. Get the hot-swap version and leave some open bays so that when the 1TB 2.5" SSDs come out you can migrate your LUNS with no downtime.
The third tier is SAS spinning disk. At something like 20TB/Rack unit (excluding servers) you can use this to serve frequently used files.
The fourth tier now is SATA spinning disk. At roughly the same density as SAS spinning disk for one-fourth the cost, this is a good candidate for deduplicated targets like virtual tape libraries or deduplicated NAS. It's also a good place to store your snapshots. With modern snapshot technologies there's no good reason to not store snaps every 15 minutes or so. Typically you would park this storage offsite for DR purposes so you can avoid the Premium Microsoft danger eXperience(**).
Storage pros probably would note that I neglected to mention tape and Fiber Channel. That's neither accident nor ignorance. The only reason for tape is legally mandated tape backups, and I consider this the IT equivalent of legally mandated hitching posts outside every business (which laws persist in some places) - if you gotta, you gotta, but there's no reason any more to consider it a necessary or good practice. As for Fiber Channel, it just doesn't fit in the model any more. I know this hurts the feelings of folks who just dropped a million bucks for a single rack of SAN storage with 100TB, or worse - popped for the new 8GBit stuff complete with a converged ethernet/FCoE solution, but it's true. There's just no reason for fiber channel any more. It just doesn't have the bandwidth to support a modern storage solution and it costs too much. Sure, it's got redundancy from the disc to the file server, but so what: modern file servers use redundant storage and clustered redundancy and don't need the diminishing returns of embarassingly expensive drives, head nodes, capacity licensing and annual support contracts. By the time you figure in oversubscribed ports in your FC network, you've lost the supposed reliable performance benefit of the whole thing. This isn't bad news for Cisco - they're going to sell a lot of 10Gbit Ethernet ports before they get cheap and they haven't lost anything by being also compatible with FC. It really bites to be EMC this week, but they'll figure it out.
Check the specs on this server [hp.com], this card [memory4less.com], this drive [newegg.com] and this array [hp.com]. This is off-the-shelf stuff, not pie in the sky. The interconnect people need to get off their butts, but this is all doable right now. The compute side becomes an almost trivial cost of what it takes to maintain this storage bandwidth and capacity. If you like proprietary solutions HP sells a thing called the LeftHand Virtual San App
Re:The straight dope (Score:3, Interesting)
Simply untrue. Oracle is still committed to BtrFS. In fact in one article I read, they quoted lead developers as saying that BtrFS fit Oracle's needs better than ZFS. This led to speculation that in the long term BtrFS would replace ZFS. Of course licensing issues would necessitate a clean-room implementation, likely. Probably what will really happen is that ZFS will remain with Solaris while BtrFS becomes the standard across the Linux world, and will obviously be heavily used by Oracle database users. The fact also is that BtrFS is a better design than ZFS. ZFS was revolutionary, no doubt about it. But BtrFS has more of a future and more potential due to its ingenious use of btrees in a way (I don't know the particulars) that was previously not thought to be possible, which is why ZFS did it differently. So while they are very similar file systems in effect and characteristics (COW, etc), under the hood they are very different and BtrFS seems to be technically superior, though lacking in features so far (no RAID-5 yet).
Most Linux kernel developers consider Ext4 to be a stopgap until BtrFS is ready. And it is being developed fairly intensely still, the lack of release notwithstanding. Once it is ready, I expect and hope a windows driver will surface, allowing it to be a more universal file format.
Re:The straight dope (Score:3, Interesting)
ZFS is a solution in search of a problem
ZFS is more or less a direct competitor to Veritas (VxFS).
If you know what later is, then you know where ZFS belongs.
ZFS is so much hyped mainly for two reasons: (1st) Veritas is f***ing expensive and (2nd) all Solaris file system(s) before sucked terribly. (Think of Win7 type of hype: it is so much better than Vista!)
Re:Why would MS bother? (Score:3, Interesting)
One small problem is that HFS+ sits on top of HFS and thinks like block allocations fall apart after 1TB. Apple has to switch filesystems. While the problems aren't severe at 2TB or 4TB at 50TB they are going to devastating.