Forgot your password?
typodupeerror
Apple

Apple Discontinues ZFS Project 329

Posted by Soulskill
from the stick-a-fork-in-it dept.
Zaurus writes "Apple has replaced its ZFS project page with a notice that 'The ZFS project has been discontinued. The mailing list and repository will also be removed shortly.' Apple originally touted ZFS as a feature that would be available in Snow Leopard Server. A few months before release, all mention of ZFS was removed from the Apple web site and literature, and ZFS was notably absent from Snow Leopard Server at launch. Despite repeated attempts to get clarification about their plans from ZFS, Apple has not made any official statement regarding the matter. A zfs-macos Google group has been set up for members of Apple's zfs-discuss mailing list to migrate to, as many people had started using the unfinished ZFS port already. The call is out for developers who can continue the forked project." Daring Fireball suggests that Apple's decision could have been motivated by NetApp's patent lawsuit over ZFS.
This discussion has been archived. No new comments can be posted.

Apple Discontinues ZFS Project

Comments Filter:
  • Great (Score:5, Funny)

    by tjones (1282) on Friday October 23, 2009 @08:03PM (#29853065)

    Now if you're using zfs on Mac OS, you can't complain if it loses your data. You already knew it was forked.

  • The straight dope (Score:5, Interesting)

    by Anonymous Coward on Friday October 23, 2009 @08:12PM (#29853107)

    Posting anon, lest someone guess who my sources are.

    The long and short of it was, Apple and Sun couldn't come to terms on the licensing. Sun wanted a lot of money for giving it to Apple under different terms and the amount they wanted was in the range of "hell, we could do it ourselves for that".

    Add to that, the Oracle buyout and Sun going into management paralysis, and Apple decided to go it alone.

    Apple's CoreOS team includes several of the lead engineers from the ZFS project (who fled the remnants of Sun in the Schwartz melt-down), and the architect of the BeFS. I'm expecting Apple to do their own next-generation file system, probably in the 10.7 timeframe.

    • Posting anon, lest someone guess who my sources are.

      The long and short of it was, Apple and Sun couldn't come to terms on the licensing. Sun wanted a lot of money for giving it to Apple under different terms and the amount they wanted was in the range of "hell, we could do it ourselves for that".

      That sounds odd to me. Why would they need different licensing terms? Especially as they apparently had a port that was already somewhat distributed, at least in developer builds - presumably they thought that the license allowed them to do that at the time?

      • by Culture20 (968837) on Friday October 23, 2009 @08:38PM (#29853269)

        Why would they need different licensing terms?

        They probably wanted to rename it without changing it. Apple likes renaming things. Microsoft OTOH, loves using the same name as everyone else, and changing stuff to break interop.

        • Haha, that did make me laugh :-) You compressed quite a lot of truth into a very small amount of text. There is a certain elegant symmetry to the two companies when you put it like that.

    • Re:The straight dope (Score:5, Interesting)

      by segedunum (883035) on Friday October 23, 2009 @08:44PM (#29853301)

      The long and short of it was, Apple and Sun couldn't come to terms on the licensing. Sun wanted a lot of money...

      That doesn't make any sense. I fail to see why Apple should agree licensing terms for a CDDL licensed open source project or how Sun could demand money for the privilege. Sun were positively overflowing with love towards Apple (as they usually are) when they heard that anyone would actually be interested in their uber new filesystem.

      • Re:The straight dope (Score:4, Interesting)

        by ThePhilips (752041) on Friday October 23, 2009 @09:31PM (#29853561) Homepage Journal

        Probably because Apple hadn't felt like supporting an FS on its own?

        Mac OS X already includes pile of licensed technologies. The sole purpose of that is to offload work from R&D so that they can do something more useful than reinventing a wheel.

        Licensing deal likely would have been needed so that if Sun/whatever goes tits up, Apple would retain all rights to the code so that they can develop and maintain it further on their own - without being in mercy of whoever buys Sun after that.

      • Re: (Score:2, Insightful)

        by seanadams.com (463190) *

        You have gravely underestimated the capacity for lawyers and bean counters to fuck up a great idea.

    • Re:The straight dope (Score:4, Informative)

      by BrentH (1154987) on Friday October 23, 2009 @08:52PM (#29853345)
      "The flip side is that I've heard that Apple's file systems team is full steam ahead on their own next-generation file system. And, perhaps not coincidentally, they're hiring." from http://daringfireball.net/linked/2009/10/23/zfs [daringfireball.net]

      This is pretty shitty because it'll fragment the momentum ZFS had in being the next-gen ubiquitous file system. When it was clear ZFS wasn't coming to Linux, those guys got btrfs going, now Apple is doing their own, while ZFS obviously will stay around too. Microsoft obviously wasnt on board for any of this, and without the momentum behind ZFS it never will. This nonsense isnt helping, and I think the best Oracle could do it release it under all the licenses that'll get it into OSX/Linux and perhaps even Windows. Can Oracle go over Sun's head on this or Sun==Oracle?
      • by dangitman (862676) on Friday October 23, 2009 @09:04PM (#29853421)

        Microsoft obviously wasnt on board for any of this, and without the momentum behind ZFS it never will.

        Microsoft is never on board for anything useful, so I'm not sure it really makes any difference.

      • Re:The straight dope (Score:5, Interesting)

        by Robotbeat (461248) on Friday October 23, 2009 @09:10PM (#29853455) Journal

        "The flip side is that I've heard that Apple's file systems team is full steam ahead on their own next-generation file system. And, perhaps not coincidentally, they're hiring." from http://daringfireball.net/linked/2009/10/23/zfs [daringfireball.net]

        This is pretty shitty because it'll fragment the momentum ZFS had in being the next-gen ubiquitous file system. When it was clear ZFS wasn't coming to Linux, those guys got btrfs going, now Apple is doing their own, while ZFS obviously will stay around too. Microsoft obviously wasnt on board for any of this, and without the momentum behind ZFS it never will. This nonsense isnt helping, and I think the best Oracle could do it release it under all the licenses that'll get it into OSX/Linux and perhaps even Windows. Can Oracle go over Sun's head on this or Sun==Oracle?

        (emphasis mine)

        Unfortunately, btrfs isn't "going" anywhere. Guess who their development was funded by? That's right, Oracle! Notice that they haven't released anything new since BEFORE Sun's shareholders approved the acquisition? (Latest release on the btrfs wiki is v .19, released in June 2009) It's not exactly improving at a breakneck pace... If btrfs is going to go anywhere, they need some real development money.

        Dang Oracle.

        • Re:The straight dope (Score:5, Informative)

          by setagllib (753300) on Friday October 23, 2009 @10:22PM (#29853781)

          You really need to subscribe to the mailing list. The rate of development is only growing -- it's just now moved on to a lot of smaller features and improvements, now that most of the work is already done.

        • Re: (Score:3, Interesting)

          by caseih (160668)

          Simply untrue. Oracle is still committed to BtrFS. In fact in one article I read, they quoted lead developers as saying that BtrFS fit Oracle's needs better than ZFS. This led to speculation that in the long term BtrFS would replace ZFS. Of course licensing issues would necessitate a clean-room implementation, likely. Probably what will really happen is that ZFS will remain with Solaris while BtrFS becomes the standard across the Linux world, and will obviously be heavily used by Oracle database users.

      • by jcr (53032)

        This is pretty shitty because it'll fragment the momentum ZFS had in being the next-gen ubiquitous file system.

        The momentum that has had all their best developers jumping ship since Schwartz got the CEO gig?

        Oracle bought Sun for Java. I see no indication that Oracle cares about ZFS or any of Sun's other technologies.

        -jcr

      • ... it'll fragment the momentum ZFS had ...

        After reading an opinion piece of one of the ZFS authors about Btrfs, I stopped thinking that ZFS deserves to have the momentum.

        Can't find the link, but ZFS (as first Sun's take on files system) has made several design mistakes which cannot be fixed without redesign. Developed later Btrfs learned on that and avoided the mistakes.

        IOW, ZFS might have had a very short momentum, but thanks to the licensing + general insanity of Sun management (which managed in past decade to lose all talented people) I'd

      • by toby (759) *
        I predicted [twitter.com] that they were working on a ZFS-alike on 2 Sept. NIH Syndrome [google.com] does seem the most likely explanation. Which is disappointing. Cooperation on ZFS seemed a natural and powerful cross-endorsement for both Apple and Sun.
        • Re: (Score:2, Insightful)

          by jcr (53032)

          How can it be NIH syndrome when the people implementing the replacement are former Sun ZFS developers?

          -jcr

          • Re: (Score:3, Interesting)

            by toby (759) *

            For the reason I stated: That using Sun's ZFS left them without control of development, and tracking an outside codebase has reputational risks to which Apple in particular is averse. Having ex-Sun people work on a new filesystem is great, but they still need to navigate the patent minefield that Sun has sown around ZFS.

            Interesting that Sun non-competes did not stop their engineers walking down the street to work on directly competitive technology... (First I heard that engineers left Sun for Apple, actual

      • Re: (Score:3, Insightful)

        by jone1941 (516270)

        I'm sorry, perhaps I'm just a bit dense, but what is the benefit of a "ubiquitous file system" that is largely targeted for server infrastructures. Generally speaking I agree that parallel efforts to accomplish a similar / identical task can be deemed wasted efforts on some level. However, that trend is pretty much the standard for all open source projects? Linux vs BSD, WebKit vs Gecko, mysql vs postgres, php/perl/python/ruby the list goes on and on. There are a multitude of reasons projects (corporat

    • by saleenS281 (859657) on Friday October 23, 2009 @08:55PM (#29853365) Homepage
      Why would Apple need different terms? CDDL and BSD are compatible, hence FreeBSD integrating ZFS. Furthermore, Apple already integrated DTRACE under the CDDL. Claiming a licensing issue doesn't make sense... at all. The only thing that does make sense is that Apple was trying to add a bunch of proprietary code to ZFS and didn't want to release their changes. Boo hoo.
  • by segedunum (883035) on Friday October 23, 2009 @08:15PM (#29853133)
    I doubt that it's a legal issue as the primary reason that this has happened, especially considering that the project seems to have stagnated steadily in successive versions of OS X. There just doesn't seem to have been the will within the OS X development group to make this work and to support and fully integrate ZFS into the inner workings of the OS. Given the pretty extensive functionality and plumbing of ZFS its probably been too much of a big ask to integrate a filesystem like that into a desktop. They might well have come to the conclusion that ZFS was simply complete overkill on a desktop and that it just wasn't possible.

    However, they still desperately need a next generation filesystem and according to the linked article they're hiring filesystem engineers. I don't see any evidence that this was anything other than a technical avenue that they've explored that has fallen by the wayside as so many have before.
    • by jcr (53032) <jcr@mac. c o m> on Friday October 23, 2009 @08:17PM (#29853147) Journal

      There just doesn't seem to have been the will within the OS X development group to make this work and to support and fully integrate ZFS into the inner workings of the OS.

      I can't agree with that. Spend ten minutes with the filesystem engineers at WWDC, and you wouldn't come away thinking there was any shortage of will to make ZFS fly on the Mac.

      -jcr

      • by segedunum (883035)

        Spend ten minutes with the filesystem engineers at WWDC, and you wouldn't come away thinking there was any shortage of will to make ZFS fly on the Mac.

        The proof of the pudding, as they say. By any stretch of the imagination ZFS within Mac OS is a project that has stagnated and gone completely stillborne from 10.5 and onwards. It's not something you could ever consider using semi-reliably, nor did you get the impression that it would ever reach that stage.

        I'm sure there was no shortage of soundbites and

    • Re: (Score:2, Insightful)

      by mysidia (191772)

      However, they still desperately need a next generation filesystem and according to the linked article they're hiring filesystem engineers.

      That doesn't make any sense.

      It only makes sense to engineer a new filesystem if the other options are inadequate or unusable.

      Engineering a new filesystem is hard and expensive.

      For them to seek to do that, they must have rejected the effort to integrate ZFS for some technical reason.

      The complexity of integrating ZFS pales into comparison to the massive cost of e

      • Re: (Score:2, Funny)

        by segedunum (883035)

        That doesn't make any sense. It only makes sense to engineer a new filesystem if the other options are inadequate or unusable.

        It makes perfect sense. HPFS is simply far too long in the tooth now.

        For them to seek to do that, they must have rejected the effort to integrate ZFS for some technical reason.

        Yer................

        The complexity of integrating ZFS pales into comparison to the massive cost of engineering and implementing a new filesystem from the ground up.

        What's even more difficult is to integrate a

        • by segedunum (883035)

          HPFS is simply far too long in the tooth now.

          ???????!!!!! Far too late here now. That should of course be HFS(+).

      • Re: (Score:3, Insightful)

        by jcr (53032)

        Engineering a new filesystem is hard and expensive.

        Sure, but when you have a team on hand that knows how to do it, and has been through the development of several different filesystems between them, why not build your own?

        I think the chance of Apple wanting to engineer a new FS so lightly are pretty slim.

        Who says they're doing it lightly? Have you seen who they've got in that group?

        -jcr

    • I doubt that it's a legal issue as the primary reason that this has happened...

      I've dealt with people in Sun who were close to ZFS and who were also excited to have it in Mac OS. It wasn't pulled because it wasn't technically ready.

      ZFS is the next generation file system that all others will have to live up to. I've never felt compelled by any file system. Use whatever is there or whatever my peers are comfortable with. ZFS is the first file system that's compelling enough to make me take a stand. I use it on servers daily at work, and I was looking forward to having it on my Macs

      • Amen... I was running a file-level checksum database in a vain attempt to detect bitrot until I discovered zfs. To be honest, I think it would be fairly easy to add bitrot detection in any existing fs (for a developer of course, not me)... Just add a hash to each sector. but you wouldn't get all the robust goodies baked into zfs.
      • by segedunum (883035) on Friday October 23, 2009 @09:25PM (#29853529)

        ZFS is the next generation file system that all others will have to live up to.

        I'm sure it will, but I'm afraid that doesn't mean that made it practical for Apple to integrate into OS X or that it fitted the use cases they needed for many desktop scenarios. The FreeBSD people still haven't been able to run and integrate it reliably.

        I use it on servers daily at work, and I was looking forward to having it on my Macs at home. Bit rot is a very real problem. ZFS handles it automagically.

        The ZFS advocates trot those lines out every time and they're total nonsense. Ultimately, the only way to deal with silent data corruption or 'bit rot' is to have multiple levels of redundancy several times over for your data - which ZFS has and deals with. No desktop Mac can ever have that. Anyone who thinks that is anywhere near being practical to deal with on a desktop system is an idiot, and no, I'm afraid booting OpenSolaris with ZFS on your desktop system at home and not having it crash and burn does not even approach the kind of issues and corner cases that Apple's engineers will have to deal with, especially in a desktop system like OS X.

        By no stretch of the imagination does ZFS handle this 'magically'. There is a severe price to be paid. If you don't have redudancy then you will simply risk losing your ZFS pool if there is corruption.

        What handles failure at the data level? Nothing. Hope you make backups of your arrays.

        I'm afraid that hardware, bad sector and disk issues are far, far more prevalent problems than data corruption at an OS level. Many apparent corruption issues at the OS level are usually down to hardware issues somewhere down the line. It might be a problem for operating systems with fairly shitty and poorly maintained disk and controller device drivers with a poor history on x86 and widely used hardware (hello Solaris!) but I'm afraid it's just not a primary concern for everyone else or for those developing desktop operating systems.

        • by 4iedBandit (133211) on Friday October 23, 2009 @11:38PM (#29854033) Homepage

          I'm sure it will, but I'm afraid that doesn't mean that made it practical for Apple to integrate into OS X or that it fitted the use cases they needed for many desktop scenarios.

          Um, the technical work was already done. It could have shipped with Snow Leopard. Again, the reason it didn't has nothing to do with the technical feasibility of it.

          Ultimately, the only way to deal with silent data corruption or 'bit rot' is to have multiple levels of redundancy several times over for your data - which ZFS has and deals with. No desktop Mac can ever have that.

          Why? Because you say so?

          Anyone who thinks that is anywhere near being practical to deal with on a desktop system is an idiot

          While I may be an idiot, you have to convince me that ZFS is not practical for a desktop. Again, just because you say so is not reason enough. I stand by my statement that ZFS is the only file system with enough benefit to make me explicitly choose it for building servers. You may argue that there's a difference between a server and a desktop but those really are nothing more than abstract concepts. A file system that has too much overhead for my desktop has too much overhead for my servers. Performance matters. ZFS may not be the fastest, but it is no slouch either and the other benefits it brings to the table far outweigh miniscule performance concerns.

          By no stretch of the imagination does ZFS handle this 'magically'. There is a severe price to be paid.

          What exactly is this severe price? Can you spell it out? Exactly? "Any sufficiently advanced technology is indistinguishable from magic." In that respect yes I will say it is magic because it is head and shoulders more advanced than anything else I've had the pleasure of working with. File systems have not had this kind of improvement in decades.

          I'm afraid that hardware, bad sector and disk issues are far, far more prevalent problems than data corruption at an OS level...but I'm afraid it's just not a primary concern for everyone else or for those developing desktop operating systems.

          How do you know? It's not a significant problem till the data you need is unavailable when you need it. At home my own modest media library sits on just 500 Gig with no guarantee that any of it will still be whole in 6 months. Sure I back it up. Routinely. But until you access the file you don't know if it's been corrupted. Then how long as it been corrupted? Do your backups go back far enough to compensate? Yes you can checksum everything routinely and maintain a database of checksums to validate file change. Part of the beauty of ZFS is it does this with every thing you put in it, at the block level, and it validates the checksum every time you read the data. If a block fails the check, it not only sends you the valid block from the mirrored copy (You do have redundancy right? Even ZFS won't save you if you only have one copy.) but also replaces the bad block with a copy of the good one.

          Storage capacity is skyrocketing. Going to backup to fix problems is a real problem in itself. Are the tapes on-site? Do we have to go to the vault to find an uncorrupted copy? Did the media pool get recycled and now there is no uncorrupted copy? Do you enjoy explaining to an executive why the data they want is unavailable despite spending millions on enterprise class storage and backup solutions? The problems of enterprise storage are becoming problems of home users. I have three terabytes of storage just to backup my home system in a replication layout I'm okay with, but I really would have loved the protection ZFS offers against bit rot to top it off. Stick your head in the sand if you want, but I consider my data and it's availability a little more important. ZFS handles it elegantly, in the background, with negligible performance hit.

  • by KillNateD (31007) on Friday October 23, 2009 @08:16PM (#29853143)

    Dustn Sallings put the code on Github and has already hacked some basic Snow Leopard support and a minimal installer:

    http://dustin.github.com/2009/10/23/mac-zfs.html [github.com]

    Code's here, fork away:

    http://github.com/dustin/mac-zfs [github.com]

  • by Lemming Mark (849014) on Friday October 23, 2009 @08:30PM (#29853221) Homepage

    Interesting - we're chugging happily along in Linux / Windows / Mac / Unix land having a load of competing filesystems where all the popular ones have *roughly* similar capabilities. Then ZFS appears in OpenSolaris and filesystem design becomes cool again. Everyone starts either porting ZFS or making filesystems with similar features ... Now a major player that actually *had* ported ZFS (somewhat) is seemingly deciding to go it alone. It seems as though the next-gen filesystem space is also going to have a variety of competing filesystems.

    I generally think this is a good thing, lets just hope that a reasonable degree of interoperability becomes possible anyway.

    • Luckily, networking and filesharing systems are robust enough to make interoperability a minor concern these days. It's not like we're exchanging zfs floppies.
      • Unless you want to dual boot and share /home; it's a royal pain in the ass between BSD / Linux / Solaris ; then your only real option is ext2. Add windows and it becomes very hard; you're pretty much down to using NTFS or fat32 and then doing hackery with the mount options and even THEN it only really works if you have one partition for each user who's going to have a $HOME shared between OSes. (And this is the only reason why I ONLY boot linux)
      • by Jesus_666 (702802)
        We are, however, going to exchange SDXC cards. And those come with a next-gen filesystem. With extremely limited interoperability (read: none). If we had something better than FAT32 that actually worked on all OSes we could just ignore the specification and use that, even if it's something as overblown (for flash cards) as ZFS.

        It seems that FAT32 is still the apex of filesystem interoperability, closely followed by NTFS, courtesy of reverse-engineered drivers. That's kind of sad.
    • Yes, I realize it contains the letters "FS", but ZFS is not just a filesystem. It incorporates a lot more than that. That's one of the reasons it's really hard to integrate into an OS, given the architecture of most OS's.

      • True, although a lot of the characteristics I find particularly interesting about it relate to the filesystem layer. Various of the "competitors" are incorporating functions that are also (to some extent) outside the normal remit of the filesystem. Btrfs in particular, however its volume management and RAID-ing functionality is not used outside of that filesystem. Whereas I believe ZFS can subsume most storage management, although AFAIK only Solaris really takes this idea to its logical conclusion. ZFS

  • by mysidia (191772) on Friday October 23, 2009 @08:46PM (#29853309)

    Hearing that ZFS support was upcoming in Snowleopard is one of the things that encouraged me to switch my desktop from Windows XP to MacOS.

    It is an understatement to say i'm disappointed to see Apple abandoning this.

    Support for ZFS is not just a little feature checkbox, it's a major component of the OS.

    It'd be like if Microsoft dropped/cancelled support for Solitaire from Windows....

    • I suggest you drop MacOS like a hot potato, send a nastygram to Apple giving them a piece of your mind, and check out both OpenSolaris and FreeBSD. They both support ZFS, OpenSolaris because Sun invented ZFS, and FreeBSD because they have competent management AND engineering. Unlike certain others (and I'm not pointing the finger at linux).

      Once again, FreeBSD has shown the fools in Cupertino how it's done.

      • by exley (221867)

        Once again, FreeBSD has shown the fools in Cupertino how it's done.

        Yeah, Steve is gonna cry himself to sleep on a huge pile of cash tonight.

        (Not sure if parent was a troll or totally serious... It's hard to tell around here sometimes!)

    • I know you're only joking, but I'm actually using zfs on a sun cluster right now... whatever technical merits it might possess are lost on me because when I type in "df -h" and it doesn't report anything meaningful -- it always shows 2 GB free however much of my 1 TB disk quota I happen to be using. To get meaningful disk usage, you have to type in "df -h .". While it's certainly trivial to do that, I find that having to directly specify the directory in order to get meaningful disk usage data is just wei
    • Re: (Score:2, Funny)

      by gomek-ramek (1340625)
      It would have been like Microsoft dropping WinFS [wikipedia.org] for Vista, right?
      • Re: (Score:3, Funny)

        by TheRaven64 (641858)
        Why do you need WinFS? OFS that came with Cairo already did everything you might need...
  • by mbessey (304651) on Friday October 23, 2009 @08:51PM (#29853335) Homepage Journal

    Leaving aside all the crazy storage pool stuff (great for servers, not necessarily that useful for desktops), there are some interesting features in ZFS that I hope make their way into Mac OS X in some filesystem.

    Snapshots and Copy-On-Write filesystem clones seem like a great way to improve the Time Machine backup feature, and would make it easy for applications to provide backup-on-save very efficiently.

    The compression and encryption features would likely be useful for some people. I don't think the increased filesystem limits (number of files, size of files) would matter for most folks.

  • by Lemming Mark (849014) on Friday October 23, 2009 @08:58PM (#29853381) Homepage

    http://en.wikipedia.org/wiki/HAMMER [wikipedia.org]

    Should be under a suitable license for their usage. It's written for DragonflyBSD which has a funny filesystem driver interface but AIUI the developer had ports to other OSes in mind, so it should still be doable. It can do cheap filesystem snapshots so it would support Time Machine-style operation well. The question is whether it could be adapted to fit Apple's uses well enough. Given one of the linked articles suggests Apple are hiring FS developers my guess would be that they've decided they'd rather build a ground-up filesystem that supports all the (slightly odd set of) features MacOS X wants.

  • Hopefully, whatever Apple develops next will. It would really suck if the only persistent way to refer to a file was its path.

The bugs you have to avoid are the ones that give the user not only the inclination to get on a plane, but also the time. -- Kay Bostic

Working...