ZFS Shows Up in New Leopard Build 351
Udo Schmitz writes "As a follow-up to rumours from May this year, World of Apple has a screenshot showing Sun's Zettabyte File System in "the most recent Build of Mac OS X 10.5 Leopard". Though I still wonder: If it is not meant to replace HFS+, could there be any other reasons to support ZFS?"
Zettabyte? (Score:3, Informative)
Exciting! (Score:4, Interesting)
--jeffk++
Re: (Score:2)
Re: (Score:2)
Otherwise... (Score:5, Funny)
Because if Apple showed them before, there was a risk that Microsoft tried to announce them as future features in their soon-to-be-released perfect Windows Vista ?
Re:Otherwise... (Score:5, Funny)
--jeffk++
Re: (Score:3, Insightful)
Re: (Score:2)
copy-on-write (Score:4, Insightful)
Re: (Score:3, Informative)
There's no real change here for ZFS, and it's unlikely that anything at the memory cache level even knows about the copy-on-write-ness of ZFS (or even cares).
a
Re:copy-on-write (Score:5, Interesting)
Mmap simple maps pages of a disk file into memory. If the disk file changes its physical location then the mapping is updated. When you call mmap, you give it a disk file, an offset, and an extent. It is up to the VFS layer to translate this into physical mappings. LFS has the same issues, and these were solved well over a decade ago.
If you invoke mmap with MAP_PRIVATE, this actually makes it easier; if someone else updates the file then you just keep the existing mapping.
Re: (Score:3, Informative)
Similar issues exist without problems when mmap()ing files from NFS. You cannot update just a few bytes with NFS, you have to write the whole disk block out.
I'm fairly confident that the current "standard" way to implement mmap at the moment is to update the pages, mark them dirty, and let the VM subsystem write them to disk.
I haven't had to look at mmap's implementation in a long time, though.. but IIRC Rich Teer and/or Adrian Crocroft had good articles about it a few years back.
Obviously, I arrive
Re:copy-on-write (Score:5, Interesting)
Makes use of copy-on-write; rather than overwriting old data with new data, it writes new data to a new location and then overwrites the pointer to the old data
It may do, but like many things there are alternative approaches.
From working on embedded hardware with flash memory, this makes me wonder whether possible addition of ZFS is meant to be for flash storage? Let me explain: flash memory has a fairly limited write-count, relative to hard disks, so to compensate for this memory is written in a circular fashion, to ensure that a given sector is written the least often possible. In addition to this, from what I can tell, Apple's main sales point are low profile computers and portables. The latter would benefit from flash storage as means of extending battery life, even if it is for a certain elements, such as for the OS which is accesed far more frequently than anything else on disk. Given this I wouldn't be surprised to see flash memory in future models of Apple portables, using ZFS, while HFS+ is still used for the hard disks.
This is pure speculation, but I feel that it has a high probabilty of being near the mark.
Re: (Score:3, Informative)
Secure Delete? (Score:4, Insightful)
I think it would pose a problem for secure deletes. Try to obliterate a file by overwriting it with garbage, you end up writing somewhere else instead? Would the next overwrite attempt get the original location or would you have to write enough garbage to cycle over all the free space of the volume? Considering how large these volumes can get, that's a lot of boiled oceans [wikipedia.org] for a multi-pass secure delete.
Re: (Score:3, Insightful)
What a moron (Score:2, Insightful)
Duh... It's called compatibility.
Re:What a moron (Score:5, Insightful)
Wouldn't full NTFS support (or well, support for any FS more in use then ZFS today) make more sense?
Re:What a moron (Score:5, Insightful)
Yeah, I mean it's not like NTFS is defined and controlled by an organization renowned for its hostility to other platforms, reluctant to document things in a way that other people can implement them, and scared of interoperability, is it?
Re: (Score:2)
Re: (Score:3, Funny)
Re: (Score:2)
It's frustrating that OSX doesn't know how to handle these.
Re:What a moron (Score:5, Informative)
Re: (Score:2)
I think this is a first step. Initial ZFS support now for non-boot volumes and then when boot volumes are supported, they have the option to move to the new format.
Reasons to support? Servers (Score:5, Interesting)
Re: (Score:2)
Re: (Score:3, Informative)
http://www.apple.com/xserve/management.html [apple.com]
Or this
http://docs.info.apple.com/article.html?artnum=30
Re:Reasons to support? Servers (Score:4, Insightful)
By the way it's nice to see dtrace, open source Java, and now ZFS coming out of Sun recently. I almost feel sorry for how little they get out of a lot of their innovations, they remind me of Bell Labs just before they died.
Re: (Score:3, Informative)
[ZFS] will be implemented for Linux pretty quickly.
*sigh*. I wish. ZFS is being implemented on FUSE. This automatically creates limitations in performance and function (no root ZFS). IMO ZFS on FUSE will be a no starter in production.
I don't think we'll see ZFS in the kernel proper either, given the history of incorporating XFS and ReiserFS 4. Along the same lines, DTRACE will probably never make it in. It's being cloned in the form of Systemtap.
Meanwhile, FreeBSD has been porting ZFS and DTRAC
Re: (Score:3, Insightful)
Re: (Score:3, Funny)
It won't survive a marriage to Hans Reiser!
Re:Reasons to support? Servers (Score:4, Informative)
That is profoundly wrong. Vanilla RAID will not discover and cannot automatically correct silent data loss. The reason is that RAID has no way of knowing which data is correct. For example, if two mirrored copies disagree on the contents of a block, the data is unrecoverable without manual intervention or external knowledge. Furthermore, in normal operation your RAID subsystem will simply read data from whichever drive is idle at the time the read request comes in; it does not ordinarily compare the two mirrors. The data will remain corrupted until the user notices a problem, at which point they have no practical recourse. Essentially the same problem occurs with parity RAID.
There is no dedicated hardware in your system that provides the end to end data integrity that ZFS does. I honestly suggest you learn more about it before airing your opinions. Here is a start:
http://blogs.sun.com/bonwick/entry/zfs_end_to_end
Re: (Score:3, Informative)
Or, you know, a checksum. Or more than one level of redundancy.
I agree that RAID-1 cannot, by itself, correctly recover from error-free reads of mis-matched data. But RAID 5 and 6 are both capable of verifying the primary data source against the parity data and transparently correcting errors that occur on less than the critical number of disks. In the common configuration
Re: (Score:3, Insightful)
With RAID-5, as with RAID-1, the critical number of disks is one. RAID-5 cannot transparently correct errors that occur on even a single stripe, unless you know a priori which stripe is affected.
With RAID-6 you can automatically correct errors that occur on a single stripe, but it still does not automatically detect such error
Re: (Score:3, Insightful)
The underlying hardware will not necessarily notice errors. Hard drives are only designed to error protect the magnetic domains on the disk. There are all sorts of other places in the increasingly long datapath to disk where data can be lost, and, in fact, routinely does get lost.
The choice to verify every read is purely an implementation decision
RAID-6 does not
For some reason... (Score:2, Interesting)
Re: (Score:3, Funny)
Not a likely replacement... (Score:5, Insightful)
Re: (Score:2)
I forget the reason now, but point is, UFS it may claim to be, but if you're using UFS for compatibility, sucks to be you.
Re: (Score:3, Insightful)
Re: (Score:2)
Yeah - Linux lists 8 different supported or semi-supported variants (which have to be selected between using a mount option - no auto-detectio
Re: (Score:2)
Re: (Score:2)
http://www.kernelthread.com/mac/osx/arch_fs.html [kernelthread.com]
Obligatory (Score:3, Informative)
ZFS is overkill for a laptop - for now (Score:5, Interesting)
- They had implemented everything I thought they should, and
- That only accounted for about 40% of the features of ZFS.
Calling it the last word in filesystems might be hyperbole, but I expect ZFS to last a good 10-20 years, which is quite respectable for a filesystem, and I wouldn't be surprised if it lasted longer. Is it a replacement for HFS+? Not yet.HFS+ is a very nice filesystem for single user systems with a single disk. It implements journalling, has reasonable performance, and has good metadata support. For the average users at the moment, the only real advantage of ZFS would be snapshots, and these are not too difficult to implement for other filesystems.
ZFS, however, is much better when you have multiple physical disks. At the moment, only the top-end Macs have more than one disk. This is likely to change in two ways:
- Cheap flash,
- Network storage
For a home user, ZFS could handle backups trivially by plugging in a large flash drive and adding it to the pool. I suspect this will be one mechanism Time Machine will use. Due to the way ZFS works, you can just mirror a part of the directory tree (e.g.ZFS is not needed as a replacement for HFS+ in 2007, but it probably will be in 2008-9. ZFS is a 128-bit filesystem, which means it is designed to last for a long time. We will probably never need a 128-bit filesystem (unless we actually want to build hard drives the size of planets with single-atom sectors), but we will need a 65-bit filesystem once we get to around 10 Exabytes. This won't happen with single drives for a while, but it will with RAID arrays.
Re: (Score:2)
Most (if not all) posts seem to direct this to a Mac Book Pro or something. I don't get it.
Re: (Score:2)
Which would spend 10 million years to come up with "What do you get when you multiply six by nine" [wikipedia.org], no doubt
Re: (Score:2)
I think ZFS is probably for their servers and workstations, especially when you connect to stuff like the Xserve RAID.
Re:Storeage size - speak for yourself (Score:5, Funny)
ZFS vs HFS vs NTFS? (Score:5, Insightful)
I've never found plain-Jane posix permissions to be all that useful on anything other than the most basic of server environments.
HFS has going for it all the fun stuff we've come to love apple for, such as transparent file customization like icons, labels, meta data, and whatnot through resource forks. I assume that these can be made to work with ZFS by making hidden files.
What I'd really like to see is both that kind of functionality along with NTFS's really excellent ACL permission system implemented. ACL permissions are a godsend for people responsible for running a file store that's used by humans as opposed to automated processes. NTFS also has a great deal of capacity for meta-data, although not to the same level as HFS.
NTFS is one of the few worthwhile things that's ever come out of Redmond. I wish more people would spend a bit learning from it without throwing it away simply because it's MS bloat.
Re:ZFS vs HFS vs NTFS? (Score:5, Insightful)
I wish MS would let us. NTFS is worthless if you don't run Windows. And it hinders interoperability with other systems because its implementation and disk layout is secret/patented.
Why, do you think, there is no stable implementation that can write NTFS volumes (other than the MS implementation)?
Contrast this with ZFS which is released under an open source license.
Re:ZFS vs HFS vs NTFS? (Score:5, Insightful)
Re:ZFS vs HFS vs NTFS? (Score:5, Informative)
What I'd really like to see is both that kind of functionality along with NTFS's really excellent ACL permission system implemented.
I wish you could read more about ZFS before suggesting how you could improve it by adding ACLs. It already supports them!
http://blogs.sun.com/marks/entry/zfs_acls [sun.com]
Re:ZFS vs HFS vs NTFS? (Score:5, Insightful)
You assume correctly, since most of that business is taken care of with Bundles. This is why it more or less works on UFS, which is already supported on Mac OS X, and has been for years. Forks & whatnot are really a legacy idea.
That's funny! The HFS+ ACL system is Microsoft's ACL system, much to the chagrin of the Unix community.
Re: (Score:2)
Using hidden files for the resource forks would also make backups WAY WAY easier and more reliable than what is available now [plasticsfuture.org].
Re: (Score:3, Interesting)
Mac OS X Server 10.4 (Tiger) already has this. See: http://www.apple.com/server/macosx/fileprint.html [apple.com]
There is a "File Services" white paper linked off of he above page but here is the relavant marketing:
Re:ZFS vs HFS vs NTFS? (Score:4, Informative)
NTFS's ACL system is horrible. While it has a lot of descriptive power, it's a pain to manage, the result being that it is almost never used. The old Unix model, while simple, is easy to manage, and as a result is often set up reasonably. Novell's "Trustees" model works much better than either, but for some reason it wasn't adopted by others.
NTFS is slow and inefficient, fragments horribly, and lacks fundamental features such as proper symlinks (and only supports directory hardlinks). It has a reasonable journal implementation, and it supported large files before other systems did, but it's very outdated and does not compare favourably with any of the modern high performance file systems.
Re: (Score:2)
And what kind of expert are you exactly?
NTFS ACLs are confusing. As useful as they may be, real human beings don't know about them or care about them. You know, the kinds of real human beings that use workstations.
Never mind that ZFS actually does sup
It's to support Time Machine (Score:5, Interesting)
So it's about the snapshot ability of ZFS, and that's exactly what will be needed for Time Machine [apple.com].
There's a LOT more to ZFS than snapshots... (Score:5, Informative)
Over past months, I've read a lot of people commenting on ZFS who have no idea what it is. What it is, is the next generation of filesystems, not a "tweak" of current fs technology. It just happens to "look like" an ordinary POSIX fs, from a distance (if you ignore the administration/pool stuff...) But inside, it's something new under the Sun, folks.
RAID experts don't grok it, because it does things RAID can't do (end-to-end).
Devotees of ext2fs, reiserfs (yay!), NTFS (LOL!), or HFS+ don't grok it, because none of those filesystems do what ZFS does.
Read about it before you write it off as old wine in a new bottle. To ask the question, "Does OS X need a new filesystem?" is a perfect example of missing the point. Once you've looked at what ZFS really brings to the table, you'll see why it's an inevitable future, sooner or later, and you'll stop looking foolish.
Some links I posted this week: [google.com]
- http://www.osnews.com/story.php/16739/Screenshot-Z FS-in-Leopard [osnews.com]
- http://mac4ever.com/news/27485/zettabyte_sur_leopa rd/ [mac4ever.com]
(older rumour http://www.osnews.com/story.php?news_id=14473 [osnews.com])
For OS X people wondering why the fuss about ZFS - summaries include: - http://www.sun.com/2004-0914/feature/ [sun.com] - http://www.sun.com/bigadmin/features/articles/zfs_ part1.scalable.html [sun.com]
"Why ZFS for home": - http://uadmin.blogspot.com/2006/05/why-zfs-for-hom e.html [blogspot.com]
"Here are ten reasons why you'll want to reformat all of your systems and use ZFS.": http://www.tech-recipes.com/rx/1446/zfs_ten_reason s_to_reformat_your_ [tech-recipes.com]...
And some more technical explanations from Chief Engineer: - http://blogs.sun.com/bonwick/entry/zfs_end_to_end_ data [sun.com]
- http://blogs.sun.com/bonwick/entry/smokin_mirrors [sun.com]
Addition (Score:2)
And then I notice [wikipedia.org] that the official name of ZFS is ZFS these days: "The name originally stood for "Zettabyte File System", but is now a pseudo-initialism." Someone should tell Apple.
P.S.: What about my rewritten in cocoay goodness Finder? Pretty please?
ZFS + Timemachine (Score:3, Interesting)
...so how does one define "capacity" therein? (Score:4, Interesting)
for starters, does the FS "know" that i've just clicked "Save As" in my word processor? what about copy and pasting a file back into the same directory to make a local copy? Also? is it just within variations on the same file? if i have a particular setup exe on my system but forget, and download it again to the desktop surely the FS has no initial way of knowing that they are one and the same, does some funky heuristic happen?
basically: does the OS's read/write/copy/delete functionality have to invoke copy-on-write via a FS API or is it built in for every single sector-sized chunk that gets stuffed into the FS?
the next question is the one in my subject: how therefore do you define "capacity"? if i've got a bunch of files that take up 700mb on a ZFS device and try to back up to a (Joliet) CD will i get a message telling me that the CD doesnt have room? i can imagine this scenario being unlikely with optimised binary data (jpegs and mpegs) but if i'm backing up a dev environment with autobackups (main.c,main.c.bak.001,main.c.bak.002,etc.) and manually created and dated directory tree "snapshots" (dev,dev_backup_2006-12-18,dev_backup_2006-12-01,
Re:...so how does one define "capacity" therein? (Score:4, Informative)
Yes, I used something similar to ZFS for mass document storage a few years back. You do a complex checksum on the block level. Any two blocks with the same sum are the same. Each unique block is only stored once, though multiple files might link to it. You're right the file system doesn't know why you are using the same blocks over and over, but it doesn't care.
if i've got a bunch of files that take up 700mb on a ZFS device and try to back up to a (Joliet) CD will i get a message telling me that the CD doesnt have room?
Assuming you have repetitive block, yes.
What? Of course it is (Score:3, Interesting)
The answer is that it probably is meant to replace HFS+, but since ZFS is not bootable yet (including for Solaris 10) Apple can take the time to introduce ZFS, build tools for easier management, and let people get familiar with the FS before they have to drop HFS+.
HFS' lifetime has already stretched far beyond what it should have, it's time for Apple to think of its next generation FS, and ZFS is an extremely promising FS with heaps of amazing features Apple has already started to integrate into its UIs with Leopard (Time Machine + ZFS Snapshots anyone?).
ZFS also shows strong promises as both a home and a server FS.
Re: (Score:2)
http://blogs.sun.com/tabriz/entry/zfs_boot [sun.com]
Some reasons... (Score:2)
And last but not least:
On a side note, I've read a lot about how ZFS is supposed to be reliable and flexible as hell. Nowhere, however, have I found information about read/write performance. I guess it's a great filesystem for huge mission-critical datasets, but for anybody else? I have my doubts.
Or can somebody enlighten me?
Re: (Score:2)
Re: (Score:2)
It's not really that slow. Writing to the disk sounds bad, but when you understand that writing can occur less frequently with ZFS, it actually ends up feeling faster.
It of course, doesn't hurt that HFS and UFS are both painfully slow to
Xsan (Score:2)
ZFS is still overkill for most home computing needs, so I'm not sold on it being the default filesystem for OS X installs. My first guess is that it's going to be an option for network-acceccible storage. Somet
Checksumming != no data corruption (Score:2)
"Finally the most important feature of data integrity os check summing and everything on ZFS is checksummed meaning zero data corruption."
Umm , last time I looked checksumming merely told you if there WAS data corruption , it doesn't prevent it. Does ZFS has some other method to prevent data corruption (eg for a gone bad sector etc) or does it just flag a file as corrupted and leave it up to you what to do next?
Re: (Score:2)
They already have UFS, and don't use it... (Score:3, Interesting)
In fact HFS+ is *so* bad that if it wasn't for a couple of apps that absolutely freak out if they don't have their pet un-emulated feature I would have gone to UFS long since... even if I lost Spotlight completely. Until my Mac I had never run into a file system that wasn't so badly damaged as to be unbootable that coudn't be repaired by fsck... but apparently with HFS+ just running it "too full" can trash it, and I lost my system disk on my old G4 three months running because of that!
So I wouldn't hold out any expectations of ZFS being implemented in any useful way. They already have better file systems than HFS+ and they're not using them.
any reason (Score:2)
Could it be that the guy was raised on Windows ? Or what ? I mean, in the real world, there are more file systems that the one which your OS uses by default. And adding native support for them is not a questionable move, it's an applaudible move.
Hell, I remember what a happy day it was when I found crossmeta's xfs reading tools for Windows. Things like this shouldn't be a thing to raise an eyebrow for. What you all should raise your eyebrows for is why some OS
Maybe I'm just paranoid... (Score:2)
Holy crap! I asked for this at WWDC '06! (Score:2)
I wonder if this was one of the "new APIs" they were talking about at the Time Machine session I attended...
Send/receive useful for .mac sorts of features (Score:4, Interesting)
Any drawbacks to using ZFS? (Score:2)
The main benefit I saw is the 'snapshot' support (copy on write). I used this a long time ago on AFS, and found it very handy.
So, are there any compelling reasons _not_ to use ZFS?
Re: (Score:2, Informative)
Re: (Score:2, Informative)
Re: (Score:3, Funny)
Last I looked, they said, "NO! NO ZFS! IF YOU TRY, THE MOONS OF URANUS WILL CRASH INTO PLUTO!"
Re: (Score:3, Informative)
http://arstechnica.com/staff/fatbits.ars/2006/8/1
Re: (Score:3, Informative)
Re:Just to get it out of the way... (Score:5, Informative)
Re:Just to get it out of the way... (Score:4, Funny)
Re: (Score:2)
Rules of grammar FTW!
If he had meant "former" modify "funny," it would be an adverb, "formerly". It's not, so it can't, and it doesn't.
Re:Just to get it out of the way... (Score:4, Informative)
Re: (Score:2)
Re: (Score:2)
Yes, that's the typical Apple solution: you can sort of use it, but if you really want to use it, you have to commit to using OS X. It's not a good proposition.
I am pretty confident that a Linux driver for ZFS will emerge, and in the long run, I wouldn't be at all surprised if it ended up being a very common filesystem on Linux systems.
I seriously doubt there will be an independent implementati
Re: (Score:2)
The GP is talking about how to (partially) use a Sun-created filesystem under Linux; why are you bringing Apple into it?
Re: (Score:3, Interesting)
Well, I would think that if you were going to move from Linux to an OS which supports ZFS, you would move to Solaris.
Re: (Score:2)
Re: (Score:2)
There are always some. The question is whether the mainstream needs or wants ZFS features, and I don't see it; that's not where storage is heading.
Re: (Score:2)
No, they just wanted something that could implement "time machine", their backup-done-right proposal.
I expect Solaris to be switchd to GPL some time in the GPLv3 era, at which point there won't be a problem porting ZFS to Linux. Not that it was technically difficult to port it to Apple/BSD (;-))
--dave
Re: (Score:2)
There are several open source file systems that support time-machine like functionality. Nor is there anything particularly new about "time machine"--it's a well-known approach that works fairly well in some circumstances and not at all in others.
I expect Solaris to be switchd to GPL some time in the GPLv3 era, at which point there won't be a problem porting ZFS to Linux.
Perhaps, or perhaps not. Either way
Re: (Score:2)
For example, they view X11 as a dead end and want to actively convert people to using Cocoa; hence, Apple's X11 support sucks and they have no interest in making it better.
You mean Quartz, not Cocoa. And Quartz is unarguably far more powerful than X11 as well as faster; I think they're absolutely correct in promoting it over X11. I'm not sure what "sucks" about their X11 support, as it certainly seems to work just f
Re: (Score:2)
Desktop integration, pre-installation, automatic on-demand launching, consistent window management, keyboard layout integration, to name just a few. Performance could also be improved further.
Apple needs X11 to get people building scientific apps on Linux and Solaris. Its actually one of the best X implementations I've used (X.org, XFree86, Irix X Server)
Yes, and that's pretty much all they are supporting: scientific apps. If you want to use X11 desktop apps, you're out in
Re: (Score:2)
Re: (Score:2)
Dunno what you mean by this. Perhaps copy/paste ? That works for me just fine. I always use xterm/nedit (it's a hangover from a previous life) and apple-C will do a copy in nedit, apple-V will paste into (whatever).
pre-installation
You get to choose whether to install it when you install the OS. It's a package that 99% of the user-base won't need or want, but it's a choice.
automatic on-demand launching
Oh dear. Open up the System Preferences panel, click on the Accounts icon, click the L
Re: (Score:2)
Have you got a flag?!