ZFS Set To Eventually Play Larger Role in OSX 196
BlueMerle writes with the news that Sun's ZFS filesystem is going to see 'rudimentary support' under OSX Leopard. That's a stepping stone to bigger and better things, as the filesystem will eventually play a much larger role in Apple OS versions. AppleInsider reports: "The developer release, those people familiar with the matter say, is a telltale sign that Apple plans further adoption of ZFS under Mac OS X as the operating system matures. It's further believed that ZFS is a candidate to eventually succeed HFS+ as the default operating system for Mac OS X -- an unfulfilled claim already made in regard to Leopard by Sun's chief executive Jonathan Schwartz back in June. Unlike Apple's progression from HFS to HFS+, ZFS is not an incremental improvement to existing technology, but rather a fundamentally new approach to data management. It aims to provide simple administration, transactional semantics, end-to-end data integrity, and immense scalability."
Re:So.... BSD or Solaris??? (Score:5, Informative)
Uh, Mac OS X is certified standard UNIX. Solaris is also certified standard UNIX. And they're both fully POSIX compliant.
What are some examples of non-standard tools?
a true end (Score:4, Informative)
Correcting errors in AppleInsider ZFS article (Score:2, Informative)
Please do not bother with this debunking (via Macjournals) unless you are truly interested. Thanks.
http://www.macjournals.com/news/2007/10/04.html#a79 [macjournals.com]
Re:So.... BSD or Solaris??? (Score:4, Informative)
I'm hoping not, since many things behave very oddly on Solaris. Non standard tools and such, but it would be one way to keep it from running on cracked PC's.
2 cents,
QueenB.
If by "non-standard tools" you mean non-GNU, yes, but they are hardly odd.
I have no idea what your "cracked PCs" comment is all about, and what it has to do with Solaris and ZFS.
Re:Buzz compliant (Score:4, Informative)
Re:Buzz compliant (Score:5, Informative)
They have defined what they mean by that claim already: they have a checksum (256-bit, I think) on every block, and that checksum is checked from the OS when the block is read.
This will catch some errors that might otherwise go uncaught, which is important for servers that move a lot of data around.
It will not catch a memory error at the wrong time, or a processor error that stores the wrong value, or an error in the brain of the person who reads the data from the screen.
Not so. ZFS could handle resources (Score:5, Informative)
You know.. Wikipedia is very handy to look these things up. Please do. http://en.wikipedia.org/wiki/ZFS [wikipedia.org]
Re:Translation: (Score:2, Informative)
Way easier to manage: only 2 commands! While now with an LVM you have to place your disks in the desired topology inside your LVM (RAID0, 1, 5, ...), format them, put a filesystem on, mount, file check, repair, whatever. With zfs you place disks in your pool and kinda mount part of it, that's it.
There are some other things you could complain about: it makes less sense on hardware RAIDs with good management tools. They missed a chance to make it a distributed or clusterable file system (though they bought Lustre lately, who knows) and it's not possible to boot from it yet, but all in all it's a major step forward.
Re:a true end (Score:3, Informative)
With OS X, Apple rolled the resource fork into the "data fork" portion of the file, meaning the information was still there for legacy purposes.
That doesn't sound right to me-- or at least I'm not sure what you mean by that. OSX still has resource forks, but Apple basically told developers not to put important information in them anymore because they get lost so easily. They can't just push the resource fork into the data fork of the file, because in many formats there's essentially no space for that information. How do you store an icon, metadata tags, and a default application setting into a normal .txt file without making it into something that isn't really a normal text file?
So right now, developers are only really supposed to use resource forks for things that don't matter much. So if they're stripped, you lose a little metadata, but nothing catastrophic. It used to be that some files would carry all their data in the resource fork, and that was a bit of a nightmare. You'd lose important information all the time. In order to protect resource forks a little while transferring them to other file systems and such, Apple made those little files that you mentioned that begin with dot-underscore. So copy a file to another filesystem, and it'll often dump the resource forks into those dot-underscore files so that OSX can recombine them later.
I don't see how ZFS will change the situation greatly. Resource forks are just how HFS+ supports metadata. ZFS supports metadata too. I'm sure the technical implementation is different, but I bet you'll still risk losing your ZFS metadata if you move your file to a FAT partition. Or am I missing something?
Re:Buzz compliant (Score:5, Informative)
Re:They said the same thing about UFS. (Score:5, Informative)
Spotlight in current form tries to index every single source file, huge framework headers and there is no practical way to stop it. I have tried the Privacy pane as suggested and no, it doesn't explain my 130 MB of spotlight metadata after installing Developer tools and couple of GNU libraries.
If they have checked the NeXT history, they would figure the UFS is the default,supported Filesystem on NeXT. As OS X is a mix of NeXT with FreeBSD and Cocoa/Carbon, it is pretty natural that UFS gets into it a bit lately but finally.
I can imagine what Apple needs for supporting ZFS on startup volumes. Complete metadata and resource support. They could be happy with their ext3 plain filesystem but Apple using professionals REALLY label their files, sometimes change their icons, sometimes has to FORCE OS to open a file with a different version of suite (e.g. Quark 7 vs 6), add comments to them and professional software developers like Adobe still stores critical data on resource forks.
If there is a way to make ZFS support all those features without huge hacks (like the ZIP _resource stuff), they would give up their HFS+. Another thing is, it must support every serious software (non hack) backwards. You may find yourself using a application from 2001 written in Carbon under OS X and only it can provide the tool you require.
I am saying these since some elitists think Apple is backwards and stupid still supporting resource forks and implement special features to OS X just to give minimum compatibility with old applications.
Before critising HFS+ and suggesting Apple to use plain, Unix filesystems, they should sit around in a professional environment such as a DTP house, Movie studio and see how all those "childish" "backwards" features are used by professionals in job.
This is not a post against ZFS, I am just trying to explain why Apple can't magically move to another filesystem just because it has better features. Not even mentioning the "overhead" required by ZFS and the fact that there are some 2k/4k (Cinema) edit environments which you can't even enable journaling let alone adding another layer of overhead.
Also while writing these, if I only used plain Unix tools without any "native Mac" Application, e.g. use OS X as Darwin with X11, UFS would be my choice of filesystem.
Re:Non-Standard my ass! (Score:1, Informative)
Although the GNU tool's are not always the published "Standard", their tools do set the usability "standard", but I think you'll find that the good extentions are incorporated into solaris in good time ( e.g. to think of one item I've noticed in solaris 10 ( the -h option to du and df )
A merge of MacOS Apple and Solaris Sun would make perfect sense.
A) they have complementary customers ( enterprise / servers/data center) and ( home/smaller company / desktops ).
B) they both have a similiar focus on quality ahead of cost.
C) they both derive a certain amount of their strength from their ability to control from top to bottom of the stack ( hardware, Operating System , Software )
Re:So.... BSD or Solaris??? (Score:5, Informative)
According to the Single Unix Standard [opengroup.org], only Mac OS X 10.5 (Leopard) can be considered "Unix". And only when deployed on Intel-based Macs. Previous versions must be considered like Linux: "Unix-like".
FWIW, Sun's operating system (SunOS) has been fairly close to Unix standards over its lifetime. In fact, the official version of System V release 4 was written by Sun and called SunOS 5, integrated into Solaris 2
Why is anyone even having this argument? GNU means "Gnu's NOT Unix" for a reason...
Re:So.... BSD or Solaris??? (Score:3, Informative)
You should get out more: (from the Linux manpage)
Re:Buzz compliant (Score:2, Informative)
Not quite... ZFS stores a checksum with each block pointer. So wherever you have a structure that indicates where the data is, there's also a checksum of that data. This also means that the block pointers themselves are checksummed with their pointers. And so forth. The only one that doesn't have a checksum with the pointer is the top-level root pointer, and they have multiple copies of that for redundant checksumming.
And yes, for true integrity, you need ECC memory, and ECC CPUs. I don't know if the Intel CPUs have any sort of internal error checking; I have worked with processors that have.
Re:Not so. ZFS could handle resources (Score:4, Informative)
They are starting to do some stuff with them. The first major use I know of was with XP SP2. With that, when you downloaded a file from the internet, IE would mark it as such in an alternate stream. When the program was run, someone (I don't know who) would check for the presence of the stream and if it was there, would display a "this program came from an untrusted source, would you like to run it?" dialog.
I would expect more uses as we move into the future, as Vista is pushing even heavier for NTFS (for instance, IIRC the installer didn't ask which file system I wanted to use and just formatted NTFS), and MS doesn't have to worry about, for instance, some 98 or ME user who upgraded to XP but is still running FAT so he didn't have to reformat. For my large partitions (~100 GB), I can't format as anything but NTFS. (I don't know about smaller ones; I have a 24 GB system partition but if I try to bring up the format dialog there it complains that I'm trying to reformat the drive with the OS and I don't want to do that.)
Personally, I think that there's a lot of awesome stuff that you could use extended attributes and alternate streams (WHY are these separate concepts on some file systems?!) if only they would be preserved when you move stuff around systems, upload them, etc., and am somewhat resentful at Unix and POSIX for the fact that for ages they didn't do this stuff and hence it's really hard to move to using them because no one supports them because there's no demand because people haven't thought of what to do with them because they haven't seen what can be done with them because no one uses them because no one supports them because...
Re:Time Machine (Score:4, Informative)
Re:folders are even worse (Score:4, Informative)
It's true that you can't expand a RAID-Z set (I think, anyway -- if you replace all of the drives, one at a time, does that work?), but you can add another RAID-Z set, and expand the pool.
That's the big thing in ZFS, combining all of the resources into a pool, rather than treating disks (or groups of disks) as part of a volume. The other part of this was making filesystems nearly as light-weight as directories.
My plan is to use twinned drives, adding them as a mirror to the pool. I can replace each drive individually, let it re-silver, and then do the same with the other, to expand it, or I can simply add another pair of drives to the pool, and get more space that way. There are advantages and disadvantages to each.
Oh, as for resource forks -- the model that Sun is choosing (as are some others) is that the extended attributes are treated as sub-files to a directory. I'm not sure that simply going to a directory is not a better idea, but that has a whole slew of its own problems. It's a bit ironic, really -- Apple had an idea from the beginning, and every application was prepared to deal with it, but nobody else did the same thing. Then, when Apple went with the flow, everyone else started trying to do what Apple did... and none of the applications are prepared for it.
I'm not sure how it'll all turn out.