Sun CEO Says ZFS Will Be 'the File System' for OSX 384
Fjan11 writes "Sun's Jonathan Schwartz has announced that Apple will be making ZFS 'the file system' in Mac OS 10.5 Leopard. It's possible that Leopard's Time Machine feature will require ZFS to run, because ZFS has back-up and snapshots build right in to the filesystem as well as a host of other features. 'Rumors of Apple's interest in ZFS began in April 2006, when an OpenSolaris mailing list revealed that Apple had contacted Sun regarding porting ZFS to OS 10. The file system later began making appearances in Leopard builds. ZFS has a long list of improvements over Apple's current file system, Journaled HFS+.'"
The were going to use Reiser (Score:4, Funny)
Re: (Score:3, Interesting)
I like having an mp3 player that doubles as a backup device for my important files. But some of my files are > 4Gb, so FAT32 doesn't work.
Re: (Score:3, Interesting)
ZFS for FUSE (Score:3, Informative)
Because I have suffered some random corruptions in the past, even with ext3 ("This mp3 didn't used to have a skip there!"), I wanted the checksumming so that I can tell when I need to restore something from a backup.
As a filesystem, it works completely, including creation of new filesystems, compression, checksums, etc. However, I've noticed a decrease in my system's gene
Re: (Score:3, Informative)
A user space filesystem is not something I would expect performance from.
Re: (Score:3, Funny)
Re:The were going to use Reiser (Score:4, Informative)
Re:The were going to use Reiser (Score:4, Insightful)
and since apple has all the rights and source to HFS(+)(Journaled) they can just as easily write a windows driver for it as well.
Re: (Score:3, Interesting)
You mean, the CDDL's restrictive nature.
Semantics. Each license restricts you from using the other. The GPL is generally more restrictive, but since you have to relicense under the GPL, further restrictions to modified software cannot be added. The CDDL has a restriction (regarding attribution) which must accompany any CDDL derivative work. Therefore, the licenses are incompatible. If the CDDL did not require attribution, or if the GPL allowed further restrictions to be placed upon derivative works, the licenses would be compatible (as I un
"No" on all three counts (Score:3, Informative)
No, no and no.
Re: (Score:2, Funny)
This is Slashdot. We do not say "they killed the project". We say "but BSD/OpenSolaris/ZFS/etc is dead".
Re:The were going to use Reiser (Score:5, Funny)
I don't use Slashdot cliches, you insensitive clod!
Re: (Score:3)
Re: (Score:3, Funny)
I'm getting this engraved on my tombstone...
Re:The were going to use Reiser (Score:4, Funny)
Oh God. I can't sign my name to that.
Harsh reply! (Score:3, Funny)
As Homer would say... (Score:3, Funny)
Re: (Score:3, Interesting)
I'm giving odds... (Score:5, Informative)
5:1 that it's not the default root file system in Leopard.
The first bootable release of ZFS (not "BUILD," but "RELEASE") isn't even due until the Fall.
I'm not alone in this skepticism. See this Ars story, for example.
http://arstechnica.com/journals/apple.ars/2007/06
Re: (Score:3, Insightful)
5:1 that it's not the default root file system in Leopard.
It would be foolish to make any new technology that touches so many other applications and parts of the OS the default when you don't have to. It's much smarter to make it an option and try to shake out any problems that arise. Then make it the default at a later date.
Re:I'm giving odds... (Score:5, Informative)
ZFS is several orders of magnitude better at streaming large files like are used in video editing, which is already a huge draw for Macs. Since it is copy-on-write, writes are done without seeking so are very fast and can be spread out across multiple drives in parallel. IIRC within a zfs pool (collection of drives) you can make different 'filesystems' mirrored or striped, so you can have a
You can take your 100gb video and instantly say 'snapshot this' then make any number of changes to it and if you don't like it just revert back again. Contrast to every other filesystem (besides spirolog) where you have to make a 100gb copy as a backup -- which takes forever, so nobody does it unless they have to.
You can drop in a new drive and say 'use this drive' and your existing filesystem instantly has more space available and it is more fault tolerant or faster or both. If you want to remove a drive you say 'dont use this drive' and you can still use the OS normally while it moves data off to other drives.
Something like ZFS, that "touches so many other applications and parts of the OS" has to be the default. Otherwise you have to support two completely different ways of using the system. And that bloat and complication costs a lot more than just getting it right through extensive testing. If you are really worried about it, don't upgrade the OS for a while.
Re: (Score:3, Insightful)
Are you kidding? This is ZFS we're talking about.
Right, I'm sure it won't wind up breaking some important application at the expense of adding a wiz-bang feature that 95% of the users couldn't care less about.
I'm not a Mac user, but even if I could (maybe I can) add ZFS to my Linux workstation I wouldn't. I prefer stable and reliable over untested new features. I think most people feel the same way, so making the default something else makes a lot of sense. If ZFS is as great as you say, it will eventual
Re:I'm giving odds... (Score:5, Interesting)
The Move from Classic (OS 9) to OS X forced people to Recompile/Port or Die from obsoleteness modernized almost all the software for Mac OS X. This removed a lot of Old Hacky code from the code base and forced developers to follow a more modern programming style.
Next it was the move from Power PC to Intel. This once again required a full recompile but this time is assured that the recompile was with their own development tools. So more hacky code was removed and replaced with more standardized system calls.
Now with ZFS on Mac OS X it is more likely that most things will work just fine with ZFS because Apple Knows what most of the calls to the OS will be. And the bulk of the legicy code has been updated.
Windows, Linux and traditional Unix OS Devlopers don't normally Break Compatibility so often so their hacks to work around a shortfall in an OLD version of the OS holds threw to the following versions of their software on newer versions of the OS. So migrating OS ZFS on Linux is much more risky then moving to ZFS on OS X.
But it is a trade off of getting Modern Software and paying more $$$ for the software. or Pay less for the software but make it hard to upgrade to a better system in the future.
Re: (Score:3, Interesting)
Apple has traditionally been in favor of forcing devices, such as not putting arrow keys on the original Mac keyboard.
Their philosophy tends to be of a benevolent dictator like Linus. Apple is going to make you do some things, and it's for your own good. If you're not happy with it, usually you can do something else if you have the technical skill, or you can just go get a Winders box. This has meant Apple's been able to do things no other company could, and is also why, IMO, they're the top of the heap f
Re:I'm giving odds... (Score:5, Informative)
So Apple does leave backwards compatibility stuff there, however they make the new stuff so much better, that developers and users want to get it ASAP.
Now compare this to Microsoft. While
Re:I'm giving odds... (Score:4, Informative)
The old PPC builds of Photoshop are also much slower on PPC than the only universal version, CS3. They moved Photoshop from Codewarrior to Xcode between CS2 and CS3, and it's the most massive rewrite they've ever done. So you can't distinguish how much of the speed difference of CS3 over CS2 on Intel is due to it being Intel native, and how much is simply due to it being faster.
Re:I'm giving odds... (Score:4, Informative)
You are right to a certain extent, but you have to realized that current file systems are old and clunky. For a desktop or a few non-critical servers moving to the new tech is a great idea. Down the road, when ZFS is more mature and understood, it's going to be a welcome addition to most production setups. If you ran real-world mission-critical prod setups needing high availability you'd understand.
Imaging you have a huge medical database on several servers and are running out of disk space. To expand, you need to plug in new hard drives, create RAID setups, create partions, move data over, restart the database, verify again and again, downtimes, etc. You can easily and efficiently grow file systems (unless you're using an expensive piece of software like veritas volume manager. With zfs, all I need to do to expand disk space in current WORKING filesytems is:
zpool add oraclefs mirror c1t1d0 c2t1d0
No luns to deal with. No other filesystem bullshit. You have no idea how excited this makes me for services that require large amounts of growing storage.
Read up on zfs here: zfs [slashdot.org]http://www.opensolaris.org/os/community/zfs/> It is the best thing to come out of Sun in a long time.
Re: (Score:3, Insightful)
For example:
If you asked do you want your software to "just work" or run the chance of having your computer burst into flames just to add some eye-candy/whatever? Then I would image that most people would opt for the stable approach.
However, if you asked do you want to try out feature X? It will make your windows semi-transparent and ALL of your wildest dreams
Re: (Score:3, Interesting)
IIRC within a zfs pool (collection of drives) you can make different 'filesystems' mirrored or striped, so you can have a /video that is striped and ultra-fast whereas /home is mirrored and fault-tolerant.
No, you can't mix and match. When you create a zpool, you get to choose between the equivalents of RAID-0 (striped), RAID-1 (mirrored), and RAID-5/6 (striped with parity). You can nest the various RAID levels, but whatever you choose will apply to all the file systems in your storage pool. If you want to have /video and /home under different RAID levels, you'll need to create two different zpools.
Now, this wouldn't be a big issue, as you could normally just partition up your drives to make the separate z
Re:I'm giving odds... (Score:4, Interesting)
Re:I'm giving odds... (Score:4, Insightful)
So, how exactly does one roll back changes to a file on an NTFS partition?
Re: (Score:3, Informative)
Re:I'm giving odds... (Score:4, Interesting)
NTFS has data checksums to detect and repair corruption caused by any component?
You can add and remove disk space from an NTFS volume dynamically?
NTFS does data-level journaling not to mention without the overhead of multiple writes of the data?
NTFS can use compression without getting horrible fragmented or other negative side effects?
NTFS snapshots do not affect performance of the normal system?
NTFS has variable block sizes?
NTFS is open source and took less than a decade to get support on multiple systems?
As far as I know that's a big no on all those. I mean NTFS is very complex and has a lot of bullet points, but to claim that ZFS is just 'ntfs with larger address space' is really missing the boat.
Re: (Score:3, Informative)
In reference to your questions:
NTFS has data checksums to detect and repair corruption caused by any component?
You can add and remove disk space from an NTFS volume dynamically?
NTFS does data-level journaling not to mention without the overhead of multiple writes of the data?
NTFS can use compression without getting horrible fragmented or
Re: (Score:3, Interesting)
http://youtube.com/watch?v=1zw8V8g5eT0 [youtube.com]
There's an english translation available for that somewhere.
ZFS can do a lot of things with ease that other file systems either can't, or it takes quite an effort.
Re:I'm giving odds... (Score:5, Interesting)
OSX 10.5 ain't due 'til Fall, either.
Booting from ZFS? (Score:5, Interesting)
Re:Booting from ZFS? (Score:5, Informative)
March 28th, 2007 at 19 hundred and 50 hours Zulu time [opensolaris.org]
Re:Booting from ZFS? (Score:5, Interesting)
Re:Booting from ZFS? (Score:5, Interesting)
I don't know any of the technology of ZFS, so I can only guess.
For a boot loader like LILO, it will need to create a list of exact hardware datablocks to read the kernel in from. ZFS might move those blocks around after the "lilo" command built the block map. Then it can't load the kernel.
For a boot loader like GRUB, it will need to have a read-only subset of the filesystem inside so it can find the kernel image file. That might be doable, but it hasn't been done, yet.
So create a small boot partition on the first few megabytes of the drive, and make another partition for the rest and let it be a part of the ZFS pool (if ZFS can accept a partition, and not just a whole disk).
A better option would be to get a computer that has legacy IDE support with bootability, in addition to the main SATA or SCSI support for major hard drives. Then add a Compact Flash adapter to the IDE [addonics.com] port and use a small Compact Flash module to load the kernel from using your favorite boot loader. Or just use an all-SATA mainboard with a different Compact Flash adapter for SATA [addonics.com]. A tiny CF memory module with 16MB or so would be enough to load a nice sized kernel. Or go with a 16GB one and have a copy of /opt and /usr on there as well (structured to work when mounted read-only).
Re: (Score:3, Insightful)
You'd only need those things to get Mac OS X running on a DOS PC, or when using ZFS with Linux, right?
---
Microsoft Surface: the Fine Clothes of a Naked Empire [roughlydrafted.com]
What happens when the core values of an empire are exposed as a fr
He's already backpeddled (Score:5, Informative)
"I don't know Apple's product plans for Leopard so it certainly wouldn't be appropriate for me to confirm anything. [...] There certainly have been plenty of published reports from various sources that ZFS is in Leopard, I guess we will all have to wait until it is released to see if ZFS made it as the default, or if they simply announce that it will become the default in a future release."
Re:He's already backpeddled (Score:5, Funny)
"I didn't know that Steve Jobs was going to call me and scream at me like a diseased monkey when I said that. I'm sorry Steve. Please don't kill me."
It WAS... (Score:5, Funny)
Re: (Score:3, Informative)
Re: (Score:3, Informative)
In Windows 2000 and XP you can format Fat32 up to 160gigs assuming you have the correct driver. With SP2 for XP you can format it up to 250gigs I believe. Most removable drives from Maxtor for instance were and are formatted Fat32.
Technically you're right though, since most Linux distros can format fat32 up to 2tb. NTFS is vastly superior though so the issue has never really affected me personally. Of course on a Windows machines you don't have to use Microsoft's formatting or partitioning tools, you can
No no no (Score:5, Informative)
What Makes Me Nervous (Score:2)
Besides the worrisome concepts of delayed writing and an always consistent file system, I can imagine never being able to bring a zfs back pretty easily. Which, the snapshots are supposed to solve, but pretty soon, my hardware storage budget just went through the roof because I'm storing terabytes of snapshots pretty quickly.
Sounds good but.. (Score:2)
I doubt it (Score:5, Insightful)
I strongly doubt he didn't know about it. This is Jonathan Schwartz, not a OS X rumors blogger. At any rate, ZFS in OS X is Sun's thunder; Time Machine is Apple's thunder, and that's already announced. How many OS X users (other than slashdot readers) will care in the slightest about the underlying filesystem? What they care about are the features, like Time Machine, that it enables.
Re: (Score:3, Interesting)
All over the place Apple advertises that OS X is "Industrial UNIX at the core".
Now, with ZFS, Apple can advertise having a next-generation omega filesystem to replace the long-in-the-tooth Journaled HFS+, which was significantly better than NTFS.
NTFS versus ZFS is a joke
Re: (Score:3, Insightful)
Their new 10.5 servers will probably be closely related to some products in the Sun stable at the moment. With a successful launch this could help start reversing the trend of server dominance.
If mac starts adding more aspects of unix to the OS and gives it a pretty interface , windows small business server markets could be in trouble. I would personally like to see more of these boxes around.
Oh, great: another DiskWarrior lag (Score:3)
(Yesyesyes, I know, ZFS is reliable that disk-recovery tools are not needed. And if you believe that, then you probably believed Microsoft when they said NTFS volumes never needed defragmentation).
Dear Apple:
Please let HFS+ still be an option.
Please let Classic still run on Power Mac processors.
Please let reasonably well-behaved software that uses resource forks still work.
Please let it be case-insensitive and case-preserving.
Re: (Score:3, Informative)
Well, it's true! NTFS volumes never need defragmentation! On the other hand, Microsoft provides you with a defragmenter service (at least in 2k and later) and allows you to defragment files on NTFS volumes... :D
Re: (Score:3, Insightful)
Re: (Score:3, Interesting)
You are aware that ZFS - and, for that matter, Solaris's UFS - supports an arbitrary number of named forks in files? (Sun calls them "extended attributes", probably because that's what NFSv4 calls them, but they're really named forks/named streams/whatever you want to call them.)
Re: (Score:3, Informative)
A common misconception. The "zfs scrub" command will scan the filesystem and try to correct any errors that are found (or panic the kernel); the difference is that ZFS can do this while the filesystem is mounted.
Re:case-insensitive: performance, i18n, safety (Score:5, Informative)
Re:case-insensitive: performance, i18n, safety (Score:4, Insightful)
You can't restore the original case of a string afterwards, but you can always make it lowercase. This is called "case folding." You can fold two strings to a lowercase form, and then compare them for equality or whatnot. Works with Unicode, too.
Here's the whole list of characters that need to "folded" to a lowercase form, accounting for instances where it will cause the string to grow (like that German 'B' thing):
http://www.unicode.org/Public/3.2-Update/CaseFold
(And you can hash those chars too, so folding a string doesn't involve hundreds of conditionals.)
If you don't care about Unicode, case folding an English ASCII char is 2 lines of C code, and a few more if you want extended ASCII.
Once you have a filename, you can store it in the filesystem as the specifically-entered characters, so you don't lose the original casing, but also store with it a hash of the case-folded version. Now whenever you need to look up a specific filename, you case-fold it, hash that folded string, and look it up that way against the hash you previous calculated when creating the file. Now it's as fast as the case-sensitive filesystem, minus the overhead of folding a small string.
If your app decides about a file's existence by using readdir() until it finds it, and doesn't properly case-fold, and didn't call open() with O_EXCL, then not only did you go the long way about it, you got what you deserved for clobbering the file.
Actually, if you don't just open(O_CREAT | O_EXCL) to check for existence and create if missing in one step, then you'll have an atomicity problem anyhow. Use the services the OS provides, they are there for a reason.
--ryan.
Is that all? (Score:2, Interesting)
Re: (Score:3, Informative)
Is it too much to ask that the summary writer... (Score:2)
Switch all filesystems to ZFS... (Score:5, Interesting)
Re: (Score:3, Informative)
Re: (Score:3, Informative)
The OpenSolaris project is currently working on getting bootable ZFS support (available in the current release as experimental).
Let's hope it's the truth (Score:4, Insightful)
Re:Let's hope it's the truth (Score:4, Informative)
I already know how TimeMachine is going to work (it was part of the filesystem presentation at last years WWDC... so I know it, but can't reveal it), and unless they have completely redone that entire system (which was quite elegant), then ZFS will not bring a single thing to it. I do know how ZFS could make that all really elegant, but Apple already has it covered on HFS+.
Re: (Score:3, Insightful)
I can think a few good cases for "backing up" to the same physical device.
Here are the reasons I back up data my employer's data:
OT Grammer/Spelling Nazi Alert (Score:4, Insightful)
I think Slashdot would benefit from adopting some of K5's approach to story submissions. The Firehose is a great start, but instead of simply saying yes or no, users should be able to give feedback to the submitter. The summary for this article is a great example. The submitter typed "build" instead of "built," resulting in an annoying distraction in an otherwise concise description of the story.
Newspapers have Copy Editors (at least they used to; most seem not-too-bothered by spelling these days). It would be nice if interested Firehose users were given the opportunity to help make sure the summary was fit for publication before it hits the front page.
I guess this should have been a journal entry, but it seemed like an opportune time to bring this up.
Video Demo of ZFS (Score:3, Interesting)
This can, potentially, make upgrades a pain.. (Score:4, Insightful)
Re: (Score:3, Interesting)
You are right that ZFS can handle volume size changes live (and HFS+ can sort-of do it), but this does not mean it is a slam-dunk. I would not want to
Re: (Score:3, Insightful)
The best of Unix? (Score:5, Interesting)
Apple breaks in stages, default in 10.6 (Score:5, Insightful)
We all expected the Intel migration to happen with 10.5, they shocked us when they did it off the 10.4 base.
While they did abandon Mac OS to move to OS X, they provided a migration strategy (Carbon) and a compatibility layer (Classic). Classic support shipped with 10.0/10.1, 10.2, and was supported in 10.3 if you already had it, as well as 10.4 I think, but they kept classic for around 5 years, which gave everyone time to migrate to Carbon. Its unfortunate that there is no long-term Classic via Rosetta just from a classic application point of view, but they didn't leave anyone in the lurch.
I expect 10.5 to introduce this OS, which will be useful for new installs, or for external drive arrays, especially for the Video market, but I wouldn't expect it to be the default. OS X has supported a Unix filed system, but defaulted to HFS+, because HFS+ was compatible with Mac OS, so you could dual-boot OS 9 and OS X for a good 2 years on new hardware to maintain compatibility. If they hadn't done that, they would have lost the Pro-Audio and Pro-Video markets that took a few years to get native OS X applications.
Getting it in the wild and for professionals would help that market, while not breaking ANYONE's compatibility. Sometime in 10.5's lifetime they may ship new computers with it, or they may wait for 10.6 in two years. But giving everyone two years is plenty of time to get utilities and applications compatible with the new file system.
The flashy consumer features are touted for the OS, but the underlying architecture has always followed a 2-cycle release. If you've used OS X Server for 10.2/10.3/10.4, you'd notice that they introduced stuff in one version with limited exposed functionality (with the rest via the Unix layer), enhanced the functionality in the next rev, and polished thereafter.
The Apple Mail Server -> Cyrus migration was someone poorly handled, but mostly because AMS was garbage. But the 10.4 Mail tools are night and day beyond the 10.3 ones.
They are actually far more careful than people give them credit for.
The different is, they don't keep backward compatibility as a long-term goal, they do a two-stage migration, giving people 2-4 years to transition.
short memories (Score:3, Interesting)
It did not shock those of us who know that NEXTSTEP was transparently portable to at least four architectures.
ZFS still has bugs (Score:5, Informative)
We run Netbackup Enterprise on Solaris 10 - during our last round of upgrades we installed ZFS on our disk staging storage units. It replaced VxFS. The way disk staging storage units (DSSUs) work in Netbackup, the disk is always near 100% full form a unix perspective. Basically, any time more disk is needed, the oldest image that has been copied to tape is expired from disk, thus freeing up more room. However, ZFS's most prominent bug from our perspective is that during periods of high activity, if all blocks become allocated, it becomes impossible to unlink(2) a file. This causes the application to no longer be able to make space for new backup images.
Going down the shell, try to rm a file and it comes back: rm failed, disk is full.
Well, if the disk is full, and you can't rm because the disk is full, how do you free up space?
Sun's response, truncate an unnecessary file using 'cat
Ok - so how do you tell a compiled application to truncate an unnecessary file before unlinking it? You can't! How can you determine what an unnecessary file is? If you delete the image before expiring it from the catalog you get errors when you try to expire, so you end up with catalog corruption.
All in all, this is a problem that should never have been introduced, let alone still exist after months of sending trace outputs and reproducing it in multiple environments. ZFS isn't ready for the real world.
Re: (Score:3, Informative)
The price of copy-on-write. Other systems with a copy-on-write file system [netapp.com] can exhibit this behavior.
ZFS not ready for prime time (Score:5, Informative)
I setup ZFS on some SAN storage in a new system. The internal boot disks were mirrored UFS. When one of the HBAs fried, the SAN storage disappeared - and the system panic'd.
Every reboot thereafter stopped in a panic. The ZFS subsystem panic'd the system at every boot when it couldn't find all its volumes. After calling Sun support, I found out that they need to do a massive code redesign to catch that issue, and it wouldn't be out for at least 6 months.
I'm sure ZFS will be great - once they clean up these type of showstopper bugs.
Time Machine will not require ZFS (Score:3, Informative)
Re:oblig... (Score:4, Interesting)
I'm not sure if that's the way they talk about it internally at Sun, but that's how their instructors portray it out in the field.
Re:oblig... (Score:5, Informative)
Re: (Score:3, Insightful)
ZFS does indeed look like the greatest thing since sliced bread, but if all I cared about `wasn't losing data', then I'd just stick with ext2 or fat. (But I care about more than that, which is why I use xfs, ext3, hfs+ and ntfs for various things.)
Re: (Score:3, Funny)
Re:oblig... (Score:5, Funny)
Re: (Score:3, Funny)
Re:oblig... (Score:4, Funny)
Re:oblig... (Score:4, Interesting)
Re: (Score:3, Informative)
Re:oblig... (Score:4, Informative)
Re:oblig... (Score:5, Interesting)
To bad no windows port is available. It would be nice to see my unix drives from windows.
Re: (Score:3, Interesting)
Aren't there still licensing issues to iron out?
Ext2/3 and ReiserFS have all been ported to Windows, so I don't see there would be any problem porting the Linux ZFS implementation as an IFS driver for Windows Vista/XP/2000
Re: (Score:3, Informative)
Which I find really deplurable , I would love to have it in the kernel.
And yeah id like an ifs driver but none exist
Re:oblig... (Score:4, Interesting)
So porting a filesystem as complicated as ZFS could take some time.
Re:oblig... (Score:5, Informative)
You definitely can port FS to Windows using only documented API, but it's a long and tedious process. I'm currently porting FUSE to Windows, so I know it
Re: (Score:3, Informative)
Re:oblig... (Score:4, Interesting)
Depends - whatcha building? An app server, a web server, a database server, or a file server? Different strokes for different folks, and I'm not clear yet if I'd like the overhead of ZFS on a database server. The jury's still out on ZFS+Oracle...
I'm not 100% on which file system I'd like. Certainly the integrity of ZFS is quite pleasing for a DBA, questions is if the overhead is worth it...
Re:oblig... (Score:5, Funny)
Every ZFS volume has a copy of the Tolstoy classic embedded for internal benchmark purposes.
Re: (Score:3, Interesting)
Re:oblig... (Score:5, Funny)