Plethora of New User Space Filesystems For Mac OS X 225
DaringDan writes "As part of the recent MacFUSE 2.0 release Amit Singh has added support for an insane number of filesystems on the Mac. This video from Google and this blog post pretty much explain everything in detail but to sum-up Singh has written a new filesystem called AncientFS which lets you mount a ton of UNIX file formats starting from the very first version of UNIX. Even more interesting is that they have also taken Linux kernel implementations of filesystems like ufs, sysv-fs, minix-fs and made them work in user-space on the Mac, which means its now possible to read disks from OSes like FreeBSD, Solaris and NeXT on OS X. ext2/ext3 don't seem to be on the list but apparently the source for everything is provided, so hopefully some enterprising soul can apply the same techniques to ext2. One of their demos even has the old UNIX kernel compiled directly on the Mac through the original PDP C compiler by somehow executing the PDP binaries on OS X!"
Re:Sounds great. (Score:5, Informative)
NTFS-3G which works with MacFUSE has full NTFS write support. It is, however, much slower at reading than the native NTFS read-only driver.
Re:Sounds great. (Score:3, Informative)
FUSE for Windows (Score:3, Informative)
Also, it seems to be a good time to plug Dokan and my FUSE4Win project :)
There's a wonderful project named "Dokan" (http://dokan-dev.net/en/), it makes posssible to write user-space filesystem in Windows.
I've adapted FUSE interface for it, so it's possible to use (almost) unmodified FUSE filesystems in Windows: http://hg.sharesource.org/fuse4win [sharesource.org]
Currently, SSHFS works fine. NTFS3g also works :)
Re:Every filesystem! Except the ones that matter.. (Score:5, Informative)
No, NTFS has support via NTFS3G. The other popular file system, FAT, is already supported natively.
Re:Sounds great. (Score:4, Informative)
These are mostly read-only (Score:4, Informative)
These are mostly read-only implementations for dealing with archival data. They're not read/write, which is more work to implement and not that useful.
QNX has had user-space file systems for decades. (It's a microkernel; almost everything is in user space.) Some users wrote file system implementations for weird file systems, like .zip files. Most of the modern QNX file system action is supporting various flash-based file systems and networked file systems.
Re:FUSE vs. FST (Score:3, Informative)
Apple's GS/OS had FSTs (File System Translators) that allowed that operating system to access HFS, ProDOS, DOS 3.3, and FAT volumes. How does FUSE compare in function to GS/OS's FSTs? You know, apart from working with non-obsolete hardware.
Fuse is not a file system or even a translation layer per se. It's just a way to let you run a filesystem in userspace, instead of inside the kernel where it would normally reside.
This is slower, but has many benefits including easy development/debugging, not requiring access to kernel sources or rebuilding of the kernel, and easy portability of these filesystems to run atop different host OSes. It also gives the "guest" file system access to all the features of userland, making it easy to, for example, launch a process to convert a file or establish a TCP connection somewhere.
GS/OS was very primitive by todays standards and did not even have a "user space" to speak of - there was no preemptive multitasking or protected memory, so FST was not some significant architectural paradigm - just that the OS knew how to read more than one kind of disk. It's not really comparable to fuse, although it is comparable to some of the things you can do with fuse.
Look around for other projects that use fuse. There's some interesting stuff you can do with it. To give you an idea, years ago somebody did a filesystem which would make a bunch of audio files in a variety of formats all appear to be .MP3 files. When you view the directories all that's happening is the extensions are being changed, but then when you open one, he would transparently launch a decoder in the background to convert the file.
You could also do things like making a remote FTP site appear as a normal file system... all kinds of stuff.
Re:Sounds great. (Score:3, Informative)
NTFS-3g is just as stable and performant as a native fs, on both OSX and Linux. Ubuntu even supports it officially with Wubi now.
Re:Reiser 4 (Score:4, Informative)
I was semi-involved in this -- basically, Reiser4 wanted to make the filesystem pluggable, but in kernel-mode. The idea was that features would be implemented in the filesystem itself -- crypto, compression, files-of-files (for example, instead of suid utilities like passwd, just let me edit /etc/passwd/sanity), metadata-as-files (Unix permissions are a file, or things like foo.mp3/id3/genre)....
Lots of cool ideas bouncing around, and most of them might work better as FUSE filesystems -- for example, there's no reason id3lib needs to be in the kernel.
Some of them, it makes sense -- certainly for spinning disks, even moreso for external media, the media is so much slower than the CPU that compression makes sense, but you want to compress on flush, and not before. The part that was cool about that was, from the benchmarks they were getting, performance was actually better with compression turned on, because of how fast the algorithm is, how fast CPUs are, and how slow spinning disks are.
But if you break down the "plug-in" concept, it was really horribly mis-named and mis-marketed -- it was just an API, like the pluggable IO schedulers were. It's really something that would probably go in the VFS layer -- something Hans had a really difficult time selling; according to his story, the first time he brought up his ideas, they told him to go put it inside his filesystem, not in the VFS, to try it out without making such drastic changes that would affect other filesystems. When he did that, and came back with the so-called "plugin" architecture, they told him that it should have been in the VFS, so that other filesystems could use it, and they refused to merge it.
Looking at it now, it looks like most of these ideas, and some filesystems currently in the kernel, are better as FUSE filesystems. Better to keep the kernel smaller and more reliable, especially when the performance advantage is minimal -- FUSE will never be the bottleneck for sshfs, for example. Crypto works well enough at the block level (for full-disk crypto). And 20 years from now, we'll have computers so absurdly fast that no one will care about the performance hit of FUSE -- arguably, we're there already.
But I do still wish that the compression, at least, would be tried, if those benchmarks have any truth to them.
Anyway -- more directly, you would never have been able to write file-system customizations in Perl, unless Perl was put in the kernel, which would be an atrocity. However, you can write whole filesystems in Perl, Python, Ruby, or pretty much anything you want -- just use FUSE -- and nothing is stopping you from letting a real filesystem handle 90% of that.
Must be rare (Score:3, Informative)
Well, for the last year or so, I have been using NTFS-3G every single day of the week, using it to running my virtual machines stored on a USB hard disk (performance is excellent, by the way -- with Linux's tendency to cache the VM images, Windows guest performance is easily better than running natively off a physical partition).
At my college, and at home, there are computers with faulty USB connectors. A few times a week, when running a VM, my USB cable will wriggle loose, and the hard disk will disconnect. Yes, while it's running NTFS-3G.
However, not once in the last 12 months has my NTFS filesystem become corrupted. Not once.