Apple is Upgrading Millions of iOS Devices To a New Modern File System Today (theverge.com) 191
Apple today began rolling out iOS 10.3, the latest point update to its mobile operating system. iOS 10.3 brings with it several new features, chief among which is a new file system -- called the Apple File System (APFS). From a report: It's a file system that was originally announced at WWDC last year, and it's designed with the iPhone, iPad, Apple Watch, Mac, and Apple TV in mind. Apple has been using its 31-year-old Hierarchical File System (HFS) for iOS devices so far. It was originally designed for Macs with floppy or hard disks, and not for modern mobile devices with solid state storage. Even its successor, HFS+, still doesn't address the needs of these mobile devices enough. Apple's new APFS is designed to scale across these new types of devices and take advantage of flash or SSD storage. It's also engineered with encryption as a primary feature, and even supports features like snapshots so restoring files on a Mac or even an iOS device might get a lot easier in the future.
First (Score:1)
Sent from APFS
here. we. go! (Score:1, Funny)
Re: (Score:2)
Is this the OS that won't run 32-bit apps?
Catch? (Score:5, Interesting)
Re: (Score:2)
Re:Catch? (Score:5, Interesting)
Apple File System is designed to avoid metadata corruption caused by system crashes. Instead of overwriting existing metadata records in place, it writes entirely new records, points to the new ones and then releases the old ones. This avoids a crash during an update resulting in a corrupted record containing partial old and partial new data. It also avoids having to write the change twice as happens with an existing HFS+ Journaled file system where changes are written first to the journal and then to the Catalog file.[3]
Still no checksum for user data like ext4. But it might help iPhones will sudden battery failure.
Re:Catch? (Score:5, Informative)
Re: (Score:1)
ZFS have checksums.
Sadly we don't use ZFS for some stupid reason (maybe RAM for instance but I'd just get more RAM if needed.)
Re: (Score:1)
ZFS have checksums.
Sadly we don't use ZFS for some stupid reason (maybe RAM for instance but I'd just get more RAM if needed.)
There are a number of reasons why we don't use ZFS; some technical, some political.
This is a good alternative.
Re: (Score:2)
I will say that APFS is a must have update from HFS+. It has copy-on-write functionality, snapshots, and other stuff that make sense. It has a very interesting facility for encryption to allow for volume, file, and almost anything in between, with keys for everything able to be different.
However, it doesn't have the good bit-rot detection that ZFS, ReFS + Storage Spaces, and btrfs have. In fact, it doesn't have any real robust drive scrubbing type facility to find and (even better) repair ECC errors. I
Re: (Score:2)
Apple even went as far as having ZFS in one of the betas for (IIRC 10.4 or 10.5?). You couldn't make the boot disk ZFS, but you could use all the z* commands in Terminal to manage your pools. And, then in the RC it was yanked.
Re: (Score:2)
Ext4 doesn't have user data checksums, only metadata: https://ext4.wiki.kernel.org/i... [kernel.org]
I think it is an option. Journaled metadata is the default, but you can choose journaled data too. It is just not very well performing. The B-tree structure of writing new entries and switching atomically works much better that way.
No overwrite (Score:2)
In other words, it functions the same way as copy-on-write filesystems such as Btrfs and ZFS, or log-structured filesystems such a various flash-oriented systems (F2FS and the likes) or as the venerable UDF.
Or thank you apple for finally having a filesystem that is not decades out-dated, but finally joining the club of other modern Unices.
I am just wondering why they needed to re-invent their own wheel, instead of opting for re-using ZFS.
Re: (Score:2)
Re: (Score:2)
Apple File System is designed to avoid metadata corruption caused by system crashes. Instead of overwriting existing metadata records in place, it writes entirely new records, points to the new ones and then releases the old ones. This avoids a crash during an update resulting in a corrupted record containing partial old and partial new data. It also avoids having to write the change twice as happens with an existing HFS+ Journaled file system where changes are written first to the journal and then to the Catalog file.[3]
Still no checksum for user data like ext4. But it might help iPhones will sudden battery failure.
This by itself is not sufficient for NAND Flash storage as a lot of SSD manufacturers have discovered and then ignored. An interrupted write to NAND Flash do to power loss can result in corrupted data in *other* blocks. So battery failure leading to power loss can still result in NAND Flash corruption.
Has this even been a problem on Apple devices? I assumed they shutdown before actual power loss.
Re: (Score:2)
I've been running iOS 10.3 beta for the whole run on an iPhone 6 with 16GB of storage. There haven't been any problems, despite the limited space that it has to work with and how much it has to go and flush cache files and whatnot. I'd be surprised if there are more than a handful of problems related to the upgrade.
Re: (Score:1)
I've been running iOS 10.3 beta for the whole run on an iPhone 6 with 16GB of storage. There haven't been any problems, despite the limited space that it has to work with and how much it has to go and flush cache files and whatnot. I'd be surprised if there are more than a handful of problems related to the upgrade.
Thanks for the info. I'd still back my iphone up TWICE before installing iOS 10.3, though...
Re: (Score:2)
Re: (Score:1)
Not readable on anything other than Apple products, at least initially. Then again, the Linux HFS+ driver still can't write to volumes that have journalling enabled...
And NTFS is STILL not documented; so...
Re: (Score:2)
Re: (Score:2)
There must be something in this update that screws over the customer somewhere.
It could just be the latest Apple copying Samsung. Specifically the original Galaxy S. The catch there was that the replacement filesystem for mobile phones (Samsung RFS) was so slow during R/W operations that the OS actually would think programs locked up while performing I/O operations and force close them. "Lagfix" patches to fix this problem actually simply reformatted the system and data partitions with JFFS or ext2.
Re: (Score:3)
Re: (Score:2)
Re: (Score:3, Insightful)
We're going to disagree on that point, of course, and that's fine. What you look for in a well-engineered product, and what I look for in the same, are different, because we have different needs and we use o
Re: (Score:2)
Re: (Score:2)
Who made mass-producible 192+ DPI displays before Samsung? Whose DRAM chip design did they copy? Who'd they steal the masks from for their Exynos CPUs? I suppose they stole the specs and designs for the various DACs, encoders, and other useful bits of silicon they sell, as well. Right?
You know, if that were true, Apple would be just as liable, since it's (supposedly) public knowledge and Apple still uses Samsung parts. Knowingly buying
Re: (Score:2)
Slamdung
Please finish Tim Cook's blowjob before typing. It's hard to read your shit when you can't spell.
Re: (Score:1)
Hmmm...this actually sounds like a useful upgrade. Given Apple's recent "innovations," I'm left wondering what the catch is. There must be something in this update that screws over the customer somewhere.
Nope, sorry.
This is their response to other COW Filesystems, such as ZFS. Another thing that could have been wonderful, if Oracle hadn't locked it away for themselves.
Re:Catch? (Score:5, Informative)
That sounds reasonable, except for every single part of the statement being a complete falsehood.
Re: (Score:1)
Re: Catch? (Score:1, Informative)
If it can't be updated past 9.3 then it must be the 2012 model. I understand the frustration of not being able to update your MP3 player but none of the comparable iPhones and iPads got the update to iOS 10 either. The 2015 model of the iPod touch does get iOS 10 and likely 11. To be fair, the 2012 model got iOS 6, 7, 8 and 9 before being dropped.
Re: (Score:2)
5 years seems to be the anticipated lifetime in the Apple world. Some survive a bit longer but on borrowed time.
Re: (Score:2)
My mid-2010 Mac mini has survived on borrowed time for five years. Since january 2016, however, it has been getting by using small time loans every few months.
Re: (Score:2)
Only a matter of time until it can't run the latest OS X. Of course you can still use it, it just wont get security updates anymore. Then there's Linux. I really just meant until Apple abandoned it.
Re: (Score:3)
Software kills computers.
I have a 2010 MacBook Air which became progressively more stupid with each software "upgrade" to the point where it was unusable.
I replaced it with a cheap Chromebook which has much better performance.
However, it's tax time so I broke out the old MacBook to do my taxes. On a whim I installed the Opera browser and I feel like I have a new MacBook! Safari, FireFox and Chrome browsers would peg the CPU at 100% for literally minutes while they did god knows what. With Opera, CPU only g
Re: (Score:2)
Until a few weeks ago, I had been using a Blue & White G3 for an internet server (HTTP, DNS, SMTP, IMAP, as well as NAT/DHCP). They date back to the turn of the century. But after moving it to a different city and a differently configured LAN, netinfo started getting confused and locking up the system for anything beyond simple UI activities, sometimes hours after boot, sometimes minutes.
It was so slow that its load average shot way up and noticeably bogged down the NAT performance when one of those "h
Re: (Score:2)
But you are a geek. You can make old stuff continue to be relevant. Imagine an ordinary user with an old machine that no longer gets updates.
Re: (Score:2)
I got 9 months of support on my Galaxy Tab 10.1. And then the 6-month late Android update came that screwed it up so badly that I had to root it to fix it. Fortunately that was a common problem, so it least there were instructions on multiple forums on how to fix it. Samsung never did address it. And that was it. Abandoned.
Re: (Score:2)
So it seems the premium for Apple gear might not be such a bad deal? And actually Samsung is almost as pricey as Apple.
Re: (Score:2)
I've got a 2 year old S5 that just got an update from AT&T the other day. I was kind of amazed.
Re: (Score:2)
Of course it was.
-Hans
About time... HFS+ is crap (Score:5, Interesting)
In an interview at Melbourne's linux.conf.au conference, Linus Torvalds called the standard file system of Mac OS X "complete and utter crap." Mac fans are only slightly outraged, pointing out that HFS+ isn't really "complete and utter crap," rather, it's just slightly crap-ish. [zdnet.com]
On a personal level, I have had multiple corrupt HFS+ filesystems, one of which was unrecoverable. I tried switching to exFAT which also proved to be corruptible but repairable. Now I just store any data I care about on a NAS running a linux ext4 filesystem.
Hopefully, AFS will fix these corruption problems. I have been sending Apple upgrade suggestions for years. Looks like they finally got around to it. One filesystem to rule them all, but will it support upper/lower case?
Re: (Score:3)
Re: (Score:2)
Actually both World if Warcraft and Eve Online can not perform an online upgrade on a case sensitive HFS+
Well, don't know it is right now, but while I had them on a case sensitive external drive a few years ago, they both complained during updates.
Re: (Score:2)
Adobe Creative Suite was a major one a while back. Don't know if that's changed, I only run CC on Windows now.
Re: (Score:2)
Comment removed (Score:5, Insightful)
Re:Don't blame a filesystem for your lack of backu (Score:5, Insightful)
Even if you don't lose data, you still lose uptime. Backups are only one layer.
Re: (Score:2)
What do you mean? Onion or cake layers?
Re: (Score:2)
The non-analogy type.
Re: (Score:3)
Re: (Score:2)
self corruption preventing (Score:2)
No, but it IS the filesystems fault for corrupting itself no?
File systems with a lesser tendency to corrupt themselves :
file systems that do not over-write themselves.
such as Copy-on-Write filesystems (Btrfs, Zfs, etc.) and Log-structured filesystems (F2FS, UDF, etc.)
according to source, APFS is also going to be copy-on-write, making it similarly more resilient to corruption.
(if system crashes or loses power mid-write, you don't end-up with a corrupted file system.
You end up with the previous version of the filesystem, plus new copies of data that may or may not be c
Re: (Score:2)
On a personal level, I have had multiple corrupt HFS+ filesystems, one of which was unrecoverable.
The problem with HFS+ isn't so much HFS+ as it is the ancient code that is still used to fsck it. Back in the day, Norton Utilities for Mac had a good disk repair, and the current good repair utility is DiskWarrior, but fsck_hfs apparently still uses the same basic code that dates back to the '90s.
but will it support upper/lower case?
HFS+ has supported case-sensitive mode for years, but you do have to make the choice when the partition is first formatted, and it's not recommended for the boot partition, just in case some random program fails
Re: (Score:2)
HFS+ has supported case-sensitive mode for years, but you do have to make the choice when the partition is first formatted, and it's not recommended for the boot partition, just in case some random program fails with it.
Problem is that very few people use it, including very few developers. Let's say a game installer creates a "World Of Warcraft" folder, then copies files into the "World of Warcraft" folder, that would work on most machines, including mine, including most likely that developer's machine, but not on a case sensitive file system.
On the other hand, iPhone file system has always been case sensitive.
Re: (Score:2)
Sent from my iMac
Re: (Score:2)
Yes, I had a hardware problem on my Mac and it was hanging on boot, so I would occasionally have to power it off/on. Rarely, the system would spontaneously reboot. This was more than HFS+ could deal with and it started corrupting the filesystems. I don't think it was able to do a good job of fsck-ing on boot which led to the corruption. In any case the hardware problem has been fixed (it was slot-creep) due to heat but I still on occasion lose the exFat partition even though there are no more hardware p
Re: (Score:2)
ë isn't part of the german alphabet
Re: (Score:2)
ë isn't part of the german alphabet
Neither are ü ;)
German accented letter are not separate letters and thus not in the alphabet. More importantly: ë isn't used in German.
Re: (Score:2)
The new file system is just a bag of bytes for names. You can store ë just fine, any way you want it (though UTF-8 and composed are probably the best options).
Re: (Score:2)
Re: (Score:2)
Seriously don't use FreeNAS. It's crap, doesn't give you enough control over how stuff is configured, and has weird issues that vanilla FreeBSD with ZFS doesn't have. Use vanilla FreeBSD if you don't need directory service integration. If you need directory service integration, use CentOS 7 and ZFS on Linux.
Backup Your Device (Score:4, Insightful)
In the spirit of "what could possibly go wrong" this may actually be one of those times you want to back up your device before upgrading.
Re: (Score:2)
In the spirit of "what could possibly go wrong" this may actually be one of those times you want to back up your device before upgrading.
Samsung's original Galaxy S ran on a special purpose filesystem called RFS. What did go wrong there is that it was so slow that the OS often force closed running apps while they were performing I/O operations as it thought that the apps had crashed. The fix was to convert the filesystem to something else.
Re: (Score:2)
The fix was to convert the filesystem to something else
An explosive improvement.
I would like my iPod Touch to be upgraded past 9.3 (Score:1)
I kinda would like my iPod Touch to be upgraded past 9.3. It's not really that old.
Access? (Score:2, Insightful)
Re: (Score:2)
And we'll all have access to the file system from the native iOS UI right? Riiiiiiight?
Bwahaha!
Re: (Score:2)
What do you mean? U+1F34B or :lemon:?
Security (Score:5, Interesting)
Will this add any security against NSA / Immigration trying to steal your data?
Re: (Score:1)
No but it will probably help apple steal it.
Will this affect known methods for breaking in? (Score:4, Interesting)
Trying to avoid talking about whether it is a good or bad thing that police can break into iPhones when necessary -- just curious if anyone has any technical insight.
Re: (Score:2)
Yes.
http://dtrace.org/blogs/ahl/20... [dtrace.org]
Re: (Score:2)
I'm wondering if this will affect known methods used by law enforcement to break into iPhones in high-profile cases (such as the San Bernardino shooting). Anyone have any insight as to whether the underlying encryption has an affect on those attack vectors?
For many years, the file system has been encrypted with 256 bit keys, and different keys per file. (I know this is supposedly an advantage of the new filesystem, but that's been there for years). And 256 bit means: Forget it.
The only way to read the files on an iPhone is to enter the right passcode. There's just no way around that. Now the problem was people using 4 digit passcodes (10,000 possible codes). Apple adds some security by adding delays when you enter the wrong passcode more than 6 times, whic
In-place File System conversion? (Score:1)
... let the bricking begin.
i just lost all my porn (Score:2, Funny)
I will never forgive you apple
Google? (Score:2)
Finder app comeing? (Score:2)
Finder app comeing?
Late to the dance? (Score:2)
I believe we have had those devices for some time now.
Another copy on write FS (Score:2)
A major problem with copy on write is that users can not scrub their data and that has to be done by a root user.
The Posix committee needs to get its act together and provide a F_OVERWRITE fcntl system call that says "when I write a block back to the disk for this file, put it in the same place".
As an example of why this is needed: /dev/sda
echo "123SomeMagic" >file
echo "XYZZY123" >file
grep "123SomeMagic"
You get the same results if you do a open, write, sync, seek 0, write.
dd if=/dev/zero of=file&r
Re: (Score:2)
Since the filesystem is designed for SSDs, it will presumably TRIM the freed blocks, and automatically wipe the data.
If not, I believe it's encrypted by default anyway, so finding old encrypted blocks won't do you much good.
Re: (Score:2)
The raw block device will give you decrypted blocks on all 3 of the file systems mentioned. Doesn't anyone test this stuff anymore?
Re: (Score:1)
True. But then back in your day you used two digits to represent the year.
Re: (Score:2)
HFS+ was introduced in MacOS 8.1 (a point release of Mac OS 8). HFS in System 2.1 (following 2.0).
Re: (Score:1)
http://dtrace.org/blogs/ahl/20... [dtrace.org]
Re: (Score:2)
Wait. I thought you said a 'modern file system'. You know....like ZFS.
You need to recalibrate your sense of time. APFS was announced last year and launched today. ZFS has been around for a decade or so at this point. I'd wager you couldn't name a production-ready filesystem intended for widespread use that's newer than APFS without doing a search for one.
To be clear, I have no problem with ZFS. It has its shortcomings and strengths, just like anything else. I'm considering using it with a NAS I'm looking at putting together (via UnRAID). But this suggestion that APFS isn't "m
Re: (Score:2)
Every other filesystem designed since ZFS has taken this lesson to heart.
So, I take it you're unaware of ext4? It's arguably the most wide-used filesystem designed since ZFS, but it doesn't support the end-to-end checksums you're talking about.
Re: (Score:2)
CPU (especially for operations like XOR for checksumming) is basically free.
'Premium storage media'. I've worked for vendors in the enterprise storage space for the past 17 years or so. Even the most expensive drives still fail in spectacular ways, from the oops (you asked to write to block 64 but it actually wrote to block 2048, but only 1/billion operations, and that's silent corruption) to the catastrophic (flying height issues, bearing issues, oil issues).
SSDs, while much better, also have software (f
Re: (Score:3)
Oh, zfs! I remember it corrupting all my file systems while I was a university student. Maybe it was the Linux implementation, I don't know, but I'm never touching that fs again. Also my NAS corrupted its file system more recently and guess which fs it was?
I have been running ZFS for ~5 years now across ~40 servers. Never failed once.
I even had one particular server with 8 drives in a RAIDZ2 lose a drive while two others started failing. I replaced them all successfully. Less than 12 hours later two additional drives failed and a third started acting flakey. I replaced them all successfully. No data loss. I'll trust AUFS when I see it handle weird hardware BS like that.
Re: (Score:3)
I'll second the observation. I have found ZFS (on both FreeBSD and linux) to be phenomenally reliable through several years. I have no worry at all when power failure hits my 75 TB worth of RAID-Z2 and RAID-Z3 storage pools using a total of 25 drives. I don't even bother with a UPS. Disclaimer - my pools are only occasionally written to; mostly read.
Re: (Score:2)
my 75 TB worth of RAID-Z2 and RAID-Z3 storage pools
My God! It's full of porn!
Re: (Score:2)
And I have run HFS and HFS+ since 1984 or '85, on several dozen machines. Never failed once.
Now what?
Have you recovered 100% of your data from a 6 our of 10 drives failing within a 48 hours window?
Can you yank the drives out of one machine, put them into a pile, and then randomly plug them into another machine and access your data?
Can you do that while the OS is running?
How about in the middle of writing data?
Without running chkdsk or fsck?
Re: (Score:2)
Can you boot from your ZFS drives?
Yes. Is there some reason I shouldn't be able to?
Re: (Score:2)
News to me. When did they get that working?
That was one of the main reasons that Apple didn't adopt it back in the Leopard days, when they had that experimental ZFS driver running.
Not sure if you missed my original post or what...but I don't use Apple crap *because* of the brain-damaged 'features'. Like APFS. I use ZFS on FreeBSD and occasionally Linux. Booting works fine from both, although FreeBSD has a nice polished installer to do it for you. In Linux it's a very manual process to set it up. As for ZFS on Apple? I have no clue. In two decades I only ever had one client that absolutely couldn't be switched from Apple to FreeBSD or Linux to achieve better results.
Re: (Score:2)
Well, I was simply asking when the ZFS folks got booting working, because it used to be a problem IIRC.
It's been working fine on FreeBSD and Linux.
But since you can't hold a rational discourse with spewing Apple Hate, I guess we're done here.
Yeah, I hate apple. I have good reason to. I just listed one--they went with their own shit-tastic filesystem and last time I checked, they aren't open enough to boot from ZFS.
It may surprise you, but my work computer is a piece-of-shit Macbook Pro. I am forced to use it by policy. It's *terrible*. If I don't fit the 'Apple mold' and just use brain-dead point-and-click applications all day, I end up with segfaults, performance problems, and out-of-memory
Re: (Score:2)
You've got something hardware-related wrong with that MacBook.
Apple's own diagnostics tools say the hard drive and memory are fine.
Apple's own techs (the company shipped it off to Apple) say it's fine.
I have had exactly ONE Kernel Panic (what you are incorrectly calling a "Seg-fault")
Nope. I'm not incorrect. The kernel has never panic'd on my Macbook. But lots of applications (like 'vim') will suddenly dump me back to the command line with a segfault message.
And before you say that is because all I use are "approved", "safe" applications, then perhaps you should look to the quality of the APPLICATIONS, and stop blaming the PLATFORM. If you search for "vim crash os x", you will find a long and storied history of that simple Editor being an unstable POS. Don't think it's the Mac's fault. It's a fucking EDITOR, FFS!
Weird. Vim workes perfectly for me on Linux, FreeBSD, and Windows. No crashes. Must just be a Mac thing.
Anyways, the Mac may be perfect for you. It's definitely not for me. Ignorin
Re: (Score:2)
And I have run HFS and HFS+ since 1984 or '85, on several dozen machines. Never failed once.
Now what?
From wikipedia:
Apple File System uses checksums to ensure data integrity for metadata, but not user data.
I'm glad ZFS cares about the integrity of *my* data, not just its metadata.
Re: (Score:2)
Well, since APFS has been in development for only a year or so ...
Uh, actually, AFPS has been in development for several years.
Re: (Score:2)
Re: (Score:2)
#0. More emojis!