Inside Apple's iPhone Software Shakeup After Buggy iOS 13 Debut (bloomberg.com) 55
Apple is overhauling how it tests software after a swarm of bugs marred the latest iPhone and iPad operating systems, Bloomberg reported Thursday. From the report: Software chief Craig Federighi and lieutenants including Stacey Lysik announced the changes at a recent internal "kickoff" meeting with the company's software developers. The new approach calls for Apple's development teams to ensure that test versions, known as "daily builds," of future software updates disable unfinished or buggy features by default. Testers will then have the option to selectively enable those features, via a new internal process and settings menu dubbed Flags, allowing them to isolate the impact of each individual addition on the system. When the company's iOS 13 was released alongside the iPhone 11 in September, iPhone owners and app developers were confronted with a litany of software glitches.
Apps crashed or launched slowly. Cellular signal was inconsistent. There were user interface errors in apps like Messages, system-wide search issues and problems loading emails. Some new features, such as sharing file folders over iCloud and streaming music to multiple sets of AirPods, were either delayed or are still missing. This amounted to one of the most troubled and unpolished operating system updates in Apple's history. The new development process will help early internal iOS versions to be more usable, or "livable," in Apple parlance. Prior to iOS 14's development, some teams would add features every day that weren't fully tested, while other teams would contribute changes weekly. "Daily builds were like a recipe with lots of cooks adding ingredients," a person with knowledge of the process said.
Apps crashed or launched slowly. Cellular signal was inconsistent. There were user interface errors in apps like Messages, system-wide search issues and problems loading emails. Some new features, such as sharing file folders over iCloud and streaming music to multiple sets of AirPods, were either delayed or are still missing. This amounted to one of the most troubled and unpolished operating system updates in Apple's history. The new development process will help early internal iOS versions to be more usable, or "livable," in Apple parlance. Prior to iOS 14's development, some teams would add features every day that weren't fully tested, while other teams would contribute changes weekly. "Daily builds were like a recipe with lots of cooks adding ingredients," a person with knowledge of the process said.
Not solely an Apple issue (Score:2, Insightful)
"DevOps" translates to "work faster", and "work faster" translates to "more bugs".
Feature flags or not.
It's not a bug, it's a FEATURE! (Score:1)
Just be glad Apple has finally gotten a handle on their real bugs [businessinsider.com].
Although, given their heavy handed product pricing and draconian app store policies, they're still a bunch of blood suckers in my opinion.
Re: (Score:1)
Can't argue with that.
In fact, I'm generally against sales pitches for consuming any apple in a walled garden.
Precedent indicates it won't have a good ROI.
Re: (Score:2)
Yes, Apple's products are priced near the top end of their markets. They've pretty much always been that way so that didn't exactly sneak up on anyone.
Re: (Score:1)
How does a company become "heavy handed" when pricing a product that no one has to buy and has a large number of alternatives made by other companies?
You can buy iPhones or iOS devices from other companies?
Obtuse (Score:2)
There's no entitlement to having multiple companies produce something called iPhone. Just as Buick doesn't make Ford Mustangs.
Re: (Score:2)
ITYM "Just as Buick (Encore) doesn't make the Chevy Trax."
Re: (Score:2)
Re: (Score:1)
No, no one missed the word. But you make meanwhile pretty clear: you are an idiot. ... perhaps you missed word: iOS, so I repeat it for you iOS!
Thee is no other phone or tablet that runs iOS
Re: (Score:2)
DevOps are people who do the "operations" for "developer machines". ... they usually are not involved in the development of the actual software, hence no work harder or faster or more bugs.
No idea what you think they do
Perhaps you want to google DevOps jobs and check the required skill sets to be a devop.
Re: (Score:1)
Really? I do software development for one one of the largest fintech companies in existence, and the underlying implications (and expectations) of "enabling continuous integration and deployment" (CI/CD) is quite a bit clearer in actual practice than you surmise.
And your description of "DevOps" is rather... well, extremely narrow. In reality, it is a broad set of practices intended to remove the traditional distinctions ("silos") between "Development" and "Operations", in effect, making developers respons
Re: (Score:2)
DevOps is not a development methodolgy, it is a job description, e.g. your example of maintaining CI/CD servers.
And it is not correct to assume the developers of the team developing are also doing the devops part ... the bigger the organization the more likely that this are not even different teams, but different departmens.
So, yes, Really!
Re: (Score:1)
Well, okay. You are simply wrong. If you don't like Atlassian's description (a leader in the industry and one for which my company is a major partner), try the -very first sentence- of Wikipedia [wikipedia.org].
"DevOps is a set of practices that combines software development (Dev) and information-technology operations (Ops) which aims to shorten the systems development life cycle and provide continuous delivery with high software quality."
In fact the history of a large company tends to be separate teams, then departments
Re: (Score:2)
Sorry, the wiki pedia article is wrong.
I work often enough as devops, the quote you make is at max a very losely fitting description-
There is no software development process that is "dev ops".
In fact the history of a large company tends to be separate teams, then departments, of specifically "Dev" and specifically "Ops"
Exactly, and is why now have devops as a bridge between dev and ops, or even replace ops with devops, but we don't replace the developers with devops. If the team is small, or you only have a
Buggy Contacts too (Score:2)
Re: (Score:2)
So far, I've only encountered one serious bug.
Unfortunately, that bug is that swiping to answer a call doesn't work reliably.
Seriously. This is a phone that costs more than a small car, and it can't answer calls properly. I really don't care about app this or store that. Just make the damn phone at least work properly as a phone before you ship it!
Re: (Score:2)
Nice try, Apple Marketing Department.
Re: (Score:2)
QA Testing daily builds? (Score:4, Insightful)
Re:QA Testing daily builds? (Score:5, Informative)
Apple has always had very limited QA testing and fairly limited automated testing, instead relying heavily on developers "dogfooding" everything as the main means of finding and reporting bugs. It has never worked all that well, but it worked well enough to prevent the majority of serious bugs from creeping in, so long as there were only a limited number of configurations, and subject to the requirement that old code eventually got replaced en masse as part of some bulk deprecation of entire APIs before it got too fragile and broken to maintain.
That kind of worked when Apple was still a small company, because they didn't have that many developers working on things. And yet, I distinctly remember when OS X v10.3 came out and started eating hard drives. I remember when the original Apple Maps was widely panned, etc. The problem with dogfooding by engineers is that real-world users are a much more diverse group than your developers, so it can only tell you when something is hopelessly broken, not necessarily prove that the tool will meet the needs of users.
Even back in the early days, I used to comment that I wouldn't install any new version of OS X or iOS until the minor version number exceeded the major version number, because each major release started out less stable than the previous major release.
But when iOS came out, their dogfooding-centric model broke badly, making their waterfall development methodology a hopeless nightmare in which the last couple of months before a release were basically just nonstop bug fixing, and even then, things were always pretty broken at release time, with nearly every OS having a .1 release that occurred simultaneously with the OS release, because they had to freeze the .0 release to ship with new hardware.
But unfortunately, rather than fix the root cause — the lack of tests — the powers that be decided that the problem was the waterfall methodology, and moved to some approximation of Agile, with shorter sprints, so that things could converge to the point of being able to usably develop on them frequently enough that work could get done. But like the previous model, that only works for so long before the base state becomes too unstable to work with.
So now, like most companies, they're moving to using feature flags, allowing them to turn off unreliable features so they don't break developers while they're trying to get work done. And that's fine and good, but in a few years, they'll hit a wall again, with them finding that critical bugs accidentally ship in a release because developers are testing with different flags than whatever ends up in customers' hands — either because a feature that they thought would ship didn't or vice versa.
There is only one cure, and that is writing mountains of unit tests and functional tests, ensuring that every interesting code path has coverage, ensuring that expected user actions result in the expected behavior under the hood, validating not just the expected cases, but also sensible behavior in failure cases, etc. My guess is that Apple as a whole probably needs an order of magnitude more unit tests and integration tests, and probably two orders of magnitude.
Unfortunately, writing tests takes time and requires a deep understanding of how to write good tests that are effective without being flaky. A company like Apple that treats QA testering as entry-level jobs for people who want to become real engineers isn't likely to ever reach that level of testing expertise. To truly fix their structural problems will likely require massively increasing the size of Apple's development team, hiring dedicated QA engineers that aren
Re: (Score:2)
> There is only one cure, and that is writing mountains of unit tests and functional tests,
hey, we wrote this piece of software, we have no idea if it is correct. What do we do?
I know, write more software for it.
Imagine if algorithms were designed this way
we would think bogosort to be the pinnacle of software engineering
and this from a 5-digit?
the current ecosystem of programming and software is unspeakably bad.
Not just that... (Score:2)
hey, we wrote this piece of software, we have no idea if it is correct. What do we do?
I know, write more software for it.
And who should write the software to determine if something is correct? Why the same people that wrote the original code of course, that have the same assumptions about what is correct as the code you are trying to test.
There's no replacement for a great QA team, testing is only of limited use and also can slow down development quite a bit if you have a LOT of tests.
Re: (Score:2)
That's largely what your code reviewers should be doing — looking at the test coverage to see whether every branch is followed, looking at the test cases and see whether they omit any obvious inputs, and so on. And of course, when someone uses the API later and discovers that somethin
Re: (Score:2)
Agreed. This is essential, regardless of any other testing.
Unit testing helps prevent easily testable issues from landing in QA and causing a boing effect when they keep bouncing it back to dev. I am sure there a plenty of QA teams who wish the developers would do a bit more thorough testing (unit or manual) before sending it to them.
This is a given, it WILL slow down development, since time must
Re: (Score:3)
Re: (Score:2)
Even back in the early days, I used to comment that I wouldn't install any new version of OS X or iOS until the minor version number exceeded the major version number, because each major release started out less stable than the previous major release.
Strange policy to follow ... for iOS the minor version has never exceeded it's respective major version, has it? In some cases the bug revision exceeds the minor version, example: iOS 4.3.5
There was only one major version number for OS X ... 10, with 10.11 released on September 30, 2015 and not since 10.6.8 on July 25, 2011 has a bug revision exceed a minor.
Maybe i'm not getting it... definitely wait a minor version or two.
Re: (Score:2)
Yeah, it was kind of sarcasm, but only halfway.
Re: (Score:2)
Unit tests are overrated. Unless you sell a library they rarely make sense, especially not for "Apps" which only consist over very few code.
The OS itself needs unit tests. Everything else needs story/use case based GUI/UI driven tests, that hopefully cover every line of code and integrate over every external system.
In static typed languages unit tests for every class are just nonsense. It is early enough for writing them, if you really want, before you refactor code. However if you have the automated UI dri
Re: (Score:2)
Re: (Score:2)
Apple isn't the only one that lacks good QA testings. :(
Re: (Score:2)
True. Doing proper QA is the exception, not the rule.
Re: (Score:2)
Re: (Score:2)
A better strategy would be: stop rewriting every app all the time.
I really don't get why the iOS 6 iBook app needed any fucking change.
Probably it is important for them to rewrite every app in Swift, who knows.
Surely the Opposite? (Score:4, Insightful)
> The new approach calls for Apple's development teams to ensure that test versions, known as "daily builds," of future software updates disable unfinished or buggy features by default.
Surely they mean all the features will be on so the testers will actually experience the bugs but be given instrumented controls so they can bisect the new functions to make a decent bug report?
If they're all off by default very few testers will ever test all the new features which would decrease code quality further. Because humans.
Unclear summary, they did mean that (Score:2)
Surely they mean all the features will be on so the testers will actually experience the bugs but be given instrumented controls so they can bisect the new functions to make a decent bug report?
If you read the article the point of being able to switch new features on and off, and having them all off by default, is that more people will adopt new builds sooner because they will still have a stable phone, and be able to turn on/off individual features to test without having to use an unstable device all the t
Safari (so goody) (Score:3)
Is it just me or does every major release of iOS seem to reintroduce old Safari bugs like edit boxes that don't work or stuck web pages after x views. It's like they start building the new version of Safari from the original slate tablets with different features and forget to add in the bug fixes.
Re: (Score:3)
Oh, probably. Apple doesn't seem to do regression tests (and this new strategy doesn't seem to fix that) because bugs constantly reappear. iOS 12.4 - released earlier this year - had a regression that allowed jailbreaking due to a flaw that iOS 12.3 had fixed. So, yeah. Bugs keep on returning and being fixed and then being reintroduced.
The thing I find hilarious is that despite iOS 13 being a horribly buggy release, it still fixed enough bugs that iOS 12 never fixed that you ended up having to decide which
Re: (Score:2)
The last update certainly made my iPad far worse, can’t scroll without slowing down quickly any more showing grey screenwhen I do.
Lost focus again (Score:4, Interesting)
They should have stuck to their original plans to focus on stability, reliability, and performance [slashdot.org].
Didn't affect me (Score:3)
I was unaffected, and iOS 13 has seemed fine to me. But then I have auto updates turned off and eventually get around to installing the new major versions a few weeks after every else got done testing it and the emergency patches are all in place.
Build/Merge Controls (Score:2)
Does not Apple have anything like build masters who are in control of when features are ready (read complete here to where a developer gets yelled at if completely broken) to be merged into say master (I am using Git parlance but all SCMs have the same basic features) and then trigger builds for test? Am I reading this as everyone has basically the authority to merge into master (or a build branch)? Are the crazies actually running the asylum?
I guess I am just to darn old school to where I place the same le
Re: (Score:3)
Some teams do, but at least historically, it happened on a per-te
Is this why my LTE stops? (Score:2)
For the most part I haven't noticed problems. I have to appreciate it though because Apple has been notoriously slow to introduce features into iOS - for years I've looked at Android friends who had cool new features every week. So now maybe Apple is ramping up software dev to keep us all happy - after all we aren't buying new phones :-) And scaling up brings problems. I do find it hard to believe that they will be able to turn off features and have things work - that no interdependence will exist.
Re:Is this why my LTE stops? (Score:4, Interesting)
Sure, maybe that's why you're having problems.
What I've noticed is that iOS seems to have a maximum number of connections (socket connections? web connections? not sure), and once it exceeds that limit, it will still let apps create a connection but these connections won't start until the number of open connections goes below that limit. So if you have something that attempts to open a bunch of connections, and some of those connections fail for whatever reason, you can end up having to wait for connections to finish timing out before the ones stuck in the queue get a chance to run.
This can happen on WiFi too, it's just more likely to happen on LTE because LTE is less reliable so you're more likely to get failed connections that you have to wait to time out before you can get past them.
What used to be able to resolve the problem, but doesn't seem to in more recent iOS versions, is toggling Airplane Mode on. With all the network devices off, all connections would instantly cancel, emptying the queue and letting you retry. Sadly, Apple seems to have "fixed" this, and instead of just immediately turning the radios off and terminating all connections, seems to attempt to cleanly close all connections and waits for them to either close or time out before turning the radios off.
Of course who knows. None of this is documented. But that's the way it seems to work. My guess would be that Amazon Music is opening new connections per song, and that enough of them are entering a zombie state that eventually you hit that limit and have to wait for things to time out before anything else is allowed to work.
Re: (Score:2)
That explains why my podcasts hang when I have a busy work and school week and can't update things for a few days, and then tell it to refresh the podcasts - it seems to hang while it's downloading 250 podcasts.
You'd think they'd have a pipeline queue management system that would only open the ones it can handle (or minus a few for extra connections).
Horrible mistake to join Beta in September (Score:1)
Apple way (Score:2)
Yeah, when I was at Apple it was challenging even on Mac OS new versions. Often there was not a layered process of getting lower level subsystem new features ironed out before new features of systems that were its clients. And truthfully it is not always possible to do such meaningful layering in a complex software system like and OS and its development tools and build in house fully. But there were days, and even weeks when my team could not really progress on its new deliverable items because one or mo
Can't understand (Score:2)
I could live with random software crashes .. (Score:2)
I had to upgrade to 13-beta to revive the charge. Now it's showing 16h with light/regular use and 84% battery condition.