Slashdot Log In
Mac OS X Secretly Cripples Non-Apple Software
Posted by
Soulskill
on Thu Feb 28, 2008 09:17 PM
from the hand-in-the-cookie-jar dept.
from the hand-in-the-cookie-jar dept.
spikedLemur writes "Vladimir Vukicevic of the Firefox team stumbled upon some questionable practices from Apple while trying to improve the performance of Firefox. Apparently, Apple is using some undocumented APIs that give Safari a significant performance advantage over other browsers. Of course, "undocumented" means that non-Apple developers have to try and reverse-engineer these interfaces to get the same level of performance. You really have to wonder what Apple is thinking, considering the kind of retaliation Microsoft has gotten for similar practices.
Related Stories
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
first post! (Score:5, Funny)
i cheated though, i'm using safari.
Um, is this an emulation thing? (Score:5, Insightful)
So writing this from a native perspective introduced new APIs found in tech notes you should have read in the first place before writing and running into performance issues?
Parent
Re:first post! (Score:5, Funny)
Parent
Re:From TFA... (Score:5, Insightful)
Seriously though, your post was really hard to read. When you referred to OS X as "X", I was thinking "X Windows". Please, for the sake of everyone here and Slashdot reputation, declining or not, refrain from using such atrocious techniques. Really, who uses "%" instead of typing "percentage"? It's not that hard.
Parent
Re:From TFA... (Score:5, Insightful)
Parent
Re:From TFA... (Score:5, Funny)
Parent
Re:From TFA... (Score:5, Informative)
That is *the* first time I have seen anyone refer to OS X as 'X'.
Very confusing.
Parent
No. (Score:5, Informative)
Parent
Re:first post! (Score:5, Insightful)
Or the SPI (System Programming Interface, the private equivalent of an API) takes advantage of inside knowledge of how some data structures are designed but which could change in the future as functionality is added to the class in question. For example, Apple might decide to change CFString to always convert data to BADC-byte-order UTF-32 under the hood for better efficiency on Vax. Not likely, but I never thought I'd see Macs using Intel CPUs, either, so you never know. :-D They could make such a change and still support the public APIs, but if they had an API that allowed you to arbitrarily manipulate the bytes under the hood, they'd be stuck.
The thing about Safari is that it is effectively insulated from SPI changes because it comes out with OS releases. If Apple needs to change an internal data structure in CF, Foundation, etc. in a way that would break an SPI, all they have to do is rev Safari to not use that SPI. Thus, it is safe for Safari to use any API or SPI. If Apple publishes the SPI as API, FireFox uses it, and Apple changes the data structure, Firefox breaks, and with "luck", so do Photoshop and Word.... :-(
So you see, Apple has only three choices: A. don't publish that portion of the API, in which case some people complain because they're not able to get that extra 1% from being able to walk inside private data structures of the HFS+ Extents B-tree or whatever, B. publish that portion of the API, in which case they're stuck with that internal architecture and can't ever change it to improve performance, add features, etc., or C. publish the API and break it later, in which case developers scream again. It's a no-win.
The only thing one could possibly argue is that Safari shouldn't be using the SPI, either, to put them on equal footing. That said, since it's safe for them to do so, where's the harm? There's no monopoly involved, certainly. :-) And as you noted, many of those SPIs that Safari is trying out might become APIs at some point in the future. Having an app like Safari exercise them allows the engineers to figure out what works and what doesn't so that they don't get stuck supporting an API that isn't scalable, is hard to enhance, or isn't easily maintainable. In the long run, everybody benefits.
I'm certain that Apple doesn't do this to cripple Firefox or to make its own software look better. It's not a vast fruit-wing conspiracy.... Apple limits its public API exposure to ensure that the APIs are sustainable so third-party code doesn't break. If you don't care about that, use the SPI... just don't come crying when your app crashes on launch after a software update or whatever.... :-)
Parent
Re:first post! (Score:5, Insightful)
Great post.
I've been in product cycles where we've gone through exactly the dilemma you pointed out: where making an API public means supporting it until kingdom come, when the scenario is too new for the API to be stable, or you have definite long-term plans that will cause breaking changes in the API and you don't want the burden of having to be backwards compatible with applications designed for the older API.
The only part I didn't agree with was this:
I think it should either be ok for all players to have internal APIs or not ok for all players. I mean, if we say that right now it's ok for Apple to do this because they are not a monopoly, what happens if they do become one? At that point do they get penalized for these internal APIs (using which they designed the products that helped them obtain the monopoly)? At what point will Apple cross a threshold at which they need to change this practice and how will they know when they have crossed it? And when this threshold is crossed, is it suddenly ok for MS to start this practice again (of having undocumented APIs).
Parent
Article is a Troll (Score:5, Insightful)
Re:Article is a Troll (Score:5, Informative)
Parent
Re:Article is a Troll (Score:5, Insightful)
Parent
Re:Article is a Troll (Score:5, Insightful)
Parent
Re:Article is a Troll (Score:5, Insightful)
Since this is exactly one of the reasons how Microsoft came to dominate the software market, and had all major third parties kowtow to them (and pay) to get the information, the Free Market was distorted. It would not be the best/fastest application that grabbed the market, but the one with knowledge about and rights to the secrets.
I'd have to seriously disappoint you on this one: This is exactly not what the term 'Free Market' means, especially if you are already the monopolist.
You yourself might already have grown up, now try to work on your thinking abilities.
Parent
Re:Article is a Troll (Score:5, Insightful)
Also, There is a 100% documented, public, and simple way to disable the feature. The Firefox dev found this configuration, added 2 lines of XML to firefox, and bam, done, speedy. So I really don't see the comparison to MS at all.
Also one of the comments on the blog is from a webkit developer at Apple who says "yeah, these APIs basically suck, and they are here for backwards compatibility with Tiger, and they aren't stable, and cause us hundreds of hours of work dealing with regressions, so don't use them, use the perfectly acceptable and documented configuration setting, if there is anything in these APIs that should be made public, it will be once it is stabilized and reliable" He then gives examples of other APIs that have gone through the same process.
In the end this is 100% open to the public, any software can use this configuration setting to get around this potential performance bottleneck. The reason FF3 was "suddenly" slower than FF2 is they changed from Carbon to Cocoa (2 totally different frameworks) and the new feature is only applied to Cocoa apps. So in short, FF changed hundreds (probably thousands) of lines of code to use a new framework, and found a performance bottleneck, and then found the documentation about it, and changed configuration to avoid the bottleneck... How this is news at all baffles me, that sounds like a normal day in my life.
Parent
Article points finger in wrong directoin (Score:5, Informative)
Indeed what apple is doing does not seem that out of whack. They have an interface that is optimized for stability not speed. That's the proper way to do it. and they figure out how one can tweak it for speed. Do you make that the defaults or do you put those in a container like webkit where one can manage the tradeoffs better? duh...
Parent
Re:Article points finger in wrong directoin (Score:5, Insightful)
The FTA even makes it clear - FF3 got the speed advantage they wanted, using the public API. The FTA even has an addition making it clear that the Slashdot article is taking the wrong slant. 'nuff sad.
Parent
Re:Article is a Troll (Score:5, Informative)
Parent
Except it IS documented. (Score:5, Informative)
-- Terry
Parent
Re:Except it IS documented. (Score:5, Informative)
Parent
Re:Article is a Troll (Score:5, Informative)
Introspection makes this easy, as in java (and c# I suppose--I'm just guessing for this one).
Also, given Apple did mention it into a post (see above reply) and that although there wasn't documentation per say, the functions were in a published API (in this particular case), one could say "the plans where on display".
Parent
The thinking is really quite simple: (Score:5, Funny)
*sniffle* (Score:5, Funny)
Parent
Round Number? (Score:5, Funny)
From the fucking comments (Score:5, Informative)
Feb 28th, 2008 at 1:24 pm
The programmatic disabling of coalesced updates should not be public API. It's actually a very dangerous thing to do. We aren't really happy with that code in WebKit, but we had to do it to avoid performance regressions in apps that embedded WebKit. Technically it's wrong though, since we turn off the coalesced updates for any app that uses WebKit! This includes drawing they do that doesn't even use WebKit.
As for the window display throttling, that was a pref designed for Safari (that we don't even use any more). It's not private or magic. It's nothing more than a pref that we can examine from Safari-land, so linking to that is just silly.
Many of the private methods that WebKit uses are private for a reason. Either they expose internal structures that can't be depended on, or they are part of something inside a framework that may not be fully formed. WebKit subclasses several private NSView methods for example, and it cost us many many man hours to deal with the regressions caused by the internal changes that were made to NSViews in Leopard.
As you yourself blogged, there was a totally acceptable public way of doing what you needed to do.
For any private methods we use that we think should be public, we (the WebKit team) file bugs on the appropriate system components. Many of these methods have become public over time (CG stuff in Leopard for example). Be careful when you dig into WebKit code, since we may continue to use the WK method even though it's not public API just because we need to work on Tiger.
Re:From the fucking comments (Score:5, Insightful)
Parent
Re:From the fucking comments (Score:5, Insightful)
Parent
Re:From the fucking comments (Score:5, Insightful)
So, no, you aren't getting it right.
Parent
Re:From the fucking comments (Score:5, Insightful)
Webkit is a framework that is open for anyone to take and put into their own application. Safari and some components of Mac OS use WebKit for their own rendering of html. There is no private, "better" linkage to WebKit, there is just a hack within the WebKit framework that is there so that other applications using WebKit will not have problems with it. The Apple developers knew the internals of the operating system well enough to do this semi-safely but even they aren't happy with themselves doing it because it can still cause problems.
There is also a public, safer, more documented way of doing the same thing for applications that don't use WebKit. This public method is not perfect either but it is much safer. The Apple developers are keeping parts of the operating system under wraps which could cause major problems if you don't know EXACTLY what the internals are doing. This is a very common thing for responsible developers to do, you don't want to expose API that could fail catastrophically if something isn't set up just exactly correctly.
In short, nothing to see here, the public API is the safest bet to use. If you choose to use undocumented methods for a bit more speed then you risk bringing down your application in a hard and messy way. The WebKit developers weighed that in their own minds and decided that the risks were worth it, since they had a hand in developing the undocumented methods and could account for the quirks in a semi-safe manner.
In the case of Internet Explorer, Microsoft had a separate set of completely safe API that were optimized versions of methods other developers got to use. If you were an internal Microsoft developer you could do more with the internal API than anyone could with the external API. This was done deliberately so that Microsoft products could get preferential treatment on the Windows operating system. Microsoft also made it so that you couldn't easily use Windows without having some part of Internet Explorer as part of the system. Under Mac OS X you can remove every mention of WebKit and all that will happen is a couple of programs won't work until you re-install them with their embedded versions of WebKit.
Parent
Re:From the fucking comments (Score:5, Informative)
I think you're reading too much malignancy into this, and searing about it and dropping the I-E word doesn't help :).
It totally is, nothing up our sleeve there, though you don't need webkit to run Finder.app, though under Leopard I bet the QuickView system will break. I think the main complaint about IE was that you can't actually delete Internet Explorer.exe from your Windows system, and it was a finagle to keep Windows from favoring IE for content. Safari.app can be deleted from a Mac with no effort, because Apple actually separated the web rendering system into a library, like the MS people were supposed to.
You put "better" in quotes because you probably anticipated where the issue is. The OS throttles display updates to the framerate of the display when you run a Cocoa application; this was done to make the drawing to the framebuffer look cleaner and for efficiency reasons (you can read about it here [apple.com], it's from TFA). If you are building a Cocoa app and want to opt-out of the beam sync, you have to set this option in your Info.plist, which is just a setting built into your deployment (it's really easy, and documented, but they suggest you not do it because it might leave you with a faster-rendering but ugly result). Setting the option in the Info.plist is global for your app, from launch to exit.
WebKit makes use of beam sync on/off, but uses a call into the system to turn it off only at certain times (this is my understanding of Hyatt's explanation). Hyatt, a former Firefox dev himself we might add, himself says this is a hack, and that if you actually expose this functionality to vendors you're totally going to be loading the gun and pointing it at peoples feet.
So what do you do if you're Apple? You can offer people a function that'll turn this efficiency feature on and off, and a few devs might like this, but a ton won't care, and if you do decide to support this, you've gotta make sure it works forever for everybody and perfectly.
Parent
Re:From the fucking comments (Score:5, Insightful)
Apple not being a convicted monopoly, this may be an acceptable practice, but technically, this is exactly the same thing (actually, one of many things) people accuse MS of, regardless of the underlying motive, and to argue that this is somehow different is dishonest.
The WebKit team has created a framework which is free and open for anyone to use. In order to make this framework as compatable as possible they used some undocumented methods in Quartz, the drawing layer of Mac OS X. Yes, they are also Apple developers and they have intimate knowledge of the internals of Mac OS X. This is why they feel reasonably safe in doing something as unsafe as using undocumented methods for means which they were never intended. They didn't do it because it provided some sort of advantage to WebKit over other applications, they did it because it kept WebKit from breaking some applications that embedded WebKit.
The Mac OS X developers also have a documented, public way of doing this very same thing and the Firefox developers used it. It worked well and everyone is happy now. In fact when you compare the public way of solving the problem and the behind-the-scenes way of doing it you find that the behind-the-scenes way is much more difficult to work with and more likely to have problems down the road.
Microsoft, on the other hand, has a history of developing two different layers of its API, both of which are equally safe. The private API is only shared with internal Microsoft Developers and is much quicker/easier/more efficient than the public API. This is what has gotten Microsoft in hot water before.
It's a far cry to say that Microsoft's dual API is at all comparable to Apple's public API and the undocumented methods being used here. If Apple was truly doing the same thing as Microsoft then the undocumented methods would do the job more easily and efficiently than the public API. They don't, they are just a hack that only an internal developer could come up with to make sure OTHER people's applications keep working well. If you look at the developer's (David Hyatt's) comments he even says that they don't use this hack in Safari, Apple's own web browser. It's meant so that other people's browsers can work well.
I'd say the dishonest thing here is your feigned innocence over your comments. "I'm not trying to read too much into it." - yes you are! All of this was explained to you by several people in several different ways and yet you still came back to try to further muddy the waters. Just admit that either you have no clue about the whole situation or that you do understand the difference between Microsoft's and Apple's behaviors and you are just trying to stir up trouble.
Parent
Display Throttling? (Score:5, Insightful)
Mac OS X 10.4 introduces a new behavior of coalescing updates that enables Quartz to more efficiently update the frame buffer during each display refresh. In addition to increasing system efficiency, Coalescing updates improved visual consistency and eliminates "tearing" during scrolling and animation. To coalesce updates, the Quartz window server composites all window buffers into a single offscreen frame buffer before flushing it to the screen. When your application issues a command to flush, the system doesn't actually flush that content until the next available display refresh. This allows all updates for multiple applications to happen at the same time. Window server operations (window resize or move, for example) are handled in the same manner--coalesced into a system-wide screen update.
I would assume Apple would be thinking this makes a lot of fucking sense.
They give app writers a way to turn it off if need be. What the hell are we crying about again?
Re:Display Throttling? (Score:5, Insightful)
You are bang on.
Display coalescing was introduced way back in 10.4. This was well documented, and as the Firefox developer points out, there's even tools provided to enable and disable this behavior to see if you're being limited by it.
While I don't know all the details, I would venture to say that far from Apple "Secretly Crippling Non-Apple Software" (which they conveniently provide release notes and debug tools with the stock Xcode tools for this secret behavior), this has exposed a serious performance problem in Firefox.
To put it another way, display coalescing will effectively penalize apps that are rendering more updates that are physically possible to display. As an extreme example, this roughly equivalent to rendering / writing 60 frames of video to the display in 1/60th of a second, which can only possibly display one of those 60 frames. The rest are just wasting CPU, GPU, and bandwidth resources which could be better spent doing other things. Display coalescing will cause an application to "stall" if it's forcing too many updates, the call to flush the buffer just won't return until the the current frame has fully rendered.
Mac OS X prior to 10.4 did not enforce this, and so as one of those compatibility trade-offs, display coalescing is turned off by default when certain conditions are detected. I believe anything linked prior to 10.4 will trigger it, and carbon apps. Carbon is essentially for those who are unwilling to (almost literally) join the 21st century.
Just how secret can it possibly be with publicly available release notes published years ago?
http://developer.apple.com/technotes/tn2005/tn2133.html [apple.com]Parent
Tag "alreadyfixed" (Score:5, Informative)
Let's be fair now... (Score:5, Insightful)
Slashdot spreads more FUD (Score:5, Interesting)
Re:the difference (Score:5, Insightful)
It's ridiculous to try and use this insane rationale in regards to any company that's not Microsoft. At what point do you then start going 'well, actually I've decided they have enough market share now, NOW they should be ethical'
Bar and truly humbug
Parent
Re:the difference (Score:5, Insightful)
Parent
Re:the difference (Score:5, Insightful)
Parent
Re:Looks pretty secret to me. (Score:5, Informative)
Parent
Indeed, where's the advantage? (Score:5, Funny)
I can just see Steve Jobs rubbing his hands and gloating to his minions..."Yes, and with Firefox handicapped, we will have five percent of the browser market all to ourselves! Ours...all ours! Muahahahaha!"
Parent
Re:the difference (Score:5, Informative)
Parent
Garbage! (Score:5, Insightful)
"Most sane companies" do not do that. MOST companies at least make some effort to engage in Good Business, which involves both parties walking away feeling they got a good deal. That is a far cry from Microsoft's practices, which have largely been "Great! They're in the store! Now, quick, lock the door behind them before they can get away!"
Those are two very, very different approaches. It gained Microsoft a lot of marketshare... at first. But as anybody can see today, those tactics do not keep customers. It pisses them off. And once they find a way out, they tend to stay out.
Parent
Re:Proprietary software is monopoly software. (Score:5, Insightful)
Parent
Re:Proprietary software is monopoly software. (Score:5, Informative)
Don't confuse terms. A monopoly occurs when a company or group uses their market position (rather than the product's merit) to cause people to buy their product. Technically, it is quite possible that a Free software vendor could be guilty of monopolistic behavior. Believing that all propriety software vendors are monopolists is a clear logical fallacy as I have demonstrated, so please stop spreading misinformation. The merits of Free software do not increase because you post an irrational rant against propriety software.
Parent
Re:Really? (Score:5, Informative)
The Safari performance improvements are coming in Safari 3.1, not yet available to the public. To see them today, you have to be running current WebKit nightlies. The difference between the new WebKit builds and vanilla Safari 3.0.4 is pretty dramatic.
Parent
Re:Really? (Score:5, Funny)
You lost me there.
Parent
Re:It IS their right... (Score:5, Insightful)
"It's an undocumented feature. We're not going to document it because we don't want to support it. Feel free to use it, but don't bother complaining when it breaks sometime in the future."
Parent
Re:David Hyatt response (Score:5, Informative)
As for the specific problem encountered in firefox, there is a perfectly acceptable and PUBLIC way of achieving the same thing, so why use the API? Especially since as he mentions in his post, it is the WRONG way to do it.
Parent