Safari vs. KHTML

Johnny Mnemonic writes "CNET has a story that describes the divergence between the code base of Safari and KHTML. Although there were high hopes that Apple would contribute significantly to the OSS project, that optimism has all but disappeared. Is an unrealized danger of OSS that others may take your project in a direction you didn't intend? Can OSS code and goals harmonize with the goals and needs of corporation designed code? Is it that Apple mismanaged the relationship, or that the KHTML guys expected too much? Interesting warning for other OSS-corporate marriages." We've previously reported on the frustration in the OSS community on this issue.

Another question

[daveschroeder]

As long as they're abiding by the terms of the license, does Apple, any corporation, or any entity for that matter, have any obligation to contribute anything back to the project? Who gets to decide when someone is contributing "enough"?

Additionally Apple posts all of its open source code [apple.com]; here's the page for WebCore [apple.com], which states:

WebCore is a framework for Mac OS X that takes the cross-platform KHTML library (part of the KDE project) and combines it with an adapter library specific to WebCore called KWQ that makes it work with Mac OS X technologies. KHTML is written in C++ and KWQ is written in Objective C++, but WebCore presents an Objective C programming interface. WebCore requires the JavaScriptCore framework.

The current version of WebCore is based on the KHTML library from KDE 3.0.2. Changes that are specific to WebCore are marked with #ifAPPLE_CHANGES. Other changes to improve performance and web page compatibility are intended for integration into future versions of the KHTML library.

Sounds like a case of sour grapes to me. I'm sure the level of cooperation and collaboration that the KDE/KHTML/Konqueror folks had hoped for wasn't there, if only because Apple keeps everything secret before its release (including everything related to Safari 2.0 in Tiger). Another example of a corporate need butting heads with a contrary OSS philosophy. And I'm sure Apple's main priority is not developing an infrastructure to cohesively and voluminously contribute changes back to projects. It's more like, "Ok, here's our stuff... [apple.com]"...it's all there for anyone to see.

Safari on Windows?

[promantek]

In the article, Apple engineer Maciej Stachowiak said,
"One thing you may want to consider eventually is back-porting (WebCore) to work on top of (KDE)... We'd be open to making our tree multi-platform."

I wonder if that means they are looking to port Safari to Windows. It would give Windows users another taste of the Mac, and I for one would use it.

Here's a quote from Zack Rusin

[Lars T.]

he made after his rant.

Did KHTML become better as a result of Apple using it? Yes of course. KHTML became a lot, lot better as a result of patches we merged from Apple folks.

Re:Another question

[CableModemSniper]

OTOH this means that WebCore changes are hard to port to KHTML, but OTOH porting WebCore to GNUstep is easier, I suppose, since most Mac APIs are more or less the same there.

The Objective-C++ bits are actually making life difficult as far as a WebCore -> GNUstep port goes. Progress had gotten petty far but last I looked it wasn't really going anywhere. Browsing the CVS repository the most recent revisions are about 9 months old. On the plus side, when i tried it, it was pretty impressive. Drop a controller on a window in interface builder and write a line or two to send it to a page. it rendereed, clicking on links and stuff of that ilk tended to throw exceptions all over the place though.

Linky: https://gna.org/projects/gswebkit/ [gna.org]

new here, but...

[greystreets]

Is an unrealized danger of OSS that others may take your project in a direction you didn't intend?

Isn't that point of OSS, hoping that someone will take interest in your project and do something with it you couldn't do yourself?

And what's dangerous about that?

Re:Its only the bad things we head about?

[argent]

Apple just spit on their courtesy

Are you honestly saying that Apple deliberately made their patches as hard as possible to deal with out of malice? Or are you just using a provocative phrase because you're upset that it hasn't worked out the way you expected, and whether Apple was honestly trying to be helpful despite their rapidly diverging source tree or not you feel justified in taking your disappointment out on them?

Re:Its only the bad things we head about?

[Anonymous Coward]

Hang on now. Is sharing your source a big favor, or is it a moral imperative?

It seems like you open-source guys aren't all singing from the same hymnal.

No problem

[drhamad]

It seems to me that there's nothing wrong with how either Apple or OSS/KDE/KHTML have gone in this project. While there were high hopes that Apple's efforts would contribute significantly to khtml, of course Apple is going to taylor the project to their needs - not the needs of everybody. There's no gain for them in that. To expect otherwise is unrealistic. khtml can use these changes or not use these changes, that's its choice. If it decides not to go along with Apple (or if it can't), that's fine. Nobody should really complain about what either company/group has done.

Re:Its only the bad things we head about?

[argent]

The LGPL says that changes have to be returned in the "preferred form" for exactly this reason

OK, I'm trying to figure out a definition of "the preferred form" that means "compatible with a toolkit they're not even using". Because that's a huge problem... the changes Apple's made involve porting what started out as KHTML to a completely different API, and no matter how frequently Apple released the patches, or in what sized chunks they released the patches, they'd still be full of changes related to the fact that Apple's APIs have evolved along a completely separate path from X11 for their entire lives. Carbon's API can be traced all the way back to the original Macintosh, and Cocoa is based on the NeXT ObjectiveC + Adobe Display Postscript code... and there's only one X11 toolkit that either of those APIs have any relationship at all with, and it's neither of the Big Two.

The only way to keep that from being a problem would be to have both projects agree to a common API for the GUI, and stick to it.

Do you honestly see that happening? Especially since the best compromise technically would involve the KDE people switching to GNUstep.

Re:Its only the bad things we head about?

[Otter]

I went to the FSF site to pull the relevant text, but was distracted by Stallman's reference to "the US decision to blame Syria for the assassination of Hariri".

The guy is a brilliant programmer but, ughh, what a completely sociopathic asshole. He goes to a police state, runs into an edge of their censorship that blocks ssh and complains "that this prevents people from participating in world-wide free software development projects, and that it needed to change." Or this gem: "I don't know whether Syrian mass media are more controlled than the likes of CNN and Fox News."

You know -- I've lost any interest in arguing Apple versus KDE. It's inconsequential. You guys can take it from here...

Re:This sounds normal

[nutshell42]

The real problem is that Apple isn't willing to throw it in a plain brown box and send it via UPS ground to KDE, it isn't even willing to show it to the KDE developers when they knock on the door, they have to go to the haunted house, with the triple locks, broken lights and stairs and look for it in a closet in the basement with the sign "Beware of the Leopard" on the door. Oh and it's written in an ancient dialect of Swahili still spoken in some remote villages

All the KDE devs asked for was access to the logs of Apple's version control system; they even would have signed NDAs. But all Apple gives them are sporadical dumps of tens of MB of code which makes diffs completely useless. Reverse engineering the patches and applying them to the KHTML codebase is more work than writing a new patch from scratch and all the while there are the Apple fanboys bitching why KHTML doesn't have this patch or that patch even though the almighty Apple gave them the patch a zillion years ago.

Re:Its only the bad things we head about?

[Carewolf]

Right, and I've already merged those that applied. Now if only we got all patches divided up like that, rather than as the once a year code bomb.

Oh look, an INFORMED and EDUCATED post !

[DeadMilkman]

One of the best insights from the outside into this whole mess of Apple vs KHTL I thought was made by Gregory Block: I just wonder how realistic the KDE guys are being about this, in the back of my head. Qt is a cross-platform UI, sure - but it's no different than any other UI when it comes to the fact that it's embedded all the way through the application. Is this about Apple modifying KHTML in ways that embed it with Apple UI calls? Is it any different than the KHTML guys embedding Qt in it? Having seen Netscape's cross-platform troubles from the inside, I can see why my gut reaction is that "cross-UI + cross-platform = total mess", because Nav 4.0's cross-platform stuff was... complicated, to say the least. Littered with #IFDEFs and a complicated build system to make that work, where each team routinely broke the mac build, over and over again. The fact that Apple has gone so far to build a Qt-alike API in order to keep KHTML in the state that WebCore is in - which the KHTML players seem to feel isn't good enough; isn't close enough to the Qt version - is already a huge gift that, given that past experience at Netscape, I have difficulty seeing as a long-term solution. So what's the answer? Is Apple expected to build and maintain a full Qt emulation library, or switch to Qt entirely? Is that a reasonable expectation? If it is not a reasonable expectation, is KHTML willing to spend its resources abstracting away its dependency on Qt? And is *that* a reasonable expectation? Or is Apple meant to do the abstraction, as they're the ones "who require it", and submit that back to the KHTML guys and hope that the dev team finds it acceptable? And if they don't, what happens next? And will all parties be willing to live with the performance problems, if any, that come from inserting that abstraction? Will both sides be willing to live with the resulting limitations that come from the differences between the systems? Talking about how the two sides 'differ in politics' misses something fundamental - the codebases appear to have fundamentally differing strategies, and I can't help but feel that I'm not actually hearing proposed solutions from the KHTML side - just issues with what's being done. I can't see how it can be done radically differently without endangering both products, as an outsider. The choice is either to move towards a Mozilla-like platform abstraction, or end up with a Navigator 4 #ifdef-hack. Sure, maybe there's a middle road, but all I'm hearing from the KHTML team is that they can't run WebCore diffs against their tree and are a little on the cross side about it. What would a diffable WebCore look like, and would the KHTML developers be able to live with the product that inevitably created? KHTML will have to change to meet requirements from the outside, requirements that may well be unacceptable to the KHTML community - and vice-versa; what then? Especially after what happened to the last Frankenbrowser? Netscape 4 is dead. Netscape 6 (Java) is dead, with its shelf still in the box. Netscape 5, dead. Everything that was left of 4 that wasn't part of the low-level cross-platform or security toolkits used by every other product went the way of the dodo, along with every attempt to build a new Frankenbrowser, aside from Mozilla - and Mozilla is what it is not only because of the decisions it makes about its architecture, but because of its xplat requirements. And I know that Hyatt knows that, because he was the guy who had to fix the browser over, and over, and over, and over again, every time one of the Windows guys checked in a change to Nav that broke the Mac. If I were him, I'd be strapping a rocket on my back to get away from any hint of a mote of an idea of a glimmer of a thought to return to that kind of cesspit of a codebase, and I can't imagine the KHTML developers would want that life for themselves either. Something tells me that neither side will be willing to sacrifice the one thing that made them stick with KHTML in the first place - the fact that it didn't have all of the cruft that c

Re:Not a big deal.

[Anonymous Coward]

> Apple has been contributing code back to the KHTML team.

Incorrect. Apple has been releasing the source code of their own fork (as they are required to do). They did not "contribute back to the KHTML team". There is a difference.

Re:This sounds normal

[Coryoth]

I think this is pretty much the whole point: WebCore is a fork. There isn't much cooperation, or joint effort between WebCore and KHTML. Sure, some code manages to go both ways, but slowly but surely the two projects are diverging more and more. This isn't really a problem, the problem is expectations. The KDE people are pissed not so much at Apple, but rather at all the people that keep expecting changes in WebCore to come to KHTML. They are pissed with all the people that don't seem to understand that WebCore is a fork, and is now quite a separate project.

Go read some of the antiquated mailinglists for GNU/Emacs, I'm sure you'll find some equally pissed developers there complaining about people constantly asking when XEmacs changes will get merged into GNU/Emacs. That's all we're dealing with here.

Jedidiah.

Re:Its only the bad things we head about?

[aulendil]

As KHTML is licensed under the LGPL, anyone, including Apple, is given the rights to use the codebase. In exchange for using the code, one must release one's changes under the LGPL. Since Apple's Webcore is under the LGPL, Apple has fulfilled their responsibilities.

Re:Its only the bad things we head about?

[farnz]

AIUI, the big problem is that people see that Safari is built on KHTML, and assume that the WebCore codebase and the KHTML codebase are closely related; thus, when KHTML doesn't function as well as Safari, idiots go shouting at the devs, accusing them of being lazy for "not merging Apple's lovely changes quickly enough".

The KHTML devs would like Apple to either make it clear that WebCore and KHTML are now very different, despite the common ancestor, or to help merge things back. Either way, they want something to hit the idiots with that makes sense to a clueless type, rather than stick with the current "Apple says", "KHTML says", "But Apple says!" back and forth.

Re:Its only the bad things we head about?

[99BottlesOfBeerInMyF]

The KHTML team have (AFAIU) only limited access to the Webcore bugtracking system, with some bugs not visible at all. From the story: "[KHTML] suddenly found themselves dealing with bug reports Apple deemed too sensitive to share, new requirements for auditing code before releasing it, and demands that developers sign nondisclosure agreements before looking at some Apple code."

Most commercial companies have to deal with customer information privately and that means restricting access to versioning systems, bug databases, trouble tickets, and code comments. I know I work for a company that uses a lot of open source code in our products. That does not mean we can allow access to privileged information that inevitably finds its way into the aforementioned systems. WebCore is as open as any open source project, but that does not mean the internal workings of the company making it or their customers has to be open to public scrutiny. Apple has to do business in the real world with government agencies and large corporations.

You can't see changes as they're made, you can't track the development trunk. All you can do is take the occasional slabs of code that they release - and the whole point is that that makes it difficult to port interesting code changes into the KHTML codebase.

Even so, Apple has gone out of their way to try and make it easier, above and beyond what is required by law. Changes that are specific to Apple technologies or that effect Apple only interfaces are commented as such.

Added to which, the Safari/Webcore developers have entirely different (ie. commercial) priorities to the KHTML project.

Actually I think this is the major problem. Apple wants a flexible web library, easily accessible using their dev tools and that is usable for the general public. KDE wants a browser for technophiles. Apple needs something that runs on OS X using their own window manager, and runs quickly. KDE needs the same for KDE of course.

And the Apple developers have done another thing which makes things even more difficult for the KHTML devs to use their changes - they're using significant closed-source functionality from Mac OSX system libraries.

Gee, I can't imagine why they would have done that. KDE has used significant KDE only libraries. Both groups want them to run on their own system, that is not a surprise.

Wow, you really haven't read any of the background on this at all, have you? The KDE/KHTML team did ask (many, many times), and the Safari/Webcore team weren't willing to (or weren't allowed to) accomodate them.

That's funny I heard the acid test patches were broken up into small chunks and had comments specifically for the KDE team telling them what applied to what as far as the codebase was still the same. I also heard that the lead Apple developer was soliciting suggestions for how they could make things easier, in fact it is still up on his blog. They can't release the bug reports, nor the versioning system, but Apple is certainly trying to be a good neighbor on this one.

They would probably also be happy to answer questions and explain particular changes.
Weren't. :-)

Got any source to back that up? What particular change was asked about that Apple developers refused to answer?

They're not running an opensource project with Webcore, they're building a closed-source project (Safari) and making occasional monolithic code releases of the one major open-source component of that app (Webcore).

Bullshit. WebCore is an open source web engine and there are even several new open source browsers based upon it as well as at least one other proprietary one (Omniweb which I'm using now). There is nothing wrong with closed source applications using open source ones. It violates neither the law nor the spirit of open source.

Apple marketing trumpet Webcore as a good wholesome example of Apple contributing back to the opensource c

Re:Its only the bad things we head about?

[molnarcs]

You miss the point. The main complain of khtml developers is that clueless users think that once a feature is present in Safari, it would be easyt ot port it to konqi. Quote:

And you know what? Thats their right. They made a conscious decision about not working with KDE developers. All Im asking for is that all the clueless people stop talking about the cooperation between Safari/Konqueror developers and how great it is. Theres absolutely nothing great about it. In fact it doesnt exist. Maybe for Apple - at the very least for their marketing people. Clear?

Also, this is not just "a problem for KDE." One reason for the difficulties is that Apple has different sets of priorities than khtml devs. Apple wants feature X present by a deadline. KHTML devs place equal importance on keeping the code clean and optimized. As a result, the Safari code is not up to KDE's coding standards:

Actually the biggest problem right now is that Apple are not keeping up with code-cleanup. We constantly try to develop more elegant easier to maintain code, where as Apple wants the right features - right now. Safari is basically still KHTML from KDE 3.1 with a ton of bug fixes and features. Many of the features takes time to port because they do not live up to our coding standards.

I think this situation could have been avoided if Apple tried to cooperate with KDE from the very beginning - and kde guys did quite a lot (creating specific mailing lists, giving cvs access, etc) to help apple devs. What KDE guys were asking for would have benefitted everyone: code cleanup could have been easily integrated into safari (some kde devs even offered to sign an NDA's to help!) while features might have been integrated into khtml. This is clearly a win-win situation that Apple missed.

A note on zealotry (not directed to parent post ... it is a general complaint). 1) It is quite funny that when I was discussing this on osnews, a bunch of people jumped on my posts calling kde devs names (whiners, zealots, whatevers) and praising apple for their huge contribution to OSS. And no matter how hard I tried, I couldn't get them understand that the main problem of khtml developers is not that Apple didn't contribute back enough (although that is part of the problem). Their problem was that - the result of Apple's marketing campaign about being first class citizens of the OSS community - users thought that they don't implement features present in Safari because they are lazy or they just don't want to or whatever. In other words, their gripe was with clueless users.
2) Check the asnwers to Carewolf's post [slashdot.org]. Apology, apology, apology... like "users DO NOT CARE if your code is 'elegant' and 'easier to mantain', users WANT THINGS TO WORK whether or not they are 'elegant' or 'adequate'." (why is he [slashdot.org] shouting? - and most importantly, why is he modded insightful?). IMHO, this kind of APPLE can't do wrong does disservice to APPLE - one of the keys to do successful business is to recognize the mistakes one makes in order to avoid them in the future. You can love APPLE and be critical at the same time!

Re:Its only the bad things we head about?

[Pete]

Shit it's hard to handle appropriate levels of contextual quoting in slashdot-HTML - at least for this long a post *wry grin*.

Most commercial companies have to deal with customer information privately and that means restricting access to versioning systems, bug databases, trouble tickets, and code comments.

And that's fine - Apple is well within their rights to do that. But doing that makes it a hell of a lot harder to consider Webcore a genuine open source project.

WebCore is as open as any open source project, [...]

As open as a project like KDE, which has a completely open source control system, a wide array of open and searchable development mailing lists, and an open bugtracking system? No, I though not :-). See below.

Bullshit. WebCore is an open source web engine and there are even several new open source browsers based upon it [ ... ]

You misunderstand my point. I didn't say that Webcore wasn't under an open source license, nor did I suggest that Apple was violating the license in any way.

What I did say was that Apple wasn't running Webcore as an open-source project. A serious OS project should first and foremost have the source developed in an open and freely accessible source control system (though this is not an absolute - many less active OS projects can achieve much the same result with infrequent tarball releases, mainly because updates to the project are infrequent). Less important, but also good, are openly accessible development mailing lists (though again there's some degree of variation here - some projects have a closed "core team" list - but the vast majority keep all lists open). An open and freely accessible bugtracker is also a big plus (though again, there's a fair amount of scope for restricting access to important security-related bugs, as does the Mozilla project with Bugzilla).

Regarding the different priorities between Webcore/KHTML:

Actually I think this is the major problem.

Well, it certainly seems to be a big part of the technical problem :).

That's funny I heard the acid test patches were broken up into small chunks and had comments specifically for the KDE team telling them what applied to what as far as the codebase was still the same.

As far as I'd heard, that was a case of too little, too late [kdedevelopers.org] and I seem to recall another KDE dev suggesting that Hyatt only did it because he (the KDE dev) dared him to (as they'd been asking for lightweight atomic patches for ages with no response). I hope that's wrong and Hyatt is genuinely trying to help the KHTML project - it certainly looks as though he is, going by his blog.

Re: a comment I made about the Apple devs not being willing to answer question or explain changes:

Got any source to back that up?

Actually, no. Dammit. I thought I'd seen a couple of references on the KDE dev blogs regarding the Apple Webcore devs being "unresponsive" to requests for information, but I can't find those references now - I might even have misremembered them. I hereby withdraw that comment.

Well you certainly are opinionated, but I don't think you're as informed as you seem to imply. WebCore is a thriving open source project with numerous new browsers for both OS X and Linux based upon it.

You're dead right on the opinionated count :). And I'm sorry, I didn't mean to imply I was that well informed :).

My main objection to the "Webcore is a genuine open source project" is covered above - the lack of an openly accessible source control server is a bit of a deal-bre