Understanding OS X Kernel Internals

jglidell writes "The OS X kernel has been in the news alot this past year, whether it's why its slow, Mach/micro-kernel makes it bad, it's going closed source and what not. Amit Singh has put up a new presentation on the innards of OS X. It does a pretty good job of summing up the OS X kernel architecture, and has some pretty detailed diagrams... for instance they show that there are so many process/threads layers in OS X. So if you are in the mood for doing some OS studying then head over."

Spelling

Grrr... "alot" isn't a word! It's "a lot."

The spelling police are on their way!

Re:Spelling

Not to mention fonts. I can never forgive anyone for using Comic Sans in a presentation.

Re:Spelling

"alot" isn't a word! It's "a lot."

I also get kinda pissed off because lotsa people write it that way.

huh?

whether it's why its slow

What the hell does that mean ? Editors drunk ?

Re:huh?

Nope, it's still gibberish.

whether it's why it's slow

Well... why is it slow?

Mach/micro-kernel makes it bad

Debating the pros and cons of Mach is a valid topic, but a phrase like this is so vague that it's meaningless.

it's going closed source

OK, that one's intelligible. But then we come across gems like this:

for instance they show that there are so many process/threads layers in OS X.

A small request for submitters: Take a minute to actually proof-read your summary. I'm not even talking about simple typos, or the correct use of "you're/your" - those look ugly, but most of the time people can still figure out what you meant. Just ask yourself: will these words make sense to a moderately intelligent English speaker who's not on a meth bender?

Needs more editor.

"alot"
"whether it's why its slow"
"they show that there are so many process/threads layers in OS X."

Do the editors even look at submissions any more? Or to put it another way, is our children learning yet?

Re:Needs more editor.

Do the editors even look at submissions any more?

I'm afraid they do. I think the problem is that they're not as skilled in writing english as they are in writing PERL. (That's not a slam, by the way. I suck at PERL.)

And before anyone goes on an "Off Topic" jag, it really does make a difference if the readers can understand what's being written. I stumbled over the "that there are so many" sentence a couple times trying to make sense of it. There are so many process threads layers in OS X that what? It slows it down? It's hard to program? Or is there simply a gee-whizz lot?

Yeah, I know, I'm off to R the FA. I just wish I had a better idea of what's in there.

Re:Needs more editor.

I think the problem is that they're not as skilled in writing english as they are in writing PERL.

I've seen slashcode.

I think the problem is that they're exactly as skilled in writing english as they are in writing PERL.

Where are good internal docs?

I have a small program that mmaps a bit of code and then points the program counter at it. Everything runs fun until a OS call happens. I've heard that Mach allows user land programs to install their own OS calls but I haven't seen any example code to do it and I suspect such a feature isn't in OS X. I've hunted through the source and I while I could write my own system call and compile it in, there should be an easier way. Can anyone point me in the correct direction?

Terrible summary.

I could barely get through the summary.

If English is a second language for the submitter, fine. But good grief, do you suppose one of the PAID editors could have done just a bit of work to make the summary more readable?

"OS X is slow" claims investigated

Before anyone starts spouting off again about Mac OS X being "slow by design" or somesuch, read this article by an Apple engineer [ridiculousfish.com] that investigates those claims.

Re:"OS X is slow" claims investigated

It shows how the author completely misses the criticism that he is trying to address and fails miserably in his arguments.

It does nothing of the kind. Read the whole article, and ALL of the comments, sparky, not just the ones that support your prejudice.

Peter's point is that the "benchmark" in question boils down to how quickly OSX will malloc and free 35KB blocks. It happens to cross the mmap() threshold for kernel versus heap allocations.

It also shows how Apple fanboys applaud the author's explaination, even though they haven't a clue about the subject matter - and even admit it in their posts.

If you had actually refuted anything that Peter had said in his article, you might have a leg to stand on, but since you didn't, and since you toss off an epithet like "fanboy" instead of doing so, I conclude that you are the clueless one.

No wonder you're an AC.

-jcr

A flash presentation

A flash presentation which is over 6 MB large (still loading)..
Good bye, dear server!

Ad

I looked at the "presentation" and no, it doesn't do a very good job of explaining anything. Maybe combined with an extensive lecture to explain what the hell he's talking about would make it a bit more clear. From what I saw it was basically just enumerating the different components. Then I noticed the second to last slide. It's basically an ad for a book coming out.

Maybe it's just me though. Did anyone else find it extremely enlightening?

Underpowered Little Machine

As the purchaser of a brand new Core Duo Mini, (my first Mac, I feel "as happy as a little Gurlll!") I noticed that my system out of the box with 512 of RAM was dog slow when you start loading iPhoto, or any more then 2 apps.

Initial startup yielded a smoking fast web browser, and other single line items.

I purchased the 2GB Ram upgrade (not from Apple at 600 USD, 280USD from Crucial) and I noticed such a difference, that I couldn't understand WHY they would even consider shipping that little silver wonder with less then 1GB of RAM.

It's not the kernel, it's the apps... They just don't give enough power to the off the shelf machines to support the great apps that come with it.

Vive le Mac... Thanks for putting excitement back into computing for me.

Re:Underpowered Little Machine

I think much of OS X's ram-hungriness comes from the fact that outside of the system frameworks, there is very little utilization of shared libraries among the different applications. Each app bundle is largely self-contained with its own shared libraries. Granted most apps that ship with OS X (from apple) just access the shared frameworks in /Library/Frameworks and have few other dependencies in their bundles. But start adding apps like MS Word, Firefox, OpenOffice, etc, and you'll start having multiple copies of various libraries loaded. The app bundle system is very simple and reliable, but because of the shared library issue, you'll always need more ram when running these apps on OS X than Windows or Linux.

Definitely 1 GB is a minimum amount of RAM needed for OS X Tiger these days. That is quite sad when you think about it, but RAM is cheap so I'm not too concerned about it. Apple has always shipped their machines short on RAM, hoping you'll pay ridiciulous amounts of money for their official RAM upgrades.

Closed?

Riiiight. Just because some idiot alarmists say that the kernel has gone closed when it simply just hasn't been released yet, the media and clueless bloggers start crying that it's gone closed source.
"Well... it hasn't. It's still open. IT JUST HASN'T BEEN RELEASED YET.
OSNews is reporting that Ernest Prabhakar, Apple's Open Source and Open Standards product manager, has stated in the Fed-Talk mailing that Apple has not actually closed Mac OS X's Darwin kernel for the Intel version of the OS; they simply haven't released it yet. Speculation about Apple closing the kernel arose from the fact that other non-kernel Darwin sources actually have been released, and the previous PowerPC-based kernel is still available as open source as well.Ernest wanted to make sure that tech media didn't confuse 'speculation' with 'fact'. A good lesson we all could benefit from...."

God damn alarmist idiots.

OSX: Highly Thread Sensitive

As an ex-NeXT developer, the historical speed bumps behind this architecture are directly related to code density. NeXTSEP ver 2.0 was nicely running on 25mHz. NeXTSTEP ver 3.0 suffered performance on 25mHz compared to ver 2.0 because of the additional kernel overhead resultant from code densification.

With every ver. release through 10.4.x MacOS X, mach/BSD layer exhibits funtional improvements with speed increases of the processor CPU and latent performance behaviors from the additional kernal overhead added by code complexity and densification.

Prima Facia evidence to the 4X speed improvement in performance from Apple's new Intel CPU bears witness to the limits of the kernal architecture.

The big mystery remains: why Mach?

I know that the original code, from NeXT, was Mach because there wasn't any choice. But in this day and age there are plenty of microkernels that actually perform well, even as a platform for full kernels. There's plenty of benchmarking for Linux on L4, and Linux on Exokernel. Both have minimal impact on performance. (L4 is usually 2-10%, Exokernel is similar.) Mach is a dinosaur. And since microkernels are small, with well-defined interfaces, porting to a more modern one should be relatively easy, even keeping backwards-compatibility of libraries. (You might need a mach emulator, or mach-on-top-of-l4 for a few mach-specific system calls, but that would hardly be a problem.)

So why hasn't apple done it?

Just a "glossy" overview

I read the Flash presentation. It's at such a gross high level and lacks any details other then the names of some modules. There is no new information here. OK there are lots of parts and it looks complicated but Linux has all those parts too (except for the bagage that is needed to support old Mac OS9 aps)

My opinion is that mac OSX is very well designed if you beleive that in the future there is be more and more "cores" inside most notebook and desktop computers. OSX is well setup to take advantage of 8-core machines. It should scale well.

Re:OS X Kernel - Why?

*snort*

Not bad! I think for extra points you should have bought up speculation that Apple closed the OS X kernel to ease the future transition to the Vista kernel.

And of course, no modern anti-apple troll should be complete without reference to Apple's 'betrayel' of PA semi [reghardware.co.uk]

Re:OS X Kernel - Why?

"OS virtualization is going to kill off the native OS X software market"

Ahh, and welcome back to another thrilling episode of "doesn't know what the fuck they're on about" theatre!

Honestly. OSV is SLOW compared to native. No one wants slow.

Re:OS X Kernel - Why?

Quoth the AC:
200 bucks for a Windows license? I don't think so. Are you so clueless that you think IT people drive down to the local computer store and buy a boxed copies of Windows???

You're assuming that the only market for software is medium to large enterprise. Autocad (which is made by a well known vendor) is used by tons and tons of architects. Since I've worked in IT support for a few architectural firms, I can tell you that most of them are comprised of small business of under 50 employees. I've used Autocad under Windows, however when I needed to use something to make some plans for a shed for my church, I used Sketch-Up, because there's an OSX native version.

Furthermore, most of the people that I've run across who are excited about virtualization are primarily interested in using it to run the existing Windows apps they own, and plan on replacing those with OSX native versions where available, and comparable OSX native apps where not available. The other major segment (where I fall into this equation)interested in virtualization, is the IT people who use/support multiple platforms.

Boot Camp is for games. Noone I have spoken with that knows about virtualization is seriously intersted in using it for anything other than games.

Saith the parent:
"If you have enough Mac customers that your Mac software is a profitable product, you are going to lose money from your decision. If you didn't have that many Mac customers, you shouldn't have been making a Mac product anyway."

Quoth the AC:
Woah!

Only someone who has never worked in a real life software dev house could make such a wildly stupid statement.

+ Savings from dumping Mac developers and overhead to support them
+ Savings from dumping Mac support and packaging
- Percentage of Mac running customers who stop buying your product due to canceled OS X version

Revenue goes down, but profits go up in almost every case outside of having 75-100 percent or so of your Mac customers stop buying your product. It is pretty funny to see Mac people freaking out about everyday business economics - "But that won't happen because I wouldn't like it to!" Heh.

What?
I'm not understanding something...

This seems like Carly Fiorina Logic. If we stop making HP calculators, people will just buy handheld HP units with similar funtionality. It doesn't matter that the calculator group provides a substantial net profit for the company. If we stop spending all that money on supporting calculators and simply use the computer support people we already have... Think of the savings!!!!

When a group/division in a company that caters to a market makes the company lots of money, leave them alone. They're doing their job, pulling a PHB stunt in wall street business tactics will only hurt your company.

Re:Profit!

somebody must have mispelled "book commercial" as "presentation".

there's nothing really new in that presentation, most of slashdotters know this stuff already, the only thing that we didn't know as of yet is that you can mispell "book commercial" in such an interesting way.

you can "troll" or "flamebait" my post, but this is the way that it is.

Re:Poetry is the reason for slowness

Oh, and by the way, the hum doesn't bother me whatsoever. 90% of the time its perfectly silent. Also, my old laptop, an HP ZD7000 P4 3.2ghz had the EXACT same problem.

Re:The average person

So go read a less technical website than Slashdot.
This story was boring because the presentation was dumb. It had a little useful information but not much. The subject is interesting.
OS/X security? How to lock it down? There are many sites on the web. Look for just about any good site about UNIX security and start from that. OS/X uses the UNIX security model.

Re:The average person

could care less

I think you mean "couldn't care less". I don't really blame you as this is a very common mistake people make, but seriously, think for a moment on what you're saying.

Re:Poetry is the reason for slowness

You're right, the composition of that poem took the entire development staff at Apple to complete.

We should all switch from Apple's "crappy kernel" to the wonderful Linux kernel, which is now so buggy even its major developers agree it needs a bug fix cycle.

Re:Poetry is the reason for slowness

That's Edgar Allan Poe [wikipedia.org], not "Allen."

Re:Comic indeed

um.. Comic Sans was made by Micros~1

Re:Why bother

Werent we told recently that we no longer have the right to make modifications and compile the apple kernel from source?

No.

Maybe it would be useful to look at it for learning purposes, but now-a-days I believe you can get into quite a bit of trouble if you look at source code thats not your own then go off to make your own project under a different licence.

"Now-a-days"? Try releasing a *BSD system under the GPL, or the Linux kernel+glibc under a BSD license, and see how much trouble you get into.

Whether looking at project X and using its ideas when writing a new project gets you into trouble is another matter. I'm not sure whether the GPL [gnu.org] or the modified BSD license [xfree86.org] or the APSL [apple.com] limits what you can do if you read code covered by that license or not.

Re:Possible interpretation.

Way off...

"The OS X kernel has been in the news alot this past year, whether it's why its slow, Mach/micro-kernel makes it bad, it's going closed source and what not. Amit Singh has put up a new presentation on the innards of OS X."

"The OS X kernel has been in the news alot this past year. Whether it is why it is slow (Mach/micro-kernel makes it bad) or it is going closed source and what not, Amit Sing has put up a new presentation on the innards of OS X."

The OS X kernel has been in the news a lot this past year; either a discussion of why it's slow, or why a mach/micro-kernel makes it bad, or because it's going closed source, etc. Amit Singh has put up a new presentation on the innards of OS X.

(You skipped over the 'a lot', messed up the meaning of a sentence with parentheses, changed a period to a comma that should have been a period, ruined someone's last name... you have no right to be a grammar nazi!)