How OS X Executes Applications

MacHore writes "0xFE has an excellent tutorial on Mach-O, which is the file format used by OS X executable files and libraries. It goes into great detail about how Mach-O works, and explains what OS X actually does when it loads and runs an application. Subtopics include Universal Binaries, The Dynamic Linker, Using otool, and other goodies."

How I execute applications :)

When FORCE QUIT doesn't work, 120V A/C to the processor does the trick every time.

How it launches?

All I do is click on the icon and it launches the application! It obviously doesn't do any of that fancy stuff the article mentions or the pixies would tell me!

Universal Binaries, The Dynamic Linker, Using otool? pah!

Mach-O

Mach-O? Further proof that Mac users feel emasculated and need to make up for it. : p

"Using otool"

Do you mean big "O" tool?

Kinda OT.. yet relevant to this thread

Great strides are made weekly on the usability of linux, package management, dependencies, New File Systems, More apps, Wine etc.. and there is no doubt in my mind that they all work towards making Linux easier to use and more accessible to Lambda Joe's.

There is one thing without which no Joe User will go, no matter how pretty or compatible linux becomes: installing Apps.

No matter if you are a Windws, Linux or other die-hard you have to admit Mac OS X Make's it damn easy to instll 99% of the apps. Drag and drop what looks like a single file (in reality a *.app folder) and clik to run.

I immediately knew this was huge when OS X came out and made this possible on *NIX machines, and was secretly hoping that Linux wold catch up with it's own version and take-off.

Unfortunately, we are still relying on the age-old install with dependencies, of-course Synaptic, apt and Yum all make that easier but still too complex for 80% of the people.

When will we get drag and drop app install for Linux?

Re:Kinda OT.. yet relevant to this thread

When I try and install something on OS X that doesn't have the required dependencies, it simply fails to work and gives no user-friendly clues why.

We're talking about applications here. On Mac OS X, a properly packaged application lives in a .app directory which contains all dependencies with the sole exception of frameworks that are present on any version of Mac OS X that the app is capable of running on. In other words, a properly packaged Mac OS X app has no external dependencies - everything it needs to run is either in the .app directory, or comes standard with Mac OS X.

Any application packaging that assumes that users will not relocate an application is broken on Mac OS X. This means that any having dependencies not contained inside the .app directory (aka the app bundle) is broken, since users can and will relocate apps to removable media for use on other machines.

Now, some misguided *nix hackers will cobble together an application to run on Mac OS X that scatters little *nix file turds all over various directories, or will hard code installation directories so the application is not relocatable. These are both wrong on Mac OS X (though common practice on *nix) because Mac users have been relocating applications since the mid 1980s, and will continue to do so.

Re:Kinda OT.. yet relevant to this thread

So - you install a version of the gimp - you get a copy of gtk, install eog - you get a copy of gtk, install.... well I think you get my point.

No, the post you were responding to phrased it poorly.

What is inherent in a MacOS X version (say, 10.4.5) already covers most of what an application might need, including Apple's "equivalent" of gtk. Go browse the Apple Developer docs sometime, and you'll see a much richer set of libraries that come with the OS by default. Thus, if an application wants an "obscure" library and decides to bundle it, the cost to the system is minimal even if another app has the same one. The MacOS bundle is not at all the equivalent of statically linking against gtk and other "fundamental" libraries in Linux.

On the Mac you have scour the web deciding Free or Paid, [...] Once you've downloaded it, things are pretty sweet, but finding the download can be a PITA.

Have you ever heard of versiontracker?

On the linux box (I am going to choose Debian as I'm familiar with it). Fire up synaptic from the gnome menu. Search for barcode. Two results returned. Both of these programs I know to be free of trojans, compatable with my system & configured for it. To install, I double click.

Which can bring in new versions of a library, which in turn brings in new versions of another application, which may be broken in some way (like any app can be). In other words, installing one app in Linux can effectively break another one. You pay for the isolation available to MacOS X apps in the form of disk space and RAM, both of which can be relatively cheap depending on what you're doing with your computer.

Re:Kinda OT.. yet relevant to this thread

Have you ever heard of versiontracker?

*puts small business owner hat on*

No.

So you're telling us that a "small business owner" doesn't know how to Google for an app and stumble onto VersionTracker (which he/she should suspect exists anyway as similar sites exist for Windows) but the same "small business owner" knows what a Terminal app is; knows how to drive a command line, and knows that he/she needs to use a command called "apt-get".

I don't believe you!

Yeah, let's talk about bloat...

How bloated must that be?

Seriously, how many copies of a library does that leave you having installed?

Generally? One.

Remember, this isn't a Linux distro, where the user has all the choices in the world except the one that matters most: the choice to have all of their apps look and behave in a consistent manner. 99.99999% of all apps targeting OS X link against the standard system Carbon or Cocoa frameworks. The odd app might have a really large third party library linked in and included in the app bundle, but that tends to be the exception rather than the rule, as the bundled libraries tend to be of the small, utility type that only run a few K. How bloated is it? Not very at all.

Now compare that with the situation on your average Linux distro: Instead of one version of one or two frameworks, every third app is written against a different toolkit. Want GIMP? Install GTK. KDevelop? Need QT too. Your text editor links against the athena toolkit, your system management utilities against Tk, your games against SDL, your audio editor against WxWidgets, your file manager against GNOME, something else against FLTK, FOX, Lesstif, Xaw...your shared libraries folder reads like an explosion at the acronym factory. Repeat ad naseum for your XML parsing libraries, your regex libraries, your sound libraries, etc, etc.

And it doesn't end there. Binary compatibility between library releases is the exception rather than the rule. Every third GTK app requires that a different point revision of the GTK libraries be installed. XMMS requires that you install libraries that are several major revisions out of date. You've got 6 different versions of three different XML parsing libraries installed, 2 regex engines, 3 copies of your JPEG, PNG, etc libraries, 3 or 4 different audio libraries with a couple different versions installed. Multiple versions of multiple libraries; an endless proliferation of crap that all does the same job with only the most miniscule of differences. All for the sake of an illusion of "choice" foisted on the end user by a fractious developer base with a raging case of Not Invented Here Syndrome and a belief that their convenience is more important than a consistent experience for the end user.

Now, how bloated must that be?

Ooh, I know this one!

With extreme prejudice?

"0xFE - 1111110b - 0376"

Am I the only one to notice that the supposed binary representation of 0xFE on the blogger's page is actually that of 0x7E (ie, it's missing a one)?

iVillage People

Mach-o mach-o man,
I want to be a mach-o man!

Decent article

I'm fairly new to OS X, and don't know much of the internals and I have no clue on developing via Xcode or a lick about Objective-C.

The otool command is new to me. Its handy because I too was getting "No such file or directory" with trying to use ldd. One thing this article does not mention is weather OS X calls mmap() on an executable and/or libraries? It probably does. I believe all modern OSes do this now, but I could be wrong.

If you like otool

there is a graphical version of otool that is perhaps a bit easier to use. It is called "TimmyOTool" and is located here [mac.com].

are osx binaries cross-architecture?

Does this executable format have anything to do with the binaries being able to execute on both ppc and intel machines?

I have a friend who just got a new intel mac laptop. There are a lot of programs which won't run correctly because they don't fully support the intel architecture. However, I know from my linux experience that if I compile something for ppc, and then something for x86, neither executable will even LOAD on the other architecture, let alone execute with any functionality. When I download things for this new mac, I never select anything that is intel specific, and programs like oggdrop and vlc seem to work just fine.

Is this some magic that the program is doing? Or is it on the system level? Or am I missing something obvious about macs?

Re:are osx binaries cross-architecture?

The answer is "sort of".

The architecture does support Universal Binaries, which have code for both PPC and intel processors. This isn't really anything new or magical, the OS is just smart enough to know which code to execute. The same thing existed in the classic MacOS to support both PPC and 680x0 systems with FAT binaries.

As with the classic MacOS, there's also an emulator involved, so the newer architecture can run (most) binaries compiled on the older architecture.

However, compile something just for the Intel processors, and it won't run on your PPC hardware. If I understand your question correctly, if things worked the way you were assuming, Mach-O itself would guarantee any Mach-O binary would run on either system.

Since OS X is UNIX based...

...isn't this information available when you type "man macho" at the command line?

Re:boo!

"I heard that Steve Jobs now is a Universal Humanary."

I think thats what Ubuntu is.. or kubuntu or edubuntu or something like that.

Re:I don't have a Mac

But i'm lazy! Can't you just RTFM to me?

Re:Obligitory...

I have a 1.67 GHz G4 with a gig of ram and I almost never have speed problems, everything is quite snappy enough (although I hear the new MacBook Pros would put my Mac to shame). More memory may help you if that is an option (I didn't follow Macs well enough back then to know what your max is).

That said, I seem to remember (and personal experience seems to bear this out) that OS X is really slow executing a fork() system call relative to Linux. I'm not sure if this is has to do with using a microkernel, or is because of specific decisions they made, but anything that fork()s a lot will run slower than on the same computer running Linux.

My only suggestion to you would be... are you running the Dashboard? As interesting as it is, I avoid it because it is a resource HOG like nothing else on the system. If you google around you can even find a way to completely disable it.

In general it is a very responsive system. The only time I can consistently cause a beachball is copying files (digital pictures) off of a CF card. This uses enough kernel time that the computer does seem to stop responding for tiny fractions of a second (and my CPU usage goes WAY up). I don't know what the computer is doing, but it has got to be able to do it faster. My theory? Hard coded uninterruptible sleep statements for timing purposes. Just a random guess though.

All that said, I regularly use a 2.3 GHz Windows laptop and I am constantly amazed at how long it seems to take some applications to launch, and how different it is. IE will come up almost instantly most times. But sometimes it takes 30 seconds to launch and show the page (then feels slow after that). Other apps can do the same thing sometimes. But most applications (IE is preloaded by the OS) take just as long as on my Mac, if not longer.

Re:Obligitory...

The icon bounces, bounces and bounces. I know my 800Mhz iBook with 640Megs RAM isn't the latest/fastest machine, but when I boot back into Ubuntu Dapper it's MUCH faster.

It's you and anyone else who has a 800Mhz PPC. (I've got a flowerpod iMac myself and it is a dog)

If I you happen to have a Intel or a 2ghz G5 then its not as much as a problem.

Even my PPC 1.5Ghz mini isn't as bad as the earlier systems.

Re:Obligitory...

A lot of that might be your applications. I use Thunderbird instead of Mail, and it's faster. OO.o beats NeoOffice, and Adium is faster than AIM.

Re:Obligitory...

What about kde, which does the exact same thing but tied to the mouse cursor?

"that it does in the background isn't helping me check my email, instant message or ssh to remote boxes."

Call me crazy, but I dont think everything linux does is helping you do those things either:

[4294669.546000] Checking for popad bug... OK.

POPAD BUG? WTF THAT IS NOT EMAIL/BROWSING/IM/SSH!!!! STOP WASTING MY MEGAHURTSZ!!!

Re:0xFE

Same reason 3com.com, 37signals.com, and 23.com exist: the RFC was a Request For Comments, and some people commented "no."

Re:0xFE

Bastards.

Re:I don't have a Mac

Oh dear, does Linux practice this awful act as well? I'm an opponent of capital punishment, so it's now clear to me that I can't, in all conscience, use a Mac or Linux. No application, however detestable a crime it may be accused of, should ever face what is no more than state-sanctioned murder!

My question to Slashdot, therefore, is what operating system should I choose?

I've heard good things about FreeBSD's jails, which are apparently very secure without being inhumane. But on the other hand, Windows also has some advantages - I understand it opposes the death penalty so strongly that that it's been known to commit suicide in protest when a user attempts to execute too many applications?

Re:Obligitory...

Huh. My old wallstreet (300Mhz w/ 384M & 10.3.9), while not speedy, doesn't seem to take forever... ofcourse I don't try iPhoto or anything that is really taxing... but then... it does work well for what I use it for: email, ssh, im, interarchy, firefox (looking up stuff while watching the tube)

To put stuff in perspective... it does feel comparable to my old PIII/500 box with KDE & debian.

But hey, ymmv.

Re:Obligitory...

I hadn't thought of that. My Mac is new enough to have accelerated Quartz Extreme graphics, so the bouncing is almost free for me (the graphics card is doing all the heavy lifting).

But for him, I'm guessing his iBook doesn't have a good enough graphics chip, so his CPU is doing all that compositing... which would really slow things down during an app launch.

Interesting point!

Re:What this means

Expect an increase of the number of Mac virii

And wormii! Don't forget wormii.

Re:What this means

Expect an increase of the number of Mac virii out there in a now to 3 month timeframe.

Yes, because this information was a closely held secret [apple.com] before this meddlesome blogger came along. I bet Steve Jobs is really pissed and is taking out a contract on his life.

Re:Obligitory...

I have an old 600MHz iBook with 640MB and it is slow to launch stuff. A 600MHz iMac is a much faster machine with the same amount of RAM but those are the compromises of a cheap portable machine. The 2.5" drive is slow too - it's not necessarily Mach's fault. You should have seen OSX 10.0 - launching without all the fancy prebinding that came later was really really slow even on 'fast' machines. I still hope for further optimisations on Mach but it's all about design compromises so I don't hold my breath.

Got a client's 2.1GHz Macbook Pro here and it's really quick to launch everything - Aperture just opens in one bounce and Safari is close to instant. The relative 'slowness' of mach will become irrelevant in the future for sure.

Re:0xFE

Offtopic, but can someone tell me how "0xfe.blogspot.com" is a valid domain?

According to RFC 1034:

<label> ::= <letter> [ [ <ldh-str> ] <let-dig> ]

Please read what is written a few lines above: "The following syntax will result in fewer problems with many applications that use domain names (e.g., mail, TELNET)."

I have some code to fix if "[e-mail]" could be a valid address...

Time to fix some code then ;-)

Re:0xFE

Actually, the full grammar in the RFC is

<domain> ::= <subdomain> | " "

<subdomain> ::= <label> | <subdomain> "." <label>

<label> ::= <letter> [ [ <ldh-str> ] <let-dig> ]

<ldh-str> ::= <let-dig-hyp> | <let-dig-hyp>

<ldh-str>

<let-dig-hyp> ::= <let-dig> | "-"

<let-dig> ::= <letter> | <digit>

<letter> ::= any one of the 52 alphabetic characters A through Z in upper case and a through z in lower case

<digit> ::= any one of the ten digits 0 through 9

So a hostname beginning with a digit is actually invalid by the RFC. Nobody actually uses that definition; it turns out it's fairly easy to distinguish a DNS name 127.0.0.com from an IP string 127.0.0.1 in other ways.

Re:Obligitory...

I take it that transferring 17MB folders is strictly off limits then?

Re:Obligitory...

Well duh, you're not supposed to start your applications more than once. Once started, you never shut down or log out. Just put your iBook to sleep (No! Don't euthanize it!)

-matthew

Re:Obligitory...

A lot of that has to do not only with Mach, but simply with the fact that up until recently, a lot of OSX applications (specifically, those built on Carbon) were not multithreaded. In other words, if one bit of the application hangs or times out, the application and everything that depends on it goes to hell.

This is why BeOS appeared to be lightning fast on even slow machines. Even the smallest tasks were executed independently, and bottlenecks were hardly noticiable.

Apple's doing a good job making everything work, and Cocoa is definitely a step in the right direction, but apple really needs to kill all of the single-threaded applications they've got now. The Finder is the most prolific and outrageous example of this, and anybody who's ever lost a network connection while a network share was mounted knows what I'm talking about (the system virtually hangs for 45 seconds until the connection times out. awful. simply awful)

Otherwise, I love OSX.

Re:Obligitory...

The Finder is the most prolific and outrageous example of this, and anybody who's ever lost a network connection while a network share was mounted knows what I'm talking about (the system virtually hangs for 45 seconds until the connection times out. awful. simply awful)

Hopefully you won't need to wait too much longer. As reported on Mac Rumors [macrumors.com], as of at least Jan. 26, Apple has been seeking a "Finder Software Engineer". Hopefully we can see a better Finder out in Leopard.

The job requirements were listed as

• Participate in all of the various stages of feature development from design brainstorms, through feature development, all the way to fixing that last critical elusive bug under a tight release deadline.
• You will be required to produce clear designs, excellent implementation and tight code.
• Deliver tight, well implemented features, fix bugs and develop Finder into the best file browser on the planet.
• Work on performance and responsiveness of the Finder, making it feel lightweight, fast, snappy and pleasant to use.

It is documented

At the bottom of the article, he gives a bunch of links to the technical documentation from Apple.

Mac OS X ABI Mach-O File Format Reference
Executing Mach-O Files
Overview of Dynamic Libraries

Re:How OSX Executes Applications...

Smoking in public is a no no in California. Medical Marijuana is acceptable though. Might as well go out grinning.

Re:Interesting, but why?

I didn't know it was a big secret. [apple.com]

Re:Interesting, but why?

But itsn't it really the vendor's responsibility to document the system and make it available to developers?

Certainly. That's why there are documents such as this one [apple.com].

Re:0xFE

>3com.com and 3m.com are technically invalid...

In old-school records management, 3Com is filed under "T", and 3M is filed under "M" (The name of the company is "Minnesota Mining and Minerals".

Re:Interesting, but why?

Did you read the article? He does give links to Apple documentation on the subject.

Re:Obligitory...

That doesn't sound quite normal; I have an 800MHz G4 iMac with a mere 512Mb running 10.4 and Word launches in 11 bounces (splash screen in 7)

Camino launches in 4 bounces (13 the first time after a reboot). Neither takes more than a couple of seconds.

It might be worth giving your Mac a good old house clean (delete caches, repair permissions, check the disks - all the usual stuff).

And no, your comment isn't flamebait, as far as I can see. ... Of course it could be you are trying to launch NeoOffice (the native Mac Open Office port) in which case, yes you can make a cup of tea in the time it takes to load.

Re:Obligitory...

am always frustrated with the spinning beach ball animation while I wait for my app to launch.

My laptop is hardly better than the system of the paent's author, but there are only two apps I experience unacceptable launch-waits for: one is a java game platform that the loads dozens more jar files, and the other is Adobe Acrobat Pro, which does something similar.

Actually, in both cases, the app starts and shows itself on the screen promptly, it just isn't ready to do anything for me for a great many seconds.

This is at 1GHz, 1GB, PPC ... and with the Dashboard running. But I must say, I have seen reduced VM trashing with java apps if I first send the Dashboard away by using Activity Monitor to quit (not force quit) the Dock. The Dock restarts right away, but the Dashboard widgets don't start until the next time I visit the Dashboard.

Re:I don't have a Mac

I don't have Linux. Instead, RTFM to learn how Mac OS X executes applications.