Does launchd Beat cron? 798
Blamemyparents writes "For those who aren't Appleheads, you may not have heard that with Tiger, Apple swapped out old Unix standby cron for their own creation, launchd (Apple mentions it on their OS X page and has the man page for it up as well). Seems like it's a bit nerdy, but this changes a LOT about how *nix systems have done things for many years. Launchd is Apple's replacement for quite a few utilities, including launching and quitting quite a few different things, and getting info from the system and other running processes. This page from Ars Technica talks a bit more in depth about it. Apple has open sourced the thing, and is apparently hoping all the unix kids will take a look."
Submitter is confused (Score:3, Informative)
Based on what I've read about it, it's more a replacement for xinetd and the sysv init scripts than cron.
B
Not a cron replacement, a init replacement (Score:5, Informative)
This is somewhat understandable for something like OS X. Doing something simple like displaying a GUI detailing startup is terribly difficult with
I'm not sure launchd is something you'd want in 99% of Linux installs but if you're looking for a end-to-end user-oriented desktop I can see how a technology like this is necessary.
I'm not sure Apple Gets It though. Why in the world would they use XML configs? Gesh.
Makes booting DAMN fast (Score:5, Informative)
From the apple page: (Score:3, Informative)
Also check out Solaris SMF (Score:5, Informative)
Re:Submitter is confused (Score:5, Informative)
Basically any task the needs to run on its own, rather than being manually started by a human being, can be handled by launchd.
Re:Er, what? (Score:3, Informative)
from reading the article, launchd sounds like a viable technology to replace quite a few convoluted system on my linux machines.
Re:Not a cron replacement, a init replacement (Score:4, Informative)
The fundamental problem launchd was created to solve is system startup time. Twenty or thirty seconds to boot a freakin' laptop is just absurd. So we used launchd to get that down to four.
But beyond that, launchd also takes care of the problem of having six different types of configuration files, none of which are self-documenting or self-validating. A launchd configuration file can be validated and, if invalid (like somebody inserted a typographical error, or the file got truncated or something), discarded without interfering with system startup. That makes the system more resilient.
Re:From the apple page: (Score:5, Informative)
Before you jump the gun:
So yes, it is a cron replacement.
Re:Wait, wait, wait... (Score:5, Informative)
To keep things simple, we divide the boot process into two halves: the hardware half and the software half. The hardware half involves power-on self-tests, and the time it takes varies wildly from computer type to computer type, or even from configuration to configuration. It takes longer to test 8 GB of RAM than 512 MB of RAM.
The software half begins with the grey Apple-logo screen turns blue and the progress window appears.
From the time that happens to the time the menu bar appears (via autologin) should be four seconds on a stock Mac OS X install with middle-of-the-road hardware (dual 1 GHz G4).
Re:Is it really usable by the Open Source Communit (Score:3, Informative)
I know making the statement, "boo patents!" on
Re:Did anyone else initially think (Score:4, Informative)
Re:Ahh yes. (Score:2, Informative)
xml is more robust than its property file counterpart.
Re:Ahh yes. (Score:3, Informative)
But don't worry. I've once used a mmapped structure to make sure the config gets preserved over crashed -- but fortunately, someone beat me back into the senses. Try to exceed _this_ level of brain-damage
SYSV.. bah. BSD-style is the way to go. (Score:5, Informative)
Speaking of which, back in about 1995, I think it was, I installed my first Linux distro that I actually used on a daily basis. It was an early version of SuSE linux. Anyway, as I scratched deeper and deeper under the surface, I came to the conclusion that the init system was a mess... the entire system's functionality was implemented as piles upon piles of shell scripts, organized neatly into a whole bunch of directories, and activated according to their name and whatnot. This seemed kind of dumb, actually, because it meant that startup, shutdown, and switching runmodes were a lot less efficient than necessary, and also because you'd have to search through a zillion (that's a big number) scripts to find what you need.
A few years later, in 1999 or so, I tried FreeBSD for the first time. (Here it comes, I'm gonna be modded troll for starting another BSD- vs. SYSV-style init script war...) There, the functionality is still implemented in scripts, but there was a much more sane system. There is one script that is the same across all FreeBSD installs of that version. Then, there's the script you get to customize if you want. There's one more file, and that's where you simply give yes or no answers, or provide other data, in the form of environment variables that influence the running of those two scripts. It's so simple, and works quite quickly. Also, since you really only mess with one file, and two if you modify the script, it's much easier to find where things are. It's more efficient, from many standpoints.
So I don't blame Apple for getting rid of that SYSV stuff. It might have been cool back in the day, but it has lost its luster.
*sigh* (Score:5, Informative)
launchd is neat. It's not simply a different way of doing the same things, it lets you do different things.
Like automatically evaluating dependencies between daemons, starting them in the right order, and running them in parallel if needed. FreeBSD's the only other OSS system I've ever seen do this; Gentoo does the dependencies but not the parallel startup. (Which is annoying while it's, say, trying to get an address from a nonexistent DHCP server.) Long story short, it dramatically reduces boot time, while eliminating dependency hacks like runlevels and numbered scripts. (Not that BSD had them.)
For those of you who posted without reading the manpage (or administering an OSX system), it also lets you do init-style startup tasks on a per-user basis. You can configure it to start daemons and other processes on the behalf of users as the log in, and shut them down -- gracefully, not by TERM; KILL -- when the user logs out.
Anyone who's ever dealt with the myriad of hacks to get ssh-agent in place will understand why this is good.
There's a lot of resentment these days toward anyone who does something that's perceived as "not the UNIX way." Change is the only way to innovation, people; perhaps the UNIX way is broader than you thought?
Re:Not a cron replacement, a init replacement (Score:5, Informative)
The launchd service isn't a daemon. It's a replacement for init. And yes, it speeds things up considerably. Previously, every startup script required its own instance of a shell to be fired off. Now that middleman is removed. And it made things considerably faster and simpler.
But why XML over say something like INI or a much simplier format?
Because XML is self-validating, like I said. The contents of the file can be checked for consistency before being used for anything.
It also quickly becomes unreadable.
You don know that we're talking about property lists here, right? It's an XML-based format, but it's not totally free-form XML. You can't just plug in any valid XML and expect launchd to handle it. If the file doesn't validate against the property list schema, launchd will just ignore it.
Besides, we've been using property lists everywhere for five years now. Nobody's complained yet.
Re:Wait, wait, wait... (Score:4, Informative)
You don't want them to. One of the benefits of the Dashboard layer is the fact that when the Dashboard isn't visible, the widgets are completely suspended, consuming no CPU cycles at all. If you put a widget on the screen outside the Dasboard layer --which of course you can do, if you run Dashboard in debug mode --it'll execute its run loop all the time, sucking up CPU cycles unnecessarily.
Re:*sigh* (Score:5, Informative)
Modify your
# Set to "yes" if you want the rc system to try and start services
# in parallel for slight speed improvement.
RC_PARALLEL_STARTUP="yes"
How to put widgets directly on your desktop (Score:4, Informative)
This lets you keep as many widgets as you want on your desktop, and they don't jump back into the Dashboard every time you display it.
The one remaining annoyance is that they float above all of your other windows. Hopefully there will be a hack for this soon.
Re:From the apple page: (Score:5, Informative)
See also (Score:4, Informative)
Brave Deian owners can apt-get install runit-run which will start your system with
And the system that's had such a sensible service supervision system since the year dot? Windows NT and its service control manager. Of course you could argue a centrally-controlled daemon restarter is much more of a priority when you expect your daemons to crash more
cron still there (Score:5, Informative)
Are there schedulers better then cron? You bet. As a scheduler for home use, cron is fine. We are currently trying to find a ore robust replacement for cron that will attempt to correct things before reattempting a script and with dependency checking. When I used to be a operator on a DOS/VSE based mainframe, our scheduler had a dependcy check that would hold jobs from running at thier assigned time if the previous job did not finish and other things like that. It had very robust logging to. No e-mails to root. I could just run a report. There are schedulers that come close to this on UNIX but cron is a very basic scheduler that is flexible which is a saving grace.
Not just MacOS X: Solaris has similar concept (Score:5, Informative)
SMF also does dependancy checks and auto matic restart on failure (or some other conditions). It also uses XML for its configuration but imports that into an SQlite database so that it doesn't need to reparse the XML on every service restart or system boot.
For more information on SMF in Solaris see the links in the main architect/developers blog [sun.com].
Re:I did RTFA, now tell me WTF it does (Score:3, Informative)
Re:ugh (Score:5, Informative)
The launchd configuration files are property lists, which are serialized Core Foundation data structures. They consist of key-value pairs. They map directly to data structures in memory. The code for parsing them is used all over the place in Mac OS X, and is very old. It dates all the way back to the first development in the late 1990s. It's highly optimized, and it's reused over and over and over and over again.
The second advantage to plists is that they're self-validating. When a program tries to load a plist, if the file doesn't validate against the PropertyList-1.0 DTD, nothing happens. An error is returned, and the program (in this case, launchd) moves on. No chance of a corrupt file producing unexpected results.
The third advantage is that we ship a handy property-list editor with our operating system. This makes it easy for developers to create and modify plists.
Finally, plists are always in UTF-8 format. That's vitally important for us, because our system is fully localized. We have to be able to launch services with names that don't fit into the ASCII/Latin-1 character set. Using plists ensures that we can do that, and gives us that capability for free.
The question here is, why would anybody want to invent their own proprietary ASCII-only, non-validating file format for something as important as system startup control?
Re:Apple's plist format (Score:4, Informative)
The CoreFoundation interface to MacOSX's built in XML parser is described here [apple.com].
launchd replaces cron? (Score:3, Informative)
Re:how can it automatically know dependencies? (Score:5, Informative)
Re:Submitter is confused - Mod parent offtopic. (Score:5, Informative)
Re:Open Source? Where's the source? (Score:5, Informative)
Re:Submitter is confused (Score:3, Informative)
Re:Not a cron replacement, a init replacement (Score:3, Informative)
They did something similar with an even more important technology: file type metadata. Read the very good overview in the referenced Ars Technica article:
http://arstechnica.com/reviews/os/macosx-10.4.ars
Re:Not a cron replacement, a init replacement (Score:5, Informative)
Most people talking about launchd in the context of parallel system startup are actually talking about SystemStarter. Also note that the plist files which drive SystemStarter can be in one of two forms. The most common of these is the NeXT style textual plist format which is very human friendly and is not XML-like in structure or appearance.
As a replacement for aither BSD style or System 5 style rc scripts, SystemStarter is a definite improvement, both in terms of ease of use, speed, and reliability. The launchd subsystem uses XML formatted property lists. I am unsure whether NeXT style plists are supported for these.
Regardless, you were dismissive of launchd claiming that it was mere init replacement. You thought you were speaking about launchd but sound like you had only heard about SystemStarter.
launchd itself it a super-server which reads process definitions. These definitions can describe processes to be run using a set of predefined keys. Some examples of key value pairs are: Program/ (path to executable), or ProgramArguments/ (The array of strings to pass as arguments to the program). The interesting thing about launchd is that the predefined key/value space is so rich, and so well considered.
The keys span several related domains which Unix users associate with a number of different tools and data sources. Here are a list of some interesting areas covered.
like sudo or vixie cron the user and group under which to launch the program can be specified using any of { UserName, UID, GroupName, GID }
WorkingDirectory causes a chdir(2) before executing the job.
RootDirectory specieis a chroot(2) for the child process.
StdOutPath and StdErrPath can specify where to redirect output of the program.
OnDemand specifies whether to run only as needed or ensure that a server process should always be available, thus respawned.
Labels and Groups provide a way to refer to processes with greater specificity or to group related processes together for reporting or action using launchctl.
A set of time specific keywords cover a superset of at and cron time specification.
Soft and hard resource limits for the job can be tuned using keys that provide acces to all ulimit functionality (file descriptor file size, memory footprint, core size, etc.. Similarly the nice value can be specified to constrain CPU usage, and the *PriorityIO keys specify what priority the kernel should use for scheduling disk IO for this task.
The Socket Key defines an optional dictionary defining IPC behaviors controlling combinations of open/bind/connect/listen. (think inetd and xinetd). Beyond this you can also specify other things here like "Bonjour" to register this service with mDNSResponder. If a SockPathName is specified attempts to open a named unix domain socket will cause this program to launch.
If these are not enough, a launchd plist can start/stop a program when a watched file changes. When passed the path to a directory, the program can be started when the program is not running and the directory is not empty. Think about this one... A robust queue processing program might be simply:
#!/bin/sh
lockfile $SOMELOCKFILE
for file in *
do
process "$file"
done
WorkingDirectory and a watched directory path, along with whatever CPU/Disk IO/Memory/user/group are all handled by launchd. Similarly launchd ensures that the process runs when there is work to do, and is never launched unless there is work to do.
Finally, if the predefined keys and actions are insufficient you can call a small number of stubs implemented in Tcl to set either one shot timers or interval timers at which point the scripts is re-interpreted. You can pass messages to syslogd, start or stop jobs. A form of system introspection is also pr
Re:Submitter is confused (Score:3, Informative)
Re:Not a cron replacement, a init replacement (Score:4, Informative)
1a. It doesn't matter, because we're not using whatever valid XML you decide to poop out. We're using Property List 1.0, which is a specific XML document type.
2. We're not forcing anybody to do anything. We're offering third parties a chance to use our innovation. If they don't want it, they don't have to take it.
2a. Nobody would ever use an XML parser to read a plist anyway. You just use the serialization/unserialization routines we've written into Core Foundation, which is also part of Darwin and therefore also open source. You turn a plist into a CFDictionary in about three lines of standard C code, counting error-checking.
Re:cron still there (Score:1, Informative)
Re:Not a cron replacement, a init replacement (Score:5, Informative)
In fact, there is. With Core Foundation, the process of unserializing a property list is atomic. The bytes are read from disk and validated. If and only if the file validates, the data structure is populated and returned by reference, all in one function call.
So yeah, property lists are self-validating.
You have to be DAMN sure that the file is easy to understand and edit by an admin, even if he only has "ed" over a serial connection available.
You mean like the universal, thoroughly documented and self-validating property list format, for example?
I'd much rather work with: TAG:VALUE
Great
Or, for that matter, where "VALUE" needs to be of a specific type. If you're stuck with "TAG:VALUE" then you're kinda screwed when it comes to explicit type-checking, aren't you?
I can confidently write a program to validate and parse inittabs, but I don't know how to that with XML
You aren't expected to. Core Foundation does it for you. There's one function call to serialize a data structure and one function call to un-serialize it.
XML files, are difficult to read, difficult to edit, difficult to edit with programs, and poorly-defined.
Come on, now. Isn't that kind of a stupid statement? XML files are spectacularly easy to read, especially compared to the random mish-mash of configuration-file formats available on Unix. Editing is trivial, of course, because it's just text. More importantly, editing is trivial wherever you are, because the files are UTF-8 and not just ASCII. And calling a document that includes a machine-readable DTD for automatic validation inside it "poorly defined" borders on the farcical.
Some of us have been complaining since the NeXT days.
That's funny. Because there were no XML property lists in NEXTSTEP. They were added for Mac OS X.
Re:*sigh* (Score:1, Informative)
This thread, so far, seems to be filled mostly with "Ah! Someone's doing something differently than UNIX has for 30 years! My knee is jerking!" So I feel I should respond.
More like: this is tricky stuff, Apple better know what they are doing. Since many have tried and failed before, it's best to be skeptical.
Like automatically evaluating dependencies between daemons, starting them in the right order, and running them in parallel if needed.
I've been doing the same with Makefiles + daemontools. It ain't sexy but the best solutions never are..
it also lets you do init-style startup tasks on a per-user basis.
Daemontools + setuidgid + a couple directory-scanning scripts.. actually all my daemons run as some unprivileged user already.. running as actual login users is just a trivial variation.
gracefully, not by TERM; KILL -- when the user logs out
Is there something wrong with using TERM to end a daemon? How does launchd do it "gracefully"???
Anyone who's ever dealt with the myriad of hacks to get ssh-agent in place will understand why this is good.
?? Why what is good?
There's a lot of resentment these days toward anyone who does something that's perceived as "not the UNIX way." Change is the only way to innovation, people; perhaps the UNIX way is broader than you thought?
Uhm, see the first point.
Creating a single robust launch system is a great idea, I'm not arguing that. That's why I've pretty much switched to daemontools for almost everything except process 1 (and some adventurous people do that, actually).
It's just that with daemontools' features (no config files, atomic operations, ultra-reliable in any situation), I'm going to "wait and see"... I've been a Mac user since 1984 (yes, I'm old) and am using a powerbook right at this moment.. but I like the tried and true solutions when it comes to *servers*.
Re:ugh (Score:5, Informative)
Well
The format we're talking about here is what we call a "property list," a serialized data structure that's written out to disk in XML format. Here's an example. This is the launchd config file for KernelEventAgent, an important Mac OS X system service.This files are generated by the serialization functions in Core Foundation. You create a data structure, in this case a CFDictionary. Then you populate it with objects. In this example, that's four key-value pairs. The keys are "Label," "ProgramArguments," "OnDemand" and "ServiceIPC." The values are a CFString, a CFArray of CFStrings (with just one object in this case), and two CFBooleans. Then you issue one function call to serialize that property list to XML and another to write it to disk as a file.
Of course, in this particular case you wouldn't go through that trouble. But the important part is the other part of the process. You make one function call, passing it a path to a file, and you get back a de-serialized Core Foundation data structure, all type-checked and everything. If at any point the property list fails --if it fails in the XML validation or the static type checking --then you get back null and an error. All atomic, no muss, no fuss.
That's what launchd does. All clear now?
And shipping one of those implies there's difficulty in editing or reading the configs manually.
Look at it and you tell me. How daunted are you?
If you're building better solutions, maybe you can architect them to work well for most unixes?
We're releasing launchd, we're releasing Core Foundation, the property list format is both wide-ass open and fully documented in many different places. What else do you want?
Re:XML plist configs are easy (Score:3, Informative)
Of course you can. A launchd configuration property list de-serializes to a CFDictionary. If the CFDictionary doesn't include the required key-value pairs, it's not a valid launchd configuration file.
Syntactic and semantic structure happen in two different places. One happens as the file is de-serialized in Core Foundation. The other happens inside the launchd code itself.
DTD/Schema sensitive editors still wont help you out with semantic structure.
Technically this is a valid point, I agree. But property lists are almost always so incredibly simple that this objection is purely academic.
Re:Ahh yes. (Score:3, Informative)
Not true. By text files, we tend to mean line-oriented files. Often to be useful, it must also be field-delimited files.. colon-delimited in password files, or worse with cron-files a mixture of space-delimiting and position-dependence. (after first sevearl spaces, remaining spaces are no longer delimiters).
With XML you are no longer rectangular in your data-structure; no longer rows and columns (and potentially sub-colums), but instead you can more naturally represent the data however complexly it needs, and not have to worry about escaping arbitrarily complex characters, because all XML files use the same escaping.
Re:Submitter is confused (Score:3, Informative)
With launchd, if the service's config file is invalid for any reason, that service is simply ignored.
Re:launchd summary (Score:2, Informative)
From an educational perspective, I'd be interested to know which Unix design ethics it violates, though.