Open File Locking and Mac OS X? 33
ArcticMyst asks: "In Mac OS X the responsibility of locking open files has moved from the operating system to individual applications. With the exception of Apple's most recent release of AppleWorks, I have not been able to find many applications written for Mac OS X that will lock a network resident file when it is opened. Not marking a file as locked allows more than one user to have a file open, then edit and save back to the original file. Even most of Apple's own applications fail to do this. Apple does provide information on how to make sure that open files are not edited while they are open. Why do so many applications fail to provide this security?"
Common to all Unixes (Score:5, Informative)
Lock the file before you open for write! (Score:5, Informative)
Re:Lock the file before you open for write! (Score:4, Informative)
Most people don't use the open() system call anyway. They use fopen(), or they use fstream in C++, or they do it in Carbon or Cocoa in MacOS X....
So no, i guess they do not lock their files...
D.
Re:Lock the file before you open for write! (Score:5, Insightful)
...and while classic MacOS locked a file when you opened it with FSpOpenDF(), the Unix-based OS X doesn't automatically handle it. So, I'd guess the answer to the original question is that Unix-background developers don't lock OS X files because they don't lock files and Mac developers don't because they never had to worry about it before.
Re:Lock the file before you open for write! (Score:5, Interesting)
Re:Lock the file before you open for write! (Score:3, Informative)
Using flock() - Advistory vrs Mandatory locking (Score:2)
You can use "Advisory Locking" (flock()) or "Mandatory Locking" (fcntl() - or lockf(), a front end to fcntl()).
The advantage of Mandatory locking is that you can choose to lock parts of a file (and leave other parts of it still open to be written to). It also gives you a _fully_ exclusive lock on a file, buy blocking read()/write() systems calls - this is both a blessing and a curse and should be used wisely.
Advisory locking, however, is much more efficent and is widely supported on all Unixes - unlike Mandatory locking which is not (BSD systems tend to support it out of the box, it's an option with Linux, but much more rarely used - vanilla Red Hat or Debian systems tend not to support it IME).
Mac OS X developers should note Apple strongly recommend only using Advisory locking and not Mandatory locking.
Sorry for shouting, but it can be a big deal
I would caution anyone NOT to use Mandatory locking unless they are sure they have a good reason for doing so as I've had problems porting small apps (even Perl scripts!) written by people who've used Mandatory locking when they shouldn't have, it's not difficult to fix, but it can be a pain to track down.
Due the previous behaviour of Mac OS's file open behaviour I can forgive old school Mac developers from making this mistake (many have a lot on their plate just moving software to Mac OS X) but for Unix developers I think it's pretty inexcusable to screw up completly (by using fcntl() in appropriately or by not checking for advisory locks with flock()).
Important point:
If you DO use Mandatory locking you should _always_ check for the existing of advisory locking with flock() *first!*. I think Apple have a man page or technote on this somewhere but it's a big deal if you've ever had your files clobbered by some oaf of a program that doesn't pay attention to Advisory locking
Re:Lock the file before you open for write! (Score:1, Interesting)
Um, have you ever had anyone edit a Korn shell script underneath you? I would guess not.
Isn't it obvious? (Score:5, Funny)
-Pete
Too easy... (Score:5, Informative)
From the provided link...
"Opening a file from classic Mac OS (pre Mac OS X) with fsWrPerm, fsRdWrPerm, or the default fsCurPerm, meant that any other application trying to open that same file with write access would not be able to do so. Usually an fsRdWrPerm error would be returned when other attempts were made to open the file for write access, though attempts to open such a file for read only access would succeed. This default behavior allows for one "writer" and multiple "readers" of the file. Mac OS X's BSD subsystem does not enforce file read/write privileges in the same way as classic Mac OS. Opening a file for writing does not ensure other processes can not write to the same file. The default behavior of BSD allows for multiple "writers" to a single file. In the current implementation, all of the File Manager calls in Mac OS X call through to the underlying BSD file system. As a result, opening a file via PBHOpenDF, PBHOpenRF, PBHOpen, PBOpenFork, FSOpenFork, HOpen, etc. on a local volume and passing in a permissions value of fsCurPerm, fsWrPerm, or fsRdWrPerm does not guarantee exclusive file access on Mac OS X."
Ergo, this is a significant and recent change of direction for Mac developers.
Also the average coder, in my experience, does not think about multi-user issues until problems roll in from the field. Finally, established test suites would not have been testing for this.
A point to bitch about sure, but to be expected if you read your submitted background link.
Bill
Big deal (Score:5, Funny)
Talking about reinvent the wheel...kids these days...
Depending on how you code... (Score:5, Interesting)
The other week I was doing disease modeling: I had written a Python program that would model the disease (using the 4th order Runge Kutta to numerically approximate the SIR model, if anyone cares) and then outputting it via the command line to a text file. I then had BBEdit open to look at the data and so I could bring it into pro Fit if the results looked interesting.
BBEdit is coded in such a way that it would automatically detect the change and update the text for me, making my life easier and preventing me from writing another script.
Not to say that this is universally good or that the above approach doesn't have flaws, but only that you *can* take advantage of something like this on a lot of levels--especially in a Un*x OS.
Re:Depending on how you code... (Score:5, Informative)
No, tail is for looking at the n last lines of a file, or to follow a file that has data being added to the end of it.
If you regenerate the entire file, or write to it via random access, the output from tail will be less than helpful. It will be of even less help if the writing process creates a brand-new inode (or whatever) on the underlying filesystem. Good text editors know you care about the filename, and not the actual file as it resides on the filesystem, and behave properly.
As a rule, however, most text editors will let you read any file that is open for write elsewhere. Well, the good ones do, anyway. I actually like when gvim reports "the file has changed, do you want to refresh it" as I know something has happened to the output I need to look at.
Re:Depending on how you code... (Score:5, Insightful)
The problem is multiple processes opening a file for write.
I think it's obvious why it's not used... (Score:5, Interesting)
Not necessarily a bad thing (Score:5, Interesting)
For example, you're claiming that apps should establish locks on network-resident files. That's fine and dandy, until some app goes to the great computer in the sky with a file locked.
On UNIX, traditionally users are savvy enough to wipe out lock files. In the case of netscape navigator, the app simply handed the user a command to run to remove the lock. Hence, write-locking files (especially remote ones) can cause nastiness for users as well.
Finally, I hate to say it, but if you're trying to set up a collaboration system (as it sounds like you are), some sort of dedicated checkout system is probably better. For text files on UNIX, this usually means cvs, but other platforms have their own favored revision control/collaboration systems.
In general, I dislike *intensely* the emphasis that many desktop operating systems place on forcible locking. Windows is by far the worst culprit -- write-opened files cannot be opened for write, cannot be moved, and cannot even be renamed. Classic MacOS is a bit better -- you can move around write-opened files -- but I really think that UNIX's approach is pretty good. You have the ability to use *both* types of locking, and the default is the one that lets the user do the most -- so dead apps don't tie up files.
Re:Not necessarily a bad thing (Score:3, Interesting)
For example, you're claiming that apps should establish locks on network-resident files. That's fine and dandy, until some app goes to the great computer in the sky with a file locked.
WTF? The operating system has no ability to handle this condition and expire locks from dead processes? What year is this?
Finally, I hate to say it, but if you're trying to set up a collaboration system (as it sounds like you are), some sort of dedicated checkout system is probably better. For text files on UNIX, this usually means cvs, but other platforms have their own favored revision control/collaboration systems.
While CVS has its uses, on the operating system level, this is not a solution. Running a CVS server on every system for single-user application work is not a solution, it's just ignoring the problem.
but I really think that UNIX's approach is pretty good. You have the ability to use *both* types of locking, and the default is the one that lets the user do the most -- so dead apps don't tie up files.
Encouraging users to haphazardly write to files and corrupt them does not count as "letting them do the most". Dead apps are no excuse for anything, see above.
The sky is falling! (Score:5, Interesting)
Applications that open files with calls to the standard Mac frameworks will use locking. A Carbon app that locks in OS 9 will still lock when running under a recent version of OS X.
Until this [major] oversight was fixed, workarounds were required in application code. This is no longer the case.
Only applications that use open() directly need to take extra steps to lock files. This is the way it's done in the Unix world, and Apple has chosen the correct implementation by not forcing the standard open() call to lock.
Mark
Re:The sky is falling! (Score:2, Informative)
I have been working with a senior Apple Engineer on the Xserve project to get Apple to understand that in a multi-user / multi-platform environment, this is a dangerous feature.
..file a bug report? (Score:3, Insightful)
Re:..file a bug report? (Score:5, Insightful)
The concept of a bug is the most overused in the software world today. Software not acting how you want it to does not mean that its buggy. Software not working how the author intended it to, that's buggy.
Re:..file a bug report? (Score:5, Informative)
It's even got the same number of letters as "bug." Handy, no?
Still an issue (Score:1)
I have been working with a senior Apple Engineer on the Xserve project to get Apple to understand that in a multi-user / multi-platform environment, this is a dangerous feature.
UNIX Advisory File Locking Implications on c-clien (Score:1)
This isn't the only behavior which has changed (Score:2)
To add to the stupidity you can still move or rename a hist directory or non-root volume thus changing the file's path. What a pain in the ass! The ability to lock a file's contents used to be a useful feature Apple!