Invisible files associated with OS X keychains

Question

It seems that a keychain file (with extension .keychain) will usually have an invisible file associated with it, located in the same directory.

This invisible file always has these properties:

• It is empty (zero bytes).
• Its permissions are 0444 (read-only for all users).

• Its name consists of .fl followed by 8 hex characters. For example:

  .fl043D1EDD
  .fl947E1BDB
  .fl9FAF0136
  .fl12663120
  .fl8E6EFC6C
  .flCF600F4B
  .fl1D8ECE61
  .fl0D1D1BA9
  .fl79E88CD1
  .fl62323D2F
  .fl75262C83
  .fl652F188E
  

The invisible file can be deleted, but when the keychain's contents are next modified, the invisible file will be recreated with the same name.

Here are some steps to demonstrate, using the Keychain Access utility:

1. Create a new keychain, by selecting File > New Keychain and choosing a directory in which to create it. An invisible file will be created in the same directory as the new keychain.
2. Delete the invisible file (using the Finder or Terminal).
3. Modify the keychain's contents. For example, add a secure note to the keychain, by selecting File > New Secure Note Item. The invisible file will be recreated with the same name.
4. Delete the keychain, by selecting File > Delete Keychain > Delete References & Files. The invisible file will be deleted too.

I've verified that this occurs in OS X 10.8 and 10.9.

---

Update

The same invisible files are created when manipulating keychains using Apple's security tool in the Terminal:

1. Create a new keychain. An invisible file is also created.

   $ cd ~/Desktop/
   $ ls -1a
   .
   ..
   $ /usr/bin/security create-keychain ~/Desktop/Test.keychain
   $ ls -1a
   .
   ..
   .fl1BCE4B9A
   Test.keychain
   

2. Delete the invisible file.

   $ rm .fl1BCE4B9A
   $ ls -1a
   .
   ..
   Test.keychain
   

3. Modify the keychain's contents (eg: by adding an internet password). The invisible file is recreated with the same name.

   $ /usr/bin/security add-internet-password -a account -s google.com -w password ~/Desktop/Test.keychain
   $ ls -1a
   .
   ..
   .fl1BCE4B9A
   Test.keychain
   

4. Delete the keychain. The invisible file is deleted too.

   $ /usr/bin/security delete-keychain ~/Desktop/Test.keychain
   $ ls -1a
   .
   ..
   

---

Questions

1. Why are these invisible files created? What is their purpose?
2. What does fl mean in the filename?
3. What are the 8 hex characters in the filename? Some kind of unique ID or hash identifying the keychain?
4. Is there a way to prevent these files from being created when keychains are created or modified?

Answer 1

After a lot of investigating, I've managed to answer most of my questions:

1. The invisible file implements a write lock for the keychain's database.
2. .fl is the filename prefix for lock files created by the AtomicFile class in the Security framework.
3. The 8 hex characters in the filename are the beginning of the SHA-1 hash of the keychain's filename. For example, if the keychain file is called Test.keychain, then the SHA-1 hash of its filename begins with 1BCE4B9A... and so the lock file will be called .fl1BCE4B9A.
4. I haven't discovered a way to prevent the lock file from being created when a keychain is created or modified. I think it's probably impossible, but I'd be very interested if someone can figure out a way to do it.

---

Here are the details of my investigation:

Keychain's locked/unlocked status

I noticed that the invisible file is not affected by the keychain's locked/unlocked status. If the invisible file has been deleted, then locking/unlocking the keychain does not recreate the invisible file.

System calls

I did some investigating using the File Activity template in Apple's Instruments tool.

These system calls are responsible for manipulating the invisible file:

• Creating the invisible file when a new keychain is created:
  • Security::AtomicFile::create(unsigned short)
  • Security::RefPointer<Security::AtomicLockedFile>::release_internal()
  • Relevant part of the call tree
• Recreating the invisible file when a keychain's contents are modified:
  • Security::AtomicFile::write()
  • Security::RefPointer<Security::AtomicLockedFile>::release_internal()
  • Relevant part of the call tree
• Deleting the invisible file when a keychain is deleted:
  • Security::AtomicFile::performDelete()
  • Relevant part of the call tree

C++ files

These are the relevant files and classes (source code available from Apple Open Source for OS X 10.9.2):

• AtomicFile.cpp
  • Security::AtomicFile
  • Security::AtomicLockedFile
  • Security::AtomicTempFile
  • Security::LocalFileLocker
• AppleDatabase.cpp
  • Security::AppleDatabase
  • Security::DbModifier

Comments in source code

The comments in those files provided some clues:

• AtomicFile::AtomicFile()
  • "compute the name of the lock file for this file"
• AtomicFile::create()
  • "Lock the file for writing and return a newly created AtomicTempFile."
  • "Now that we have created the lock and the new db file create a tempfile object."
• LocalFileLocker::lock()
  • "if the lock file doesn't exist, create it"
  • "try to get exclusive access to the file"
  • "check and see if the file we have access to still exists. If not, another file shared our file lock due to a hash collision and has thrown our lock away -- that, or a user blew the lock file away himself."
• DbModifier::modifyDatabase()
  • "Now we are holding the write lock"
• AtomicFile::write()
  • "Lock the database file for writing and return a newly created AtomicTempFile."
• AtomicFile::performDelete()
  • "Aquire the write lock and remove the file."
  • "unlink our lock file"

Generation of lock file's name

I found this code in the AtomicFile constructor:

char buffer[256];
sprintf(buffer, "%08X", hash);
mLockFilePath = mDir + ".fl" + buffer;

where hash is the first 4 bytes of the SHA-1 hash of the keychain's filename.

Note: using only 4 bytes (32 bits) of the hash, there's a reasonable chance of a hash collision, which is mentioned in the comment in LocalFileLocker::lock().

Operation of lock

The flock() system call is used to manipulate the lock on the lock file.

Here's the call tree when the keychain's database is being locked for writing:

DbModifier::createDatabase() or ::modifyDatabase() or ::deleteDatabase()
  AtomicFile::create() or ::write() or ::performDelete()
    AtomicLockedFile::lock()
      LocalFileLocker::lock()
        flock(mLockFile, LOCK_EX)  // exclusive lock

and when it's being unlocked after writing:

DbModifier::commit()
  AtomicTempFile::commit()
    RefPointer<AtomicLockedFile>::setPointer(AtomicLockedFile*)
      RefPointer<AtomicLockedFile>::release_internal()
        AtomicLockedFile::~AtomicLockedFile()  // destructor
          AtomicLockedFile::unlock()
            LocalFileLocker::unlock()
              flock(mLockFile, LOCK_UN)  // unlock