Accessing files attributes vs accessing sqlite records

Question

In one of our apps I was asked to keep record of the last modified date of images . That way I can check with the server if a certain image has changed and update my cache accordingly.

My first approach was to access the files attributes and perform a comparison but a few places online mentioned a drastic bottleneck in terms of latency.

My second choice was to create a SQLite table to manage it. (Using fmdb)

I've decided to write a simple latency test. In the next test I'm accessing 500 file attributes and 500 sqlite records:

- (void)latencyTest
{
    NSMutableArray *arrayTest1 = [[NSMutableArray alloc]init];
    NSMutableArray *arrayTest2 = [[NSMutableArray alloc]init];
    FMResultSet *results = [_database executeQuery:@"SELECT * FROM `tb_media`"];
    NSDateFormatter *formatter = [[NSDateFormatter alloc] init];
    [formatter setDateFormat:@"dd-MM-yyyy HH:mm:ss:SSS"];
    NSLog(@"Time1: %@",[formatter stringFromDate:[NSDate date]]);
    int i=1;
    while(i<501)
    {
        NSString *test = [NSString stringWithFormat:@"%@/_media/media/19/%d.jpg",_outputPath,i];
        NSDictionary *attributes = [[NSFileManager defaultManager] attributesOfItemAtPath:test error:nil];
        NSDate *dateX = [attributes fileModificationDate];
        [arrayTest1 addObject:dateX];
        i++;
    }
    NSLog(@"Time2: %@",[formatter stringFromDate:[NSDate date]]);
    while([results next])
    {        
        NSDate *myDate = [NSDate dateWithTimeIntervalSince1970:[results intForColumn:@"last_update"]];
        [arrayTest2 addObject:myDate];
    }
    NSLog(@"Time3: %@",[formatter stringFromDate:[NSDate date]]);
}

Results:

    //iPhone 5 (Actual Device) 500 Pics

    Files Start:                 05-03-2014 09:31:20:375 
    Files End & Sqlite start:    05-03-2014 09:31:20:491 
    Sqlite end:                  05-03-2014 09:31:20:507

    Files Start:                 05-03-2014 09:31:56:305 
    Files End & Sqlite start:    05-03-2014 09:31:56:421 
    Sqlite end:                  05-03-2014 09:31:56:437

    Files Start:                 05-03-2014 09:32:19:053 
    Files End & Sqlite start:    05-03-2014 09:32:19:170 
    Sqlite end:                  05-03-2014 09:32:19:187

As you can see the results are pretty much the same. My questions are:

1. I was under the assumption that accessing one file at a time using attributesOfItemAtPath will take a lot longer than sql. Am I missing something?

2. Does attributesOfItemAtPath really accessing the file or the iOS filesystem keeps all the attributes in some sort of database for easy access?

3. After seeing the above results I've decided to go with the attributesOfItemAtPath method. Is there anything else I'm not considering passing on sqlite?

Answer 1

Before I discuss the solution , there are certain problems in your evaluation strategy.

1) You haven't incorporated time taken by NSLog and while loop. 75% of time is taken by them whereas you just want to compare intForColumn vs attributesOfItemAtPath. A correct way of doing this is run instruments Timer Profiler and compare time for retrieving a single record.

2) You have used FMDB as a file manager. Internally FMDB serialize data in a file. The core of FMDB / SQL Lite lies in its data structure specially indexing which you haven't used at all. So even if you compare time taken for your records you will observer FMDB is taking more time than file manager because of extra overhead to serialize data in specific format.

3) Access time of X number of records is compared by the # of times access is done to disk (hard drive) not heap. What you are doing is heap access of data store in both cases. So you won't be able to see any difference at all.

Does this mean File Manager is better than FMDB , absolutely not!! Here are few reasons why :

FMDB performs well only when its configured to do so. The core of FMDB lies in two things paging (caching into heap) and index. Let me explain you each one at a time.

1)Assume that you are trying to access timestamp for 100 images. Where each image is having 1000 timestamps. This mean you have to make 100*1000 = 100,000 access to data store. If images are small then Filemanager will load the file into heap and access will be faster than FMDB but if you don't have sufficient heap space your application will issue memory warning and access file from disk rather than cache which is significantly slower.

So its a binary state either all from heap or all from disk

FMDB outperforms this state and retrieves partial records depending upon the available heap space. This makes access faster when you have a huge chuck of records.

The ideal way to test this scenario is to run your fucntion latencyTest for atleast 10,000 images (not timestamp). This way the log times and iteration speed will be negligible as compared to overall time taken.

2) Index structure, this goes back to the basics of SQL Lite. You might want to add an extra attribute call as number of access to images and index your table on that. This will boost up perforce significantly. Not quite possible with Filemeanger.

Solution which I recommend.
1) If you have data less than 2 MB (images plus timestamp ) go for Filemenager

2) If data is > 2MB go for Core Data / FMDB.

Core data has additional performance tuning for multithreaded environment and many more additional features like seamless integration for encryption.