SQLite VFS implementation guide lines with FOpen*

Question

I am about to implement a custom VFS (virtual file system) for a Netburner embedded device (non windows) using FOpen, FRead, FWrite, FSeek, and FClose. I was surprised that i could not find a FOpen* version of the VFS available. It would make it a lot more portable to embedded devices.

I found some information on creating the VFS for SQLite here http://sqlite.org/c3ref/vfs.html but the information is very detailed and I have lots of other questions about the implementation.

I have some example VFS in the SQLite source code for Win, OS2, Linux but they don't have a lot of comments, only source code.

I could use the information provided in the link above and the examples to create my custom VFS but i'm sure that I would miss something if I did it that way.

My questions are:

• Is there any more documentation about the SQLite VFS that I am missing? Maybe an implementation guide?
• Is there an Fopen version of the SQLite VFS that is available?
• Is there a unit testing code available to test my custom SQLite VFS once I have created it?
• Suggestions, comments, experiences with implementing SQLite VFS that you would like to share.

Answer 1

Did you notice that there is an additional source of documentation in the header file sqlite3.h? Also, Google code search is your friend.

Don't worry too much about missing things, this is what the test suite is for. Take a guess at the purpose of every method from their name, the documentation and the example implementations; go for a first-draft implementation; run the tests on your target platform; iterate until the bar is green. From a cursory reading of the interface doc that you quoted, here are some educated guesses:

  int (*xOpen)(sqlite3_vfs*, const char *zName, sqlite3_file*,
               int flags, int *pOutFlags);
  int (*xDelete)(sqlite3_vfs*, const char *zName, int syncDir);
  int (*xAccess)(sqlite3_vfs*, const char *zName, int flags, int *pResOut);
  int (*xFullPathname)(sqlite3_vfs*, const char *zName, int nOut, char *zOut);

Those are your run-off-the-mill file management functions. You'll notice that xOpen() in turn returns a structure sqlite3_file, that has pointer methods of its own for reading and writing.

  void *(*xDlOpen)(sqlite3_vfs*, const char *zFilename);
  void (*xDlError)(sqlite3_vfs*, int nByte, char *zErrMsg);
  void (*(*xDlSym)(sqlite3_vfs*,void*, const char *zSymbol))(void);
  void (*xDlClose)(sqlite3_vfs*, void*);

Those are for shared libraries (see the dlopen() man page on Linux). In an embedded environment, you probably can leave these unimplemented (try setting these to NULL).

  int (*xRandomness)(sqlite3_vfs*, int nByte, char *zOut);

You might have to implement a random number generator, if your OS' standard library doesn't provide one already. I suggest a linear feedback register, which is small yet good.

  int (*xSleep)(sqlite3_vfs*, int microseconds);
  int (*xCurrentTime)(sqlite3_vfs*, double*);
  int (*xCurrentTimeInt64)(sqlite3_vfs*, sqlite3_int64*);

These are the time management functions, to hook up with your OS.

  int (*xGetLastError)(sqlite3_vfs*, int, char *);

You can get away by always returning 0 here :-) See unixGetLastError in os_unix.c (thanks Google Code Search!)

Good luck!

Answer 2

One option is to use a memory based VFS then simply dump the memory to file when you are done. See: http://article.gmane.org/gmane.comp.db.sqlite.general/46450 for a memory based VFS that already supports serialization/deserialization.

The disadvantage is that you must manually write the file out for it to persist. If your application suddenly dies any intermediate changes to the DB will not be persisted.