Flat file databases [closed]

Question

Well, what is the nature of the flat databases. Are they large or small. Is it simple arrays with arrays in them? if its something simple say userprofiles built as such:

$user = array("name" => "dubayou", 
              "age" => 20,
              "websites" => array("dubayou.com","willwharton.com","codecream.com"),
              "and_one" => "more");

and to save or update the db record for that user.

$dir = "../userdata/";  //make sure to put it bellow what the server can reach.
file_put_contents($dir.$user['name'],serialize($user));

and to load the record for the user

function &get_user($name){
    return unserialize(file_get_contents("../userdata/".$name));
}

but again this implementation will vary on the application and nature of the database you need.

You might consider SQLite. It's almost as simple as flat files, but you do get a SQL engine for querying. It works well with PHP too.

In my opinion, using a "Flat File Database" in the sense you're meaning (and the answer you've accepted) isn't necessarily the best way to go about things. First of all, using serialize() and unserialize() can cause MAJOR headaches if someone gets in and edits the file (they can, in fact, put arbitrary code in your "database" to be run each time.)

Personally, I'd say - why not look to the future? There have been so many times that I've had issues because I've been creating my own "proprietary" files, and the project has exploded to a point where it needs a database, and I'm thinking "you know, I wish I'd written this for a database to start with" - because the refactoring of the code takes way too much time and effort.

From this I've learnt that future proofing my application so that when it gets bigger I don't have to go and spend days refactoring is the way to go forward. How do I do this?

SQLite. It works as a database, uses SQL, and is pretty easy to change over to MySQL (especially if you're using abstracted classes for database manipulation like I do!)

In fact, especially with the "accepted answer"'s method, it can drastically cut the memory usage of your app (you don't have to load all the "RECORDS" into PHP)

One framework I'm considering would be for a blogging platform. Since just about any possible view of data you would want would be sorted by date, I was thinking about this structure:

One directory per content node:

./content/YYYYMMDDHHMMSS/

Subdirectories of each node including

/tags  
/authors  
/comments  

As well as simple text files in the node directory for pre- and post-rendered content and the like.

This would allow a simple PHP glob() call (and probably a reversal of the result array) to query on just about anything within the content structure:

glob("content/*/tags/funny");  

Would return paths including all articles tagged "funny".

Here's the code we use for Lilina:

<?php
/**
 * Handler for persistent data files
 *
 * @author Ryan McCue <[email protected]>
 * @package Lilina
 * @version 1.0
 * @license http://opensource.org/licenses/gpl-license.php GNU Public License
 */

/**
 * Handler for persistent data files
 *
 * @package Lilina
 */
class DataHandler {
    /**
     * Directory to store data.
     *
     * @since 1.0
     *
     * @var string
     */
    protected $directory;

    /**
     * Constructor, duh.
     *
     * @since 1.0
     * @uses $directory Holds the data directory, which the constructor sets.
     *
     * @param string $directory 
     */
    public function __construct($directory = null) {
        if ($directory === null)
            $directory = get_data_dir();

        if (substr($directory, -1) != '/')
            $directory .= '/';

        $this->directory = (string) $directory;
    }

    /**
     * Prepares filename and content for saving
     *
     * @since 1.0
     * @uses $directory
     * @uses put()
     *
     * @param string $filename Filename to save to
     * @param string $content Content to save to cache
     */
    public function save($filename, $content) {
        $file = $this->directory . $filename;

        if(!$this->put($file, $content)) {
            trigger_error(get_class($this) . " error: Couldn't write to $file", E_USER_WARNING);
            return false;
        }

        return true;
    }

    /**
     * Saves data to file
     *
     * @since 1.0
     * @uses $directory
     *
     * @param string $file Filename to save to
     * @param string $data Data to save into $file
     */
    protected function put($file, $data, $mode = false) {
        if(file_exists($file) && file_get_contents($file) === $data) {
            touch($file);
            return true;
        }

        if(!$fp = @fopen($file, 'wb')) {
            return false;
        }

        fwrite($fp, $data);
        fclose($fp);

        $this->chmod($file, $mode);
        return true;

    }

    /**
     * Change the file permissions
     *
     * @since 1.0
     *
     * @param string $file Absolute path to file
     * @param integer $mode Octal mode
     */
    protected function chmod($file, $mode = false){
        if(!$mode)
            $mode = 0644;
        return @chmod($file, $mode);
    }

    /**
     * Returns the content of the cached file if it is still valid
     *
     * @since 1.0
     * @uses $directory
     * @uses check() Check if cache file is still valid
     *
     * @param string $id Unique ID for content type, used to distinguish between different caches
     * @return null|string Content of the cached file if valid, otherwise null
     */
    public function load($filename) {
        return $this->get($this->directory . $filename);
    }

    /**
     * Returns the content of the file
     *
     * @since 1.0
     * @uses $directory
     * @uses check() Check if file is valid
     *
     * @param string $id Filename to load data from
     * @return bool|string Content of the file if valid, otherwise null
     */
    protected function get($filename) {
        if(!$this->check($filename))
            return null;

        return file_get_contents($filename);
    }

    /**
     * Check a file for validity
     *
     * Basically just a fancy alias for file_exists(), made primarily to be
     * overriden.
     *
     * @since 1.0
     * @uses $directory
     *
     * @param string $id Unique ID for content type, used to distinguish between different caches
     * @return bool False if the cache doesn't exist or is invalid, otherwise true
     */
    protected function check($filename){
        return file_exists($filename);
    }

    /**
     * Delete a file
     *
     * @param string $filename Unique ID
     */
    public function delete($filename) {
        return unlink($this->directory . $filename);
    }
}

?>

It stores each entry as a separate file, which we found is efficient enough for use (no unneeded data is loaded and it's faster to save).

IMHO, you have two... er, three options if you want to avoid homebrewing something:

1. SQLite

If you're familiar with PDO, you can install a PDO driver that supports SQLite. Never used it, but I have used PDO a ton with MySQL. I'm going to give this a shot on a current project.

2. XML

Done this many times for relatively small amounts of data. XMLReader is a lightweight, read-forward, cursor-style class. SimpleXML makes it simple to read an XML document into an object that you can access just like any other class instance.

3. JSON (update)

Good option for smallish amounts of data, just read/write file and json_decode/json_encode. Not sure if PHP offers a structure to navigate a JSON tree without loading it all in memory though.

If you're going to use a flat file to persist data, use XML to structure the data. PHP has a built-in XML parser.