Limit Derby Log File Size

Question

Our app sends the contents of the derby.log file to our server whenever Apache Derby throws a SQLException in our app.

In order to get detailed logs, we are setting the 'derby.infolog.append' property to true.

However, we are noticing enormously large logs files since the logs also contain bootup output each time a connection is made to the database.

NOTE: we are using Derby in embedded mode.

Is there a way to have derby limit the total number of lines it logs to derby.log file?

For example, only logging the most recent 1000 lines of logs and then begin to overwrite the oldest entries.

Our goal is to get useful debugging info from end users but to prevent the log files from growing to unmanageable sizes.

Thanks in advance,

Jim

Answer 1

I'm not that familiar with derby but I couldn't find an "easy" way to do this.

But there are some derby properties you could set to implement this yourself.

Check these

derby.stream.error.field

derby.stream.error.file

derby.stream.error.method

derby.stream.error.logSeverityLevel

So I imagine you writing some class which subclasses java.io.OutputStream or java.io.Writer and then you either

• implements the wanted behaviour or
• do it similar to How do I limit the size of log file? + wrap as one of the above or
• you ripp-off get some ideas for a RollingFileLoggerClass from some other project (RollingFileAppender log4j, RollingFileWriter clapper, ...)

Answer 2

You can create a custom logging class, and specify this using derby.stream.error.field as mentioned above. The logging class doesn't have to implemented as a file - if you will be limiting the size of the logging data you can easily hold it in memory.

The second advantage to this is that when a problem is encountered, you have a great deal of flexibility in what to do with the logging data. Perhaps compress (or encrypt) the data and automatically open a ticket in your help system (as an example).

Here's an example of a very simple custom logging solution:

import java.io.CharArrayWriter;

public class CustomLog {

    public static CharArrayWriter log = new CharArrayWriter();

    public static void dump() {
        System.out.println(log.toString());
    }
}

You can replace the CharArrayWriter with a size limited buffer of some sort, and add an implementation of dump() to do what you will with the resulting log data.

A short example program demonstrating this follows:

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.Statement;

public class DerbyLoggingExample {

    public DerbyLoggingExample() {
        System.setProperty( "derby.stream.error.field", "CustomLog.log");

        String driver = "org.apache.derby.jdbc.EmbeddedDriver";
        String dbName = "logdemoDB";
        String connectionURL = "jdbc:derby:" + dbName + ";create=true";

        String createCommand = "create table test_table ("
            + "test_id int not null generated always as identity, "
            + "test_name varchar(20)"
            + ")";

        try {
            Class.forName(driver);
        }
        catch( java.lang.ClassNotFoundException e ) {
            System.out.println( "Could not load Derby driver." );
            return;
        }

        Connection conn = null;
        Statement statement = null;

        try {
            conn = DriverManager.getConnection(connectionURL);
            statement = conn.createStatement();

            statement.execute(createCommand);
        }
        catch( SQLException sqle ) {
            sqle.printStackTrace();
            System.out.println( "SQLException encountered. Dumping log.");
            CustomLog.dump();
            return;
        }
        finally {
            try {
                statement.close();
                conn.close();
            }
            catch( SQLException e ) {
                // Do nothing.
            }
        }

        System.out.println( "Processing done. Dumping log." );
        CustomLog.dump();
    }

    public static void main(String[] argv) {
        DerbyLoggingExample thisApp = new DerbyLoggingExample();
    }
}