Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can I precompile the format string in String.format? (Or do any other thing to make formatting logs faster?)

It is well known that String.format() performance is terrible. I see big possible improvements in my (and probably very common) typical case. I print same structure of data many times. Let imagine the structure like "x:%d y:%d z:%d". I expect that the main trouble with String.format() is that it has to always parse formatting string. My question is: Is there some ready made class which would allow to read formatting string only once and then allow to quickly give string when variable parameters filled?? Usage shall look like this:

PreString ps = new PreString("x:%d y:%d z:%d");
String s;
for(int i=0;i<1000;i++){
    s = ps.format(i,i,i); 
}

I know it is possible - following is my quick & dirty example which do what I'm talking about and is about ~10 times faster at my machine:

public interface myPrintable{
    boolean isConst();
    String prn(Object o);
    String prn();
}

public class MyPrnStr implements myPrintable{
    String s;
    public MyPrnStr(String s){this.s =s;}
    @Override public boolean isConst() { return true; }
    @Override public String prn(Object o) { return s; }
    @Override public String prn() { return s; }
}

public class MyPrnInt implements myPrintable{
    public MyPrnInt(){}
    @Override  public boolean isConst() { return false; }
    @Override  public String prn(Object o) { return String.valueOf((Integer)o);  }
    @Override  public String prn() { return "NumMissing";   }
}

public class FastFormat{
    myPrintable[]      obj    = new myPrintable[100];
    int                objIdx = 0;
    StringBuilder      sb     = new StringBuilder();

    public FastFormat() {}

    public void addObject(myPrintable o) {  obj[objIdx++] = o;   }

    public String format(Object... par) {
        sb.setLength(0);
        int parIdx = 0;
        for (int i = 0; i < objIdx; i++) {
            if(obj[i].isConst()) sb.append(obj[i].prn());
            else                 sb.append(obj[i].prn(par[parIdx++]));
        }
        return sb.toString();
    }
}

It is used like this:

FastFormat ff = new FastFormat();
ff.addObject(new MyPrnStr("x:"));
ff.addObject(new MyPrnInt());
ff.addObject(new MyPrnStr(" y:"));
ff.addObject(new MyPrnInt());
ff.addObject(new MyPrnStr(" z:"));
ff.addObject(new MyPrnInt());
for (int i = 0; i < rpt; i++) {
    s = ff.format(i,i,i);
}

when I compare with

long beg = System.nanoTime();
for (int i = 0; i < rpt; i++) {
    s = String.format("x:%d y:%d z:%d", i, i, i);
}
long diff = System.nanoTime() - beg;

For 1e6 iteration pre-formatting improves result by factor of ~10:

time [ns]: String.format()     (+90,73%)  3 458 270 585 
time [ns]: FastFormat.format() (+09,27%)    353 431 686 

[EDIT]

As Steve Chaloner replied there is a MessageFormat which is quite doing what I want. So I tried the code:

MessageFormat mf = new MessageFormat("x:{0,number,integer} y:{0,number,integer} z:{0,number,integer}");
Object[] uo = new Object[3];
for (int i = 0; i < rpt; i++) {
    uo[0]=uo[1]=uo[2] = i;
    s = mf.format(uo);
}

And it is faster only by factor of 2. Not the factor of 10 which I hoped. Again see measurement for 1M iteration (JRE 1.8.0_25-b18 32bit):

time [s]: String.format()     (+63,18%)  3.359 146 913 
time [s]: FastFormat.format() (+05,99%)  0.318 569 218 
time [s]: MessageFormat       (+30,83%)  1.639 255 061 

[EDIT2]

As Slanec replied there is org.slf4j.helpers.MessageFormatter. (I tried library version slf4j-1.7.12)

I did tried to compare code:

Object[] uo2 = new Object[3];
beg = System.nanoTime();
for(long i=rpt;i>0;i--){
    uo2[0]=uo2[1]=uo2[2] = i;
    s = MessageFormatter.arrayFormat("x: {} y: {} z: {}",uo2).getMessage();
}

with code for MessageFormat given above in section [EDIT]. I did get following results for looping it 1M times:

Time MessageFormatter [s]: 1.099 880 912
Time MessageFormat    [s]: 2.631 521 135
speed up : 2.393 times

So MessageFormatter is best answer so far yet my simple example is still little bit faster... So any ready made faster library proposal?

like image 365
Vit Bernatik Avatar asked Apr 20 '15 12:04

Vit Bernatik


People also ask

What is format and string format?

In java, String format() method returns a formatted string using the given locale, specified format string, and arguments. We can concatenate the strings using this method and at the same time, we can format the output concatenated string. Parameter: The locale value to be applied on the format() method.

Is it good to use string format?

tl;dr. Avoid using String. format() when possible. It is slow and difficult to read when you have more than two variables.


2 Answers

It sounds like you want MessageFormat

From the documentation:

The following example creates a MessageFormat instance that can be used repeatedly:

 int fileCount = 1273;
 String diskName = "MyDisk";
 Object[] testArgs = {new Long(fileCount), diskName};

 MessageFormat form = new MessageFormat(
     "The disk \"{1}\" contains {0} file(s).");
 System.out.println(form.format(testArgs));
like image 136
Steve Chaloner Avatar answered Oct 05 '22 06:10

Steve Chaloner


If you're looking for a fast implementation, you need to look outside the JDK. You probably use slf4j for logging anyway, so let's look at its MessageFormatter:

MessageFormatter.arrayFormat("x:{} y:{} z:{}", new Object[] {i, i, i}).getMessage();

On my machine (and a crude and flawed microbenchmark), it's around 1/6 slower than your FastFormat class, and around 5-10 times faster than either String::format or MessageFormat.

like image 24
Petr Janeček Avatar answered Oct 05 '22 07:10

Petr Janeček