Record method calls in one session for replaying in future test sessions?

Question

I have a backend system which we use a third-party Java API to access from our own applications. I can access the system as a normal user along with other users, but I do not have godly powers over it.

Hence to simplify testing I would like to run a real session and record the API calls, and persist them (preferably as editable code), so we can do dry test runs later with API calls just returning the corresponding response from the recording session - and this is the important part - without needing to talk to the above mentioned backend system.

So if my application contains line on the form:

 Object b = callBackend(a); 

I would like the framework to first capture that callBackend() returned b given the argument a, and then when I do the dry run at any later time say "hey, given a this call should return b". The values of a and b will be the same (if not, we will rerun the recording step).

I can override the class providing the API so all the method calls to capture will go through my code (i.e. byte code instrumentation to alter behavior of classes outside my control is not necessary).

What framework should I look into to do this?

---

EDIT: Please note that bounty hunters should provide actual code demonstrating the behavior I look for.

Answer 1

Actually You can build such framework or template, by using proxy pattern. Here I explain, how you can do it using dynamic proxy pattern. The idea is to,

1. Write a proxy manager to get recorder and replayer proxies of API on demand!
2. Write a wrapper class to store your collected information and also implement hashCode and equals method of that wrapper class for efficient lookup from Map like data structure.
3. And finally use recorder proxy to record and replayer proxy for replaying purpose.

How recorder works:

1. invokes the real API
2. collects the invocation information
3. persists data in expected persistence context

How replayer works:

1. Collect the method information (method name, parameters)
2. If collected information matches with previously recorded information then return the previously collected return value.
3. If returned value does not match, persist the collected information (As you wanted).

Now, lets look at the implementation. If your API is MyApi like bellow:

public interface MyApi {     public String getMySpouse(String myName);     public int getMyAge(String myName);     ... } 

Now we will, record and replay the invocation of public String getMySpouse(String myName). To do that we can use a class to store the invocation information like bellow:

    public class RecordedInformation {        private String methodName;        private Object[] args;        private Object returnValue;          public String getMethodName() {             return methodName;         }          public void setMethodName(String methodName) {             this.methodName = methodName;         }          public Object[] getArgs() {             return args;         }          public void setArgs(Object[] args) {             this.args = args;         }          public Object getReturnValue() {             return returnType;         }          public void setReturnValue(Object returnValue) {             this.returnValue = returnValue;         }          @Override         public int hashCode() {             return super.hashCode();  //change your implementation as you like!         }          @Override         public boolean equals(Object obj) {             return super.equals(obj);    //change your implementation as you like!         }     } 

Now Here comes the main part, The RecordReplyManager. This RecordReplyManager gives you proxy object of your API , depending on your need of recording or replaying.

    public class RecordReplyManager implements java.lang.reflect.InvocationHandler {          private Object objOfApi;         private boolean isForRecording;          public static Object newInstance(Object obj, boolean isForRecording) {              return java.lang.reflect.Proxy.newProxyInstance(                     obj.getClass().getClassLoader(),                     obj.getClass().getInterfaces(),                     new RecordReplyManager(obj, isForRecording));         }          private RecordReplyManager(Object obj, boolean isForRecording) {             this.objOfApi = obj;             this.isForRecording = isForRecording;         }           @Override         public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {             Object result;             if (isForRecording) {                 try {                     System.out.println("recording...");                     System.out.println("method name: " + method.getName());                     System.out.print("method arguments:");                     for (Object arg : args) {                         System.out.print(" " + arg);                     }                     System.out.println();                     result = method.invoke(objOfApi, args);                     System.out.println("result: " + result);                     RecordedInformation recordedInformation = new RecordedInformation();                     recordedInformation.setMethodName(method.getName());                     recordedInformation.setArgs(args);                     recordedInformation.setReturnValue(result);                     //persist your information                  } catch (InvocationTargetException e) {                     throw e.getTargetException();                 } catch (Exception e) {                     throw new RuntimeException("unexpected invocation exception: " +                             e.getMessage());                 } finally {                     // do nothing                 }                 return result;             } else {                 try {                     System.out.println("replying...");                     System.out.println("method name: " + method.getName());                     System.out.print("method arguments:");                     for (Object arg : args) {                         System.out.print(" " + arg);                     }                      RecordedInformation recordedInformation = new RecordedInformation();                     recordedInformation.setMethodName(method.getName());                     recordedInformation.setArgs(args);                      //if your invocation information (this RecordedInformation) is found in the previously collected map, then return the returnValue from that RecordedInformation.                     //if corresponding RecordedInformation does not exists then invoke the real method (like in recording step) and wrap the collected information into RecordedInformation and persist it as you like!                  } catch (InvocationTargetException e) {                     throw e.getTargetException();                 } catch (Exception e) {                     throw new RuntimeException("unexpected invocation exception: " +                             e.getMessage());                 } finally {                     // do nothing                 }                 return result;             }         }     } 

If you want to record the method invocation, all you need is getting an API proxy like bellow:

    MyApi realApi = new RealApi(); // using new or whatever way get your service implementation (API implementation)     MyApi myApiWithRecorder = (MyApi) RecordReplyManager.newInstance(realApi, true); // true for recording     myApiWithRecorder.getMySpouse("richard"); // to record getMySpouse     myApiWithRecorder.getMyAge("parker"); // to record getMyAge     ... 

And to replay all you need:

    MyApi realApi = new RealApi(); // using new or whatever way get your service implementation (API implementation)     MyApi myApiWithReplayer = (MyApi) RecordReplyManager.newInstance(realApi, false); // false for replaying     myApiWithReplayer.getMySpouse("richard"); // to replay getMySpouse     myApiWithRecorder.getMyAge("parker"); // to replay getMyAge     ... 

And You are Done!

Edit: The basic steps of recorder and replayers can be done in above mentioned way. Now its upto you, that how you want to use or perform those steps. You can do what ever you want and whatever you like in the recorder and replayer code blocks and just choose your implementation!

Answer 2

I should prefix this by saying I share some of the concerns in Yves Martin's answer: that such a system may prove frustrating to work with and ultimately less helpful than it would seem at first blush.

That said, from a technical standpoint, this is an interesting problem, and I couldn't not take a go at it. I put together a gist to log method calls in a fairly general way. The CallLoggingProxy class defined there allows usage such as the following.

Calendar original = CallLoggingProxy.create(Calendar.class, Calendar.getInstance()); original.getTimeInMillis(); // 1368311282470  CallLoggingProxy.ReplayInfo replayInfo = CallLoggingProxy.getReplayInfo(original);  // Persist the replay info to disk, serialize to a DB, whatever floats your boat. // Come back and load it up later...  Calendar replay = CallLoggingProxy.replay(Calendar.class, replayInfo); replay.getTimeInMillis(); // 1368311282470 

You could imagine wrapping your API object with CallLoggingProxy.create prior to passing it into your testing methods, capturing the data afterwards, and persisting it using whatever your favorite serialization system happens to be. Later, when you want to run your tests, you can load the data back up, create a new instance based on the data with CallLoggingProxy.replay, and passing that into your methods instead.

The CallLoggingProxy is written using Javassist, as Java's native Proxy is limited to working against interfaces. This should cover the general use case, but there are a few limitations to keep in mind:

• Classes declared final can't be proxied by this method. (Not easily fixable; this is a system limitation)
• The gist assumes the same input to a method will always produce the same output. (More easily fixable; the ReplayInfo would need to keep track of sequences of calls for each input instead of single input/output pairs.)
• The gist is not even remotely threadsafe (Fairly easily fixable; just requires a little thought and effort)

Obviously the gist is simply a proof of concept, so it's also not been very thoroughly tested, but I believe the general principle is sound. It's also possible there's a more fully baked framework out there to achieve this sort of goal, but if such a thing does exist, I'm not aware of it.

If you do decide to continue with the replay approach, then hopefully this will be enough to give you a possible direction to work in.