NullPointerException at Lucene-Appengine's SegmentIndexInput readByte method

Question

I am getting the following error trace from running Lucene. I am including the code segment that prompted the error. The specific project/connector is Lucene for Appengine

CODE

public static void resetAllIndicesUsingDatastore() {
    LOG.info("Inside  resetAllIndicesUsingDatastore ");
    StandardAnalyzer analyzer = new StandardAnalyzer();
    GaeDirectory directory = new GaeDirectory(LuceneWorker.DOGS);// create dog index
    IndexWriterConfig config = GaeLuceneUtil.getIndexWriterConfig(LuceneWorker.LUCENE_VERSION, analyzer);
    IndexWriter writer = null;
    LOG.info("GOING TO TRY ");
    try {
      // delete all elements
      writer = new IndexWriter(directory, config);//LINE 142
      LOG.info("Going to delete all that's there ");
      writer.deleteAll();
      writer.commit();

      // get all dogs from datastore
      List<Dog> dogs = DogDaoImpl.getAll();
      LOG.info("All dogs size is : " + dogs.size());

      for (Dog p : dogs) {
        LuceneIndexDogTaskQueue.indexDog(p.getDogId());
        LOG.info("Dog sent to ask queue for lucene is : " + p.getTitle() + " with id: " + p.getDogId());
      }
      LOG.info("Leaving  resetAllIndicesUsingDatastore with no error ");
    } catch (IOException e) {
      LOG.info("Leaving  resetAllIndicesUsingDatastore with ERROR " + e.getLocalizedMessage() + "   msg:: " + e.getMessage());
      e.printStackTrace();
    } finally {
      if (null != writer) {
        try {
          writer.close();
        } catch (IOException e) {
          e.printStackTrace();
        }
      }
    }

ERROR

com.google.api.server.spi.SystemService invokeServiceMethod: cause={0}
java.lang.NullPointerException
    at com.googlecode.luceneappengine.SegmentIndexInput.readByte(SegmentIndexInput.java:70)
    at org.apache.lucene.store.BufferedChecksumIndexInput.readByte(BufferedChecksumIndexInput.java:41)
    at org.apache.lucene.store.DataInput.readInt(DataInput.java:98)
    at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:347)
    at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:458)
    at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:913)
    at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:759)
    at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:454)
    at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:783)
    at com.company.dogappcloud.taskqueue.lucene.LuceneSearchDog.resetAllIndicesUsingDatastore(LuceneSearchDog.java:142)






//remaining of trace but not as important

at com.company.dogappcloud.dogappApi.findDog(dogappApi.java:968)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:45)
at com.google.api.server.spi.SystemService.invokeServiceMethod(SystemService.java:359)
at com.google.api.server.spi.SystemServiceServlet.execute(SystemServiceServlet.java:160)
at com.google.api.server.spi.SystemServiceServlet.doPost(SystemServiceServlet.java:118)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
at com.googlecode.objectify.cache.AsyncCacheFilter.doFilter(AsyncCacheFilter.java:58)
at com.googlecode.objectify.ObjectifyFilter.doFilter(ObjectifyFilter.java:48)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:437)
at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:444)
at com.google.tracing.CurrentContext.runInContext(CurrentContext.java:230)
at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:308)
at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:300)
at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:441)
at java.lang.Thread.run(Thread.java:745)

Here is my appengine-web.xml file and also I am using the modified RamUsageEstimator as suggested

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
    <application>myApplicationId</application>
    <version>1</version>
    <threadsafe>true</threadsafe>

    <system-properties>
        <property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>
        <property name="java.util.logging.config.file"
                  value="WEB-INF/java-util-logging.properties"/>
        <property name="os.version" value="1.0.GAE whatever"/>
        <property name="os.arch" value="GAE whatever"/>
    </system-properties>

    <class-loader-config>
        <priority-specifier filename="luceneappengine-2.2.0.jar"/>
    </class-loader-config>
</appengine-web-app>

Answer 1

Looking at the code of SegmentIndexInput hunk is null.

@Override
    public byte readByte() throws IOException {
70     if(hunkPointer >= hunk.bytes.length) {
                    hunkPointer = 0;

Looking at code of GaeDirectory the null is because segment has no hunks created. Before you use the new directory, you should call createOutput(String name) or touchFile(StringName) so that segment with at least one hunk is created