HBase shell "OutOfOrderScannerNextException" error on scanner & count calls

Question

Either I run a scan command or a count, this error pops up and the error message doesn't make sense to me. What does it say & how to solve it?

org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: Expected nextCallSeq: 1 But the nextCallSeq got from client: 0; request=scanner_id: 788 number_of_rows: 100 close_scanner: false next_call_seq: 0

Commands:

count 'table', 5000 scan 'table', {COLUMN => ['cf:cq'], FILTER => "ValueFilter( =, 'binaryprefix:somevalue')"}

EDIT:

I have added the following settings in hbase-site.xml

<property>
    <name>hbase.rpc.timeout</name>
    <value>1200000</value>
  </property>
  <property>
    <name>hbase.client.scanner.caching</name>
    <value>100</value>
 </property>

NO IMPACT

EDIT2: Added sleep

                Result[] results = scanner.next(100);

                for (int i = 0; i < results.length; i++) {
                    result = results[i];
                    try {
                        ...
                        count++;
                        ...
                        Thread.sleep(10); // ADDED SLEEP
                    } catch (Throwable exception) {
                        System.out.println(exception.getMessage());

                        System.out.println("sleeping");
                    }
                }

New Error after Edit2:

org.apache.hadoop.hbase.client.ScannerTimeoutException: 101761ms passed since the last invocation, timeout is currently set to 60000
...


Caused by: org.apache.hadoop.hbase.UnknownScannerException: org.apache.hadoop.hbase.UnknownScannerException: Name: 31, already closed?
    ...

Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.UnknownScannerException): org.apache.hadoop.hbase.UnknownScannerException: Name: 31, already closed?
    ...

FINALLY BLOCK: 9900
Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.hbase.client.ScannerTimeoutException: 101766ms passed since the last invocation, timeout is currently set to 60000
    ...


Caused by: org.apache.hadoop.hbase.client.ScannerTimeoutException: 101766ms passed since the last invocation, timeout is currently set to 60000
    ...

Caused by: org.apache.hadoop.hbase.UnknownScannerException: org.apache.hadoop.hbase.UnknownScannerException: Name: 31, already closed?
    ...

Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.UnknownScannerException): org.apache.hadoop.hbase.UnknownScannerException: Name: 31, already closed?
    ...

Answer 1

EDIT: By using the same client version shipped with the downloaded hbase(not maven 0.99), i was able to solve this issue. Server version is 0.98.6.1 Contains client jars inside ./lib folder

Don't forget to attach the zookeeper library

---

OLD:

Right now I did two things, changed the table connection API (0.99)

                    Configuration conf = HBaseConfiguration.create();
                    TableName name = TableName.valueOf("TABLENAME"); 
                    Connection conn = ConnectionFactory.createConnection(conf);
                    Table table = conn.getTable(name);

Then when the error pops up, i try to recreate the connection

                    scanner.close();
                    conn.close();
                    conf.clear();
                    conf = HBaseConfiguration.create();
                    conn = ConnectionFactory.createConnection(conf);
                    table = conn.getTable(name);
                    table = ConnectionFactory.createConnection(conf).getTable(name);
                    scanner = table.getScanner(scan);

This works but is might slow after the first error it receives. Very slow to scan through all the rows