Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can not read large data from phoenix table

Hi All i am getting below error message while running phoenix count query on a large table.

0: jdbc:phoenix:hadoopm1:2181> select Count(*) from PJM_DATASET;
+------------+
|  COUNT(1)  |
+------------+

java.lang.RuntimeException: org.apache.phoenix.exception.PhoenixIOException: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Fri Jan 09 02:18:10 CST 2015, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=62365: row '' on table 'PJM_DATASET' at region=PJM_DATASET,,1420633295836.4394a3aa2721f87f3e6216d20ebeec44., hostname=hadoopctrl,60020,1420790733247, seqNum=27753

    at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2440)
    at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2074)
    at sqlline.SqlLine.print(SqlLine.java:1735)
    at sqlline.SqlLine$Commands.execute(SqlLine.java:3683)
    at sqlline.SqlLine$Commands.sql(SqlLine.java:3584)
    at sqlline.SqlLine.dispatch(SqlLine.java:821)
    at sqlline.SqlLine.begin(SqlLine.java:699)
    at sqlline.SqlLine.mainWithInputRedirection(SqlLine.java:441)
    at sqlline.SqlLine.main(SqlLine.java:424)
0: jdbc:phoenix:hadoopm1:2181>

please help.

like image 969
user3683741 Avatar asked Jan 09 '15 08:01

user3683741


People also ask

What is Phoenix in big data?

Apache Phoenix is an open source, massively parallel, relational database engine supporting OLTP for Hadoop using Apache HBase as its backing store.

How does Apache Phoenix work?

Apache Phoenix abstracts the underlying data store and allows you to query data using standard SQL. Apache Phoenix provides features such as secondary indexes to help you boost the speed of your queries without relying on specific row-key designs and enables users to use star schemes.

How does HBase work with Phoenix?

Apache Phoenix breaks up SQL queries into multiple HBase scans and runs them in parallel. Phoenix is developed with notion of bringing the computation to the data by using: Coprocessors to perform operations on the server-side thus minimizing client/server data transfer.


1 Answers

You need to increase the below hbase configuration property to a higher value in hbase server and client configurations, default value is 1 min(60000 Milli Sec)

  <property>
    <name>hbase.rpc.timeout</name>
    <value>600000</value>
  </property>

The most important thing is that the below command should return the correct hbase config directory where hbase-site.xml file contains the above property. If the value of hbase_conf_path is empty or ., Execute the command export HADOOP_CONF_DIR=<correct_hbase_dir>; for pointing to correct hbase configuration before executing the command sqlline.py.

phoenix_utils.py  | grep hbase_conf_path
like image 137
SachinJ Avatar answered Oct 11 '22 07:10

SachinJ