Scenario: Apache Phoenix connectivity issues in Azure HDInsight
This article describes troubleshooting steps and possible resolutions for issues when interacting with Azure HDInsight clusters.
Issue
Unable to connect to Apache HBase with Apache Phoenix. Reasons may vary.
Cause: Incorrect IP
Incorrect IP of active Zookeeper node.
Resolution
The IP of the active Zookeeper node can be identified from the Ambari UI by following the links to HBase > Quick Links > ZK (Active) > Zookeeper Info. Correct the IP as needed.
Cause: SYSTEM.CATALOG table offline
When running commands such as !tables, you receive an error message similar to:
Error while connecting to sqlline.py (Hbase - phoenix) Setting property: [isolation, TRANSACTION_READ_COMMITTED] issuing: !connect jdbc:phoenix:10.2.0.7 none none org.apache.phoenix.jdbc.PhoenixDriver Connecting to jdbc:phoenix:10.2.0.7 SLF4J: Class path contains multiple SLF4J bindings.
When running commands such as count 'SYSTEM.CATALOG', you receive an error message similar to:
ERROR: org.apache.hadoop.hbase.NotServingRegionException: Region SYSTEM.CATALOG,,1485464083256.c0568c94033870c517ed36c45da98129. is not online on 10.2.0.5,16020,1489466172189)
Resolution
From the Apache Ambari UI, complete the following steps to restart the HMaster service on all ZooKeeper nodes:
From the Summary section of HBase, go to HBase > Active HBase Master.
From the Components section, restart the HBase Master service.
Repeat these steps for all remaining Standby HBase Master services.
It can take up to five minutes for the HBase Master service to stabilize and finish the recovery. After the SYSTEM.CATALOG table is back to normal, the connectivity issue to Apache Phoenix should get resolved automatically.