Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

what is HiveServer and Thrift server [closed]

Tags:

hadoop

hive

I just started learning Hive.There are three terms which often I seen in Hive books or Hive tutorials. Hive Server,Hive Service and Thrift Server. What is these ? how they are related ?. what is the difference ?. when each of these are used? please explain

like image 281
Surendiran Balasubramanian Avatar asked Dec 02 '16 04:12

Surendiran Balasubramanian


People also ask

What does Thrift server mean?

Thrift comes in the architectural part of Hive, Thrift is a protocol for the application which were developed in a different programming languages to communicate. So the Thrift server sits in a hive services layer and this is the one which receives the request or hive queries from client programs.

Why Thrift server is used?

Thrift is an interface definition language and binary communication protocol used for defining and creating services for numerous programming languages. It was developed at Facebook for "scalable cross-language services development" and as of 2020 is an open source project in the Apache Software Foundation.

What is hive Thrift server?

HiveServer is an optional service that allows a remote client to submit requests to Hive, using a variety of programming languages, and retrieve results.

How does Thrift server work?

Thrift uses a special Interface Description Language (IDL) to define data types and service interfaces which are stored as . thrift files and used later as input by the compiler for generating the source code of client and server software that communicate over different programming languages.

What is hive thrift server?

What is Hive thrift server? Basically, a service which permits a remote client to submit requests to Hive, with the help of a variety of programming languages, as well as retrieve results, is what we call HiveServer. Since, It is built on Apache Thrift, that’s why we call it Thrift server sometimes.

What is hiveserver?

Basically, a service which permits a remote client to submit requests to Hive, with the help of a variety of programming languages, as well as retrieve results, is what we call HiveServer. Since, It is built on Apache Thrift, that’s why we call it Thrift server sometimes.

What is the difference between Spark thrift and Hive thrift?

I guess both do the same except when you start Hive Thrift server from spark, it adds one more CLI service to the thrift server which should add spark SQL context to the thrift API.

What is spark thrift server?

Introduction to Spark Thrift Server Spark SQL Thrift server is a port of Apache Hive’s HiverServer2 which allows the clients of JDBC or ODBC to execute queries of SQL over their respective protocols on Spark.


1 Answers

  1. HiveServer2 (HS2) is a service that enables clients to execute queries against Hive. HiveServer2 is the successor to HiveServer1 which has been deprecated. HS2 supports multi-client concurrency and authentication. It is designed to provide better support for open API clients like JDBC and ODBC. You can find more details about hiveserver at https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Overview
  2. Hive Service is nothing but daemon which runs on your client node which sends requests to Hive Server.

  3. Thrift is an RPC framework for building cross-platform services. Its stack consists of 4 layers: Server, Transport, Protocol, and Processor. You can find more details about the layers at https://thrift.apache.org/docs/concepts.

Relation between all these:

  • The Thrift-based Hive service is the core of HS2 and responsible for servicing the Hive queries (e.g., from Beeline). In simple terms Hive server is based on thrift protocols which sends queries from hive client i.e., your command line interface or from HUE interface to the underlying data which can be in your HDFS or any other data sources.

Usage:

  • When you query any hive tables or database, in background automatically your requests is transferred between hive service and hive server
  • when you want to create your own service or project you can use thrift protocols which will help you in creating layers, think this as you are creating your user defined functions using libraries, so in that case libraries will be thrift.

What is Apache Thrift: It is framework for scalable for cross-language service development.

When we can use Apache Thrift: Developing web-service that uses service developed in one language access that is in another language.

What is HiveServer : It is a service that allows a remote client to submit requests to hive. Using a variety of programming languages, and retrieve results.

like image 189
Rijul Avatar answered Oct 08 '22 09:10

Rijul