1 描述

Hive MetaStore - It is a central repository that stores all the structure information of various tables and partitions in the warehouse. It also includes metadata of column and its type information, the serializers and deserializers which is used to read and write data and the corresponding HDFS files where the data is stored.

2 Hive的元数据存储(Metastore三种配置方式)

Embedded，Local，Remote

https://blog.csdn.net/epitomizelu/article/details/117091656

https://zhuanlan.zhihu.com/p/473378621

https://blog.csdn.net/qq_40990732/article/details/80914873

3 Hive元数据库介绍

https://blog.csdn.net/victorzzzz/article/details/81874674

https://cloud.tencent.com/developer/article/1785857

https://cwiki.apache.org/confluence/display/hive/design#Design-HiveArchitecture

https://zhuanlan.zhihu.com/p/87545980

https://blog.csdn.net/oTengYue/article/details/91129850

https://jiamaoxiang.top/2020/06/27/Hive%E7%9A%84%E6%9E%B6%E6%9E%84%E5%89%96%E6%9E%90/

https://www.javatpoint.com/hive-architecture

Hive Client

Hive allows writing applications in various languages, including Java, Python, and C++. It supports different types of clients such as:-

• Thrift Server - It is a cross-language service provider platform that serves the request from all those programming languages that supports Thrift.
• JDBC Driver - It is used to establish a connection between hive and Java applications. The JDBC Driver is present in the class org.apache.hadoop.hive.jdbc.HiveDriver.
• ODBC Driver - It allows the applications that support the ODBC protocol to connect to Hive.

Hive Services

The following are the services provided by Hive:-

• Hive CLI - The Hive CLI (Command Line Interface) is a shell where we can execute Hive queries and commands.
• Hive Web User Interface - The Hive Web UI is just an alternative of Hive CLI. It provides a web-based GUI for executing Hive queries and commands.
• Hive MetaStore - It is a central repository that stores all the structure information of various tables and partitions in the warehouse. It also includes metadata of column and its type information, the serializers and deserializers which is used to read and write data and the corresponding HDFS files where the data is stored.
• Hive Server - It is referred to as Apache Thrift Server. It accepts the request from different clients and provides it to Hive Driver.
• Hive Driver - It receives queries from different sources like web UI, CLI, Thrift, and JDBC/ODBC driver. It transfers the queries to the compiler.
• Hive Compiler - The purpose of the compiler is to parse the query and perform semantic analysis on the different query blocks and expressions. It converts HiveQL statements into MapReduce jobs.
• Hive Execution Engine - Optimizer generates the logical plan in the form of DAG of map-reduce tasks and HDFS tasks. In the end, the execution engine executes the incoming tasks in the order of their dependencies.

计算引擎

Hive支持MapReduce、Tez、Spark

https://cloud.tencent.com/developer/article/1893808

https://blog.csdn.net/kwu_ganymede/article/details/52223133

数据存储

https://cloud.tencent.com/developer/article/1411821

Hive是基于hdfs的，它的数据存储在Hadoop分布式文件系统中。Hive本身是没有专门的数据存储格式，也没有为数据建立索引，只需要在创建表的时候告诉Hive数据中的列分隔符和行分隔符，Hive就可以解析数据。

default数据库中的表的存储位置 /user/hive/warehouse
其他数据库的表自己指定

常见问题

1.FAILED: SemanticException Failed to get a spark session: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create Spark client for Spark session

https://blog.csdn.net/qq_41504585/article/details/108064512

启动metastore

https://blog.csdn.net/u010670689/article/details/41576647

hive —service metastore 2>&1 >> /var/log.log &

启动hiveserver2

hiveserver2

连接方式

https://blog.csdn.net/qq_41851454/article/details/79833306

https://www.shuzhiduo.com/A/RnJW4Z2r5q/

1.cli

hive

2.beeline

https://www.jianshu.com/p/97bbe79d88d2