metastore_db created wherever I run Hive

Question

Folder metastore_db is created in any directory where I run Hive query. Is there any way to have only one metastore_db in a defined location and stop it from being created all over the places? Does it have anything to do with hive.metastore.local?

Answer 1

The property of interest here is javax.jdo.option.ConnectionURL. The default value of this property is jdbc:derby:;databaseName=metastore_db;create=true. This value specifies that you will be using embedded derby as your Hive metastore and the location of the metastore is metastore_db. Also the metastore will be created if it doesn't already exist.

Note that the location of the metastore (metastore_db) is a relative path. Therefore, it gets created where you launch Hive from. If you update this property (in your hive-site.xml) to be, say an absolute path to a location, the metastore will be used from that location.

I must warn you though that embedded derby metastore can only be accessed by one user at a time. Hive uses embedded derby by default to allow an out of the box experience and for ease of testing. For any practical system, I would recommend moving to standalone "real" database like MySQL or PostgreSQL. Instructions on how to do that are available here.