recap-build / hive-metastore-standalone

Apache Hive Metastore in Standalone Mode With Docker

Hive Metastore Standalone Docker Image 🐝 🐳

This repo creates a Docker image of a completely standalone Hive Metastore suitable for running in a Github Workflow or as a Docker image without the need for compose.

Unlike other Hive Metastore Docker images, this Docker image can be run without any other containers.

• ✅ No S3
• ✅ No localstack
• ✅ No minio
• ✅ No PostgreSQL
• ✅ No Hive
• ✅ No Hadoop
• ✅ No other containers

The image uses Hive Metastore Standalone configured to run embedded Derby for the DB and the local filesystem for the FS.

NOTE: Only one process can connect to the metastore at a time because embedded Derby only supports one connection at a time. This should be fine for integration tests.

Usage

The package is published to the Github Container Registry.

docker pull ghcr.io/recap-build/hive-metastore-standalone:latest

Github Workflow

You can use the Hive Metastore for integration tests in your Github Workflows using this service block:

    services:
      hive-metastore:
        image: ghcr.io/recap-build/hive-metastore-standalone:latest
        ports:
          - 9083:9083

Logging

By default the Hive Metastore will log at INFO level. You can override this by setting the HMS_LOGLEVEL environment variable for the container.

Acknowledgements

This image is based on @naushadh's hive-metastore repo.

hive-metastore-standalone's Issues

Add MySQL Support

Add support for MySQL database. Derby is suitable for local testing but not production deployment or or load testing

Add linux/arm64/v8 support

Is it possible to generate a docker image that supports ARM support?

this is what I see trying to run on my M1.

docker pull ghcr.io/recap-build/hive-metastore-standalone:latest --platform linux/arm64

latest: Pulling from recap-build/hive-metastore-standalone
1efc276f4ff9: Pull complete
a2f2f93da482: Pull complete
1a2de4cc9431: Pull complete
9013b84ebbe7: Pull complete
13a75c41290c: Pull complete
7e16fa9c54fa: Pull complete
354596e2e1ce: Pull complete
61f6fbf5b34e: Pull complete
Digest: sha256:779bc136cbc7e0959ba19ef85a19a669b466ff228d4e5ecf977d343ac0201cb6
Status: Downloaded newer image for ghcr.io/recap-build/hive-metastore-standalone:latest
image with reference ghcr.io/recap-build/hive-metastore-standalone:latest was found but does not match the specified platform: wanted linux/arm64, actual: linux/amd64
% docker run --rm --detach --name hms-standalone
--publish 9083:9083
ghcr.io/recap-build/hive-metastore-standalone:latest
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
2464403a60a62f6c25a8e7703f84e220feaa071a360d53d0fce24e33c5091d60

Proposal for repo merge

Thank you for recommending my project in #3 (comment) -- that's how I came to know of this project and really that HMS can be run without a database at all!

I was wondering whether we can combine our projects together using Docker build args that would allow keeping 90% identical code and still releasing derby, mysql, postgres variants to Dockerhub from a single repo. We'd both gain a co-contributor, reduce overall copy-pasta/maintenance and end-users benefit as well with less fragmentation.

I've implemented CI Actions (albeit it needs work to automate secrets management for PRs) and ARM support already, so your users also gain new functionality.