Query failed: java.net.UnknownHostException: hadoop about docker-hive

[mcapitanio]

Hi @Filippo89 , you can run:

docker exec -it hive bash

if you you the latest branch

[mcapitanio]

@mikestnx
Hi!
If you look at the docker.compose.yml in Hive you'll notice that the hive service is linked to zookeeper, hadoop and postgred (for the Hive metadata) and all them are on the same network. This is enough to say docker that the "hive" service can contact the "hadoop" service using the hostname "hadoop". The hostname is equals to the service name in that case.

I don't know how would be in Kitematic, but if you just start hive with docker-compose you should get all services up and running and you would be able to use presto upon them.

[mikestnx]

Hi, Yes I am running with docker-compose so KiteMatic is really just a UI to the regular Mac docker. Everything else is working it is just this hostname that is not resolvable it seems. I’ll keep trying. If docker-compose allows me to hardcode IPs I might be able to get it working like that. Cheers, Mike.

…

On Jun 10, 2017, at 12:21 PM, Matteo Capitanio ***@***.***> wrote: @mikestnx <https://github.com/mikestnx> Hi! If you look at the docker.compose.yml in Hive you'll notice that the hive service is linked to zookeeper, hadoop and postgred (for the Hive metadata) and all them are on the same network. This is enough to say docker that the "hive" service can contact the "hadoop" service using the hostname "hadoop". The hostname is equals to the service name in that case. I don't know how would be in Kitematic, but if you just start hive with docker-compose you should get all services up and running and you would be able to use presto upon them. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ARk3GgSMrUfRjgLxFIxI12gfzBNEmnxEks5sCuzFgaJpZM4N1gHF>.

[mcapitanio]

Hi Mike,

to have Hive up and running you have to get also the services "hadoop", "zookeeper" and "postgres" (for the metastore) up and running.
The following docker-compose.yml should do the thing, because it sets th.e links dependecies and Docker do it all what is needed to define the hostaneme and set them in the dns for the "hadoop" network. I have written it without testing, but it should work.. .;)

version: '2'
services:
  postgres:
    image: mcapitanio/postgres:9.6
    networks:
    - hadoop
    ports:
    - 5432:5432
    environment: POSTGRES_PASSWORD:postgres
  zookeeper:
    image: mcapitanio/zookeeper:latest
    ports:
    - 2181:2181
    - 2888:2888
    - 3888:3888
    - 8086:8080
    networks:
    - hadoop
  hadoop:
    image: mcapitanio/hadoop:latest
    networks:
    - hadoop
    links:
    - zookeeper
    ports:
    - 50070:50070
    - 50075:50075
    - 8020:8020
    - 8042:8042
    - 8088:8088
    - 8188:8188
    - 19888:19888
  hive:
    image: mcapitanio/hive:latest
    networks:
    - hadoop
    links:
    - hadoop
    - zookeeper
    - postgres
    environment:
      PGPASSWORD: postgres
    ports:
    - 10000:10000
    - 10001:10001
    - 10002:10002
    - 10003:10003
    - 9083:9083
    - 50111:50111
    - 9999:9999
networks:
  hadoop:
    driver: bridge

[mikestnx]

Thanks for the compose file! I can start these (as I can with the old file). I still get the same error about the hostname. The funny thing is if I exec into all four containers all the hostnames resolve (below)

…

----- sh-4.2# ping hive PING hive (172.19.0.5) 56(84) bytes of data. 64 bytes from a662ef8d6813 (172.19.0.5): icmp_seq=1 ttl=64 time=0.028 ms sh-4.2# ping hadoop PING hadoop (172.19.0.4) 56(84) bytes of data. 64 bytes from dockerhivelatest_hadoop_1.dockerhivelatest_hadoop (172.19.0.4): icmp_seq=1 ttl=64 time=0.121 ms sh-4.2# ping postgres PING postgres (172.19.0.2) 56(84) bytes of data. 64 bytes from dockerhivelatest_postgres_1.dockerhivelatest_hadoop (172.19.0.2): icmp_seq=1 ttl=64 time=0.056 ms ---- So not sure what/where it is not being able to resolve. I think I’ll just try running non-docker versions of these other services. Thanks, Mike.

On Jun 12, 2017, at 11:53 PM, Matteo Capitanio ***@***.***> wrote: Hi Mike, to have Hive up and running you have to get also the services "hadoop", "zookeeper" and "postgres" (for the metastore) up and running. The following docker-compose.yml should do the thing, because it sets th.e links dependecies and Docker do it all what is needed to define the hostaneme and set them in the dns for the "hadoop" network. I have written it without testing, but it should work.. .;) version: '2' services: postgres: image: mcapitanio/postgres:9.6 networks: - hadoop ports: - 5432:5432 environment: POSTGRES_PASSWORD:postgres zookeeper: image: mcapitanio/zookeeper:latest ports: - 2181:2181 - 2888:2888 - 3888:3888 - 8086:8080 networks: - hadoop hadoop: image: mcapitanio/hadoop:latest networks: - hadoop links: - zookeeper ports: - 50070:50070 - 50075:50075 - 8020:8020 - 8042:8042 - 8088:8088 - 8188:8188 - 19888:19888 hive: image: mcapitanio/hive:latest networks: - hadoop links: - hadoop - zookeeper - postgres environment: PGPASSWORD: postgres ports: - 10000:10000 - 10001:10001 - 10002:10002 - 10003:10003 - 9083:9083 - 50111:50111 - 9999:9999 networks: hadoop: driver: bridge — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ARk3GtK0Kpm39SZ1DoSRHxR_BjMUxVWLks5sDjHPgaJpZM4N1gHF>.

[mcapitanio]

@mikestnx Sorry not having answered you. Did you solve the problem?

[Filippo89]

@mcapitanio Hi, there is a way after you compose-up the service to access it via bash? For example I would like to use the Hive beeline to create tables.

[Filippo89]

Yes, thank you a lot! I had a problem with Hadoop then but using the new docker-compose.yml you post in this discussion solved the issue also for me.