I have a 3-node hadoop and started 3 docker containers. As we all know, the ports served in the docker container can be mapped to the ports of the host. I use -P to map all the ports of the container to the host ports. The command is as follows:
docker run -it -v /Users/yc/hadoop/docker:/test -P -h h1 --name h1 sequenceiq/hadoop-docker /etc/bootstrap.sh -bash
This is the startup script of the master node, mapping a host directory, mapping all container ports to the host, and setting the hostname of the container
When I look at the process
The results are as follows: I did the following beautification:
b96b897c6532 sequenceiq/hadoop-docker "/etc/bootstrap.sh -b" 24 minutes ago Up 24 minutes 0.0.0.0:32797->2122/tcp, 0.0.0.0:32796->8030/tcp, 0.0.0.0:32795->8031/tcp, 0.0.0.0:32794->8032/tcp, 0.0.0.0:32793->8033/tcp, 0.0.0.0:32792->8040/tcp, 0.0.0.0:32791->8042/tcp, 0.0.0.0:32790->8088/tcp, 0.0.0.0:32789->19888/tcp, 0.0.0.0:32788->49707/tcp, 0.0.0.0:32787->50010/tcp, 0.0.0.0:32786->50020/tcp, 0.0.0.0:32785->50070/tcp, 0.0.0.0:32784->50075/tcp, 0.0.0.0:32783->50090/tcp h1
Our commonly used 50070 and 8088 are mapped out, but 9000 is not. I know 9000 is hdfs, which may not belong to tcp, so how to access it in my host
I want to use IDEA to develop the java program of hadoop. The input and output directories have to be hdfs:ip:9000/ directory. I don’t know how to access the port of docker container.
The ports lacking HDFS ipc are
sequenceiq/hadoop-dockerA bug in the mirrorsequenceiq/hadoop-docker#48, although has been repaired (see:https://github.com/sequenceiq …However, its maintainers have not built a new version of the mirror, which causes you to still have problems using it now.
The solution is very simple, in
docker runAdd the following parameters manually
By the way, although Hadoop does mention it
sequenceiq/hadoop-dockerThis docker image, but from it
DockerfileThe author’s understanding of Docker is superficial and does not follow Docker’s official guidelines.
DockerfileBest practicesGuidelines, mirror image is very bloated, the production is not good. If you have the ability, you should follow the official best practice guidelines to customize the image you need.