Docker 三剑客之 Docker Swarm（基于 overlay 组网通信）-白红宇

Docker 三剑客之 Docker Swarm（基于 overlay 组网通信）

阅读量：5930 次

发布时间：2019-06-19

本文共 9582 字，大约阅读时间需要 31 分钟。

这一篇主要是对 Docker Swarm 的完善，增加基于 overlay 组网通信，以便 Docker 容器可以跨主机访问。

不同主机间的容器之间通信方式，大概有三种：

使用端口映射：直接把容器的服务端口映射到主机上，主机直接通过映射出来的端口通信。

把容器放到主机所在的网段：修改 docker 的 ip 分配网段和主机一致，还要修改主机的网络结构。

第三方项目：flannel，weave 或者 pipework 等，这些方案一般都是通过 SDN 搭建 overlay 网络达到容器通信的。

在使用 overlay 组网通信之前，我们先安装 Docker，以及 Docker Machine（Linux 下）：

$ sudo curl -L https://github.com/docker/machine/releases/download/v0.13.0/docker-machine-`uname -s`-`uname -m` > /usr/local/bin/docker-machine$ sudo chmod +x /usr/local/bin/docker-machine

使用脚本一键安装（Docker 镜像加速地址可以更换）：

#!/bin/bashset -ecreate_kv() {    echo Creating kvstore machine.    docker-machine create -d virtualbox \        --engine-opt="registry-mirror=https://kvo9moak.mirror.aliyuncs.com" \        kvstore    docker $(docker-machine config kvstore) run -d \        -p "8500:8500" \        progrium/consul --server -bootstrap-expect 1}create_master() {    echo Creating cluster master    kvip=$(docker-machine ip kvstore)    docker-machine create -d virtualbox \        --swarm --swarm-master \        --swarm-discovery="consul://${kvip}:8500" \        --engine-opt="cluster-store=consul://${kvip}:8500" \        --engine-opt="cluster-advertise=eth1:2376" \        --engine-opt="registry-mirror=https://kvo9moak.mirror.aliyuncs.com" \        swarm-manager}create_nodes(){    kvip=$(docker-machine ip kvstore)    echo Creating cluster nodes    for i in 1 2; do        docker-machine create -d virtualbox \            --swarm \            --swarm-discovery="consul://${kvip}:8500" \            --engine-opt="cluster-store=consul://${kvip}:8500" \            --engine-opt="cluster-advertise=eth1:2376" \            --engine-opt="registry-mirror=https://kvo9moak.mirror.aliyuncs.com" \            swarm-node${i}    done}teardown(){    docker-machine rm kvstore -y    docker-machine rm -y swarm-manager    for i in 1 2; do        docker-machine rm -y swarm-node${i}    done}case $1 in    up)        create_kv        create_master        create_nodes        ;;    down)        teardown        ;;    *)        echo "Unknow command..."        exit 1        ;;esac

运行./cluster.sh up，就能自动生成四台主机：

一台 kvstore 运行 consul 服务。

一台 swarm master 机器，运行 swarm manager 服务。

两台 swarm node 机器，都是运行了 swarm node 服务和 docker daemon 服务。

查看四台主机的具体信息：

$ docker-machine lsNAME            ACTIVE      DRIVER       STATE     URL                         SWARM                    DOCKER        ERRORSkvstore         -           virtualbox   Running   tcp://192.168.99.100:2376                            v18.03.1-ceswarm-manager   * (swarm)   virtualbox   Running   tcp://192.168.99.101:2376   swarm-manager (master)   v18.03.1-ceswarm-node1     -           virtualbox   Running   tcp://192.168.99.102:2376   swarm-manager            v18.03.1-ceswarm-node2     -           virtualbox   Running   tcp://192.168.99.103:2376   swarm-manager            v18.03.1-ce

接下来验证集群是否正确安装？在主机上运行下面命令（主机，不是 Docker 主机）：

$ eval $(docker-machine env --swarm swarm-manager)$ docker infoContainers: 6 Running: 6 Paused: 0 Stopped: 0Images: 5Server Version: swarm/1.2.8Role: primaryStrategy: spreadFilters: health, port, containerslots, dependency, affinity, constraint, whitelistNodes: 3 swarm-manager: 192.168.99.101:2376  └ ID: K6WX:ZYFT:UEHA:KM66:BYHD:ROBF:Z5KG:UHNE:U37V:4KX2:S5SV:YSCA|192.168.99.101:2376  └ Status: Healthy  └ Containers: 2 (2 Running, 0 Paused, 0 Stopped)  └ Reserved CPUs: 0 / 1  └ Reserved Memory: 0 B / 1.021 GiB  └ Labels: kernelversion=4.9.93-boot2docker, operatingsystem=Boot2Docker 18.03.1-ce (TCL 8.2.1); HEAD : cb77972 - Thu Apr 26 16:40:36 UTC 2018, ostype=linux, provider=virtualbox, storagedriver=aufs  └ UpdatedAt: 2018-05-08T10:20:39Z  └ ServerVersion: 18.03.1-ce swarm-node1: 192.168.99.102:2376  └ ID: RPRC:AVBX:7CBJ:HUTI:HI3B:RI6B:QI6O:M2UQ:ZT2I:HZ6J:33BL:HY76|192.168.99.102:2376  └ Status: Healthy  └ Containers: 2 (2 Running, 0 Paused, 0 Stopped)  └ Reserved CPUs: 0 / 1  └ Reserved Memory: 0 B / 1.021 GiB  └ Labels: kernelversion=4.9.93-boot2docker, operatingsystem=Boot2Docker 18.03.1-ce (TCL 8.2.1); HEAD : cb77972 - Thu Apr 26 16:40:36 UTC 2018, ostype=linux, provider=virtualbox, storagedriver=aufs  └ UpdatedAt: 2018-05-08T10:21:09Z  └ ServerVersion: 18.03.1-ce swarm-node2: 192.168.99.103:2376  └ ID: MKQ2:Y7EO:CKOJ:MGFH:B77S:3EWX:7YJT:2MBQ:CJSN:XENJ:BSKO:RAZP|192.168.99.103:2376  └ Status: Healthy  └ Containers: 2 (2 Running, 0 Paused, 0 Stopped)  └ Reserved CPUs: 0 / 1  └ Reserved Memory: 0 B / 1.021 GiB  └ Labels: kernelversion=4.9.93-boot2docker, operatingsystem=Boot2Docker 18.03.1-ce (TCL 8.2.1); HEAD : cb77972 - Thu Apr 26 16:40:36 UTC 2018, ostype=linux, provider=virtualbox, storagedriver=aufs  └ UpdatedAt: 2018-05-08T10:21:06Z  └ ServerVersion: 18.03.1-cePlugins: Volume: Network: Log:Swarm: NodeID: Is Manager: false Node Address:Kernel Version: 4.9.93-boot2dockerOperating System: linuxArchitecture: amd64CPUs: 3Total Memory: 3.063GiBName: 85be09a37044Docker Root Dir:Debug Mode (client): falseDebug Mode (server): falseExperimental: falseLive Restore Enabled: falseWARNING: No kernel memory limit support

可以看到集群的具体信息。

然后，接下来在主机上创建 overlay 网络，创建命令：

$ docker network create -d overlay net1d6a8a22298485a044b19fcbb62033ff1b4c3d4bd6a8a2229848

然后我们查看刚创建名称为net1的 overlay 网络，命令：

$ docker network lsNETWORK ID          NAME                          DRIVER              SCOPEd6a8a2229848        net1                          overlay             global9c7f0e793838        swarm-manager/bridge          bridge              local93787d9ba7ed        swarm-manager/host            host                local72fd1e63889e        swarm-manager/none            null                localc73e00c4c76c        swarm-node1/bridge            bridge              local95983d8f1ef1        swarm-node1/docker_gwbridge   bridge              locala8a569d55cc9        swarm-node1/host              host                locale7fa8403b226        swarm-node1/none              null                local7f1d219b5c08        swarm-node2/bridge            bridge              locale7463ae8c579        swarm-node2/docker_gwbridge   bridge              local9a1f0d2bbdf5        swarm-node2/host              host                localbea626348d6d        swarm-node2/none              null                local

接下来，我们创建两个容器（主机上执行，使用 Docker Swarm 很方便），测试使用net1网络，是否可以相互访问，创建命令：

$ docker run -d --net=net1 --name=c1 -e constraint:node==swarm-node1 busybox topdab080b33e76af0e4c71c9365a6e57b2191b7aacd4f715ca11481403eccce12a$ docker run -d --net=net1 --name=c2 -e constraint:node==swarm-node2 busybox top583fde42182a7e8f27527d5c55163a32102dba566ebe1f13d1951ac214849c8d

查看刚创建的容器运行情况：

$ docker psCONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES583fde42182a        busybox             "top"               3 seconds ago       Up 2 seconds                            swarm-node2/c2dab080b33e76        busybox             "top"               18 seconds ago      Up 18 seconds                           swarm-node1/c1

接下来，我们查看net1网络的具体详情：

$ docker network inspect net1[    {        "Name": "net1",        "Id": "d6a8a2229848d40ce446f8f850a0e713a6c88a9b43583cc463f437f306724f28",        "Created": "2018-05-08T09:21:42.408840755Z",        "Scope": "global",        "Driver": "overlay",        "EnableIPv6": false,        "IPAM": {            "Driver": "default",            "Options": {},            "Config": [                {                    "Subnet": "10.0.0.0/24",                    "Gateway": "10.0.0.1"                }            ]        },        "Internal": false,        "Attachable": false,        "Ingress": false,        "ConfigFrom": {            "Network": ""        },        "ConfigOnly": false,        "Containers": {            "583fde42182a7e8f27527d5c55163a32102dba566ebe1f13d1951ac214849c8d": {                "Name": "c2",                "EndpointID": "b7fcb0039ab21ff06b36ef9ba008c324fabf24badfe45dfa6a30db6763716962",                "MacAddress": "",                "IPv4Address": "10.0.0.3/24",                "IPv6Address": ""            },            "dab080b33e76af0e4c71c9365a6e57b2191b7aacd4f715ca11481403eccce12a": {                "Name": "c1",                "EndpointID": "8a80a83230edfdd9921357f08130fa19ef0b917bc4426aa49cb8083af9edb7f6",                "MacAddress": "",                "IPv4Address": "10.0.0.2/24",                "IPv6Address": ""            }        },        "Options": {},        "Labels": {}    }]

可以看到，我们刚刚创建的两个容器信息（包含 IP 地址），也在里面。

然后我们测试下两个容器是否可以相互访问（直接 ping 容器名称，也可以访问）：

$ docker exec c1 ping -c 3 10.0.0.3PING 10.0.0.3 (10.0.0.3): 56 data bytes64 bytes from 10.0.0.3: seq=0 ttl=64 time=0.903 ms64 bytes from 10.0.0.3: seq=1 ttl=64 time=0.668 ms64 bytes from 10.0.0.3: seq=2 ttl=64 time=0.754 ms--- 10.0.0.3 ping statistics ---3 packets transmitted, 3 packets received, 0% packet lossround-trip min/avg/max = 0.668/0.775/0.903 ms$ docker exec c2 ping -c 3 10.0.0.2PING 10.0.0.2 (10.0.0.2): 56 data bytes64 bytes from 10.0.0.2: seq=0 ttl=64 time=0.827 ms64 bytes from 10.0.0.2: seq=1 ttl=64 time=0.702 ms64 bytes from 10.0.0.2: seq=2 ttl=64 time=0.676 ms--- 10.0.0.2 ping statistics ---3 packets transmitted, 3 packets received, 0% packet lossround-trip min/avg/max = 0.676/0.735/0.827 ms$ docker exec c2 ping -c 3 c1PING c1 (10.0.0.2): 56 data bytes64 bytes from 10.0.0.2: seq=0 ttl=64 time=1.358 ms64 bytes from 10.0.0.2: seq=1 ttl=64 time=0.663 ms64 bytes from 10.0.0.2: seq=2 ttl=64 time=0.761 ms--- c1 ping statistics ---3 packets transmitted, 3 packets received, 0% packet lossround-trip min/avg/max = 0.663/0.927/1.358 ms

另附 Docker Swarm 架构图：