内网下已经搭建好Kubernetes集群，由两台CentOS组成。

192.168.1.11 Master
192.168.1.12 Node

由于之前搭建时操作的都是CentOS系统中的安装与配置，与Ubuntu下还是有些区别。这里主要记录下将Ubuntu系统的主机加入到Kubernetes集群中的操作步骤。

待操作的主机基本情况

Ubuntu 版本: 16.04
IP: 192.168.1.13

因为在kubernetes节点上需要运行kubelet、kube-proxy、docker和flannel服务并进行相关配置才能正常加入到集群中，所以先一个服务一个服务来安装。

kubelet和kube-proxy两个都属于Kubernetes源码相关的，所先下载下来。
Client二进制下载：https://dl.k8s.io/v1.5.5/kubernetes-client-linux-amd64.tar.gz
Server二进制下载：https://dl.k8s.io/v1.5.5/kubernetes-server-linux-amd64.tar.gz

实际上只需要用到kubernetes-server-linux-amd64，kubernetes-client-linux-amd64中只有一个kubectl,这是master上需要用的，这里只是负责node节点的部署所以用不上。
解压kubernetes-server-linux-amd64.tar.gz，将/server/bin/目录下的kubelet、kube-proxy拷贝到/usr/bin/目录下。

通用配置

这部分内容是Master与Node上通用的配置操作

创建Kubernetes配置目录

1
sudo mkdir /etc/kubernetes

创建Kubernetes通用配置文件

/etc/kubernetes/config文件中，存储的是Kubernetes各组件的通用配置信息。

1
2
3
4
5
6
sudo vim /etc/kubernetes/config

KUBE_LOGTOSTDERR="--logtostderr=true"
KUBE_LOG_LEVEL="--v=0"
KUBE_ALLOW_PRIV="--allow-privileged=false"
KUBE_MASTER="--master=http://192.168.1.11:8080"

配置 Kubelet

1. 创建kubelet的数据目录

1
sudo mkdir /var/lib/kubelet

2. 创建kubelete配置文件

1
2
3
4
5
6
7
8
sudo vim /etc/kubernetes/kubelet

KUBELET_ADDRESS="--address=127.0.0.1"
KUBELET_HOSTNAME="--hostname-override=192.168.1.13"
KUBELET_API_SERVER="--api-servers=http://192.168.1.11:8080"
# pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"
KUBELET_ARGS="--enable-server=true --enable-debugging-handlers=true"

3. 创建kubelet.service文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
sudo vim /lib/systemd/system/kubelet.service

[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service

[Service]
WorkingDirectory=/var/lib/kubelet
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/kubelet
ExecStart=/usr/bin/kubelet \
        $KUBE_LOGTOSTDERR \
        $KUBE_LOG_LEVEL \
        $KUBELET_API_SERVER \
        $KUBELET_ADDRESS \
        $KUBELET_PORT \
        $KUBELET_HOSTNAME \
        $KUBE_ALLOW_PRIV \
        $KUBELET_POD_INFRA_CONTAINER \
        $KUBELET_ARGS
Restart=on-failure
KillMode=process

[Install]
WantedBy=multi-user.target

4. 启动kubelet服务

1
2
3
4
sudo systemctl daemon-reload
sudo systemctl enable kubelet
sudo systemctl start kubelet
sudo systemctl status kubelet

配置 kube-proxy

1. 创建kube-proxy配置文件

1
2
3
4
5
6
sudo vim /etc/kubernetes/proxy

# kubernetes proxy config
# default config should be adequate
# Add your own!
KUBE_PROXY_ARGS=""

2. 创建kube-proxy.service

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
sudo vim /lib/systemd/system/kube-proxy.service

[Unit]
Description=Kubernetes Proxy
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/proxy
ExecStart=/usr/bin/kube-proxy \
        $KUBE_LOGTOSTDERR \
        $KUBE_LOG_LEVEL \
        $KUBE_MASTER \
        $KUBE_PROXY_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

3. 启动kube-proxy服务

1
2
3
4
sudo systemctl daemon-reload
sudo systemctl enable kube-proxy
sudo systemctl start kube-proxy
sudo systemctl status kube-proxy

配置 Flannel

0. 下载和编译 Flannel

1
2
3
git clone -b v0.5.6 https://github.com/coreos/flannel.git
cd flannel
./build

具体的编译方法可能会不同，请参考flannel目录下的README.md文件。 

1
2
3
4
5
6
## Building flannel

* Step 1: Make sure you have required dependencies installed on your machine. On Ubuntu, run `sudo apt-get install linux-libc-dev golang gcc`.
On Fedora/Redhat, run `sudo yum install kernel-headers golang gcc`.
* Step 2: Git clone the flannel repo: `git clone https://github.com/coreos/flannel.git`
* Step 3: Run the build script: `cd flannel; ./build`

在README.md文件中有如下一段是介绍怎么编译flannel的。其中在第一步中有介绍在Ubuntu中需要sudo apt-get install linux-libc-dev golang gcc 然后就可以./build了

编译完成后有两步

  1. 将可执行文件flanneld拷贝到/usr/bin/目录。
  2. 创建/usr/bin/flannel目录，并将dist目录下的mk-docker-opts.sh文件拷贝到/usr/bin/flannel/中。

1. 创建flanneld.conf配置文件

1
2
3
4
5
6
7
8
9
10
11
12
13
sudo vim /etc/default/flanneld.conf

# Flanneld configuration options  

# etcd url location.  Point this to the server where etcd runs
FLANNEL_ETCD_ENDPOINTS="http://192.168.1.11:2379"

# etcd config key.  This is the configuration key that flannel queries
# For address range assignment
FLANNEL_ETCD_PREFIX="/coreos.com/network"

# Any additional options that you want to pass
#FLANNEL_OPTIONS=""

其中，FLANNEL_ETCD_PREFIX选项是在Master中配置的etcd网络。

2. 创建flanneld.service文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
sudo vim /lib/systemd/system/flanneld.service

[Unit]
Description=Flanneld
Documentation=https://github.com/coreos/flannel
After=network.target
After=etcd.service
Before=docker.service

[Service]
User=root
EnvironmentFile=/etc/default/flanneld.conf
ExecStart=/usr/bin/flanneld \
        -etcd-endpoints=${FLANNEL_ETCD_ENDPOINTS} \
        -etcd-prefix=${FLANNEL_ETCD_PREFIX} \
        $FLANNEL_OPTIONS
ExecStartPost=/usr/bin/flannel/mk-docker-opts.sh -k DOCKER_OPTS -d /run/flannel/docker
Restart=on-failure
Type=notify
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
RequiredBy=docker.service

3. 启动flanneld服务

1
2
3
4
sudo systemctl daemon-reload 
sudo systemctl enable flanneld
sudo systemctl start flanneld
sudo systemctl status flanneld

配置 Docker

1. 安装 Docker

1
sudo apt -y install docker.io

2. 使Flannel作用Docker网络

1
2
3
4
5
sudo mkdir /lib/systemd/system/docker.service.d
sudo vim /lib/systemd/system/docker.service.d/flannel.conf

[Service]
EnvironmentFile=-/run/flannel/docker

3. 重启docker服务

1
2
sudo systemctl daemon-reload
sudo systemctl restart docker

查看docker是否有了flannel的网络。

1
sudo ps -ef | grep docker

其他

启动各服务的统一脚本

1
2
3
4
5
6
7
for SERVICES in flanneld kube-proxy kubelet docker;
do
sudo systemctl daemon-reload
sudo systemctl restart $SERVICES
sudo systemctl enable $SERVICES
sudo systemctl status $SERVICES
done