开发 on F嘉阳

ArgoCD落地GitOps

Sat, 23 Aug 2025 22:49:50 +0800

背景

在过去的DevOps实践中，跨环境的基础设施部署和迁移经常存在遗漏，导致环境差异，部署一个新环境往往需要几天。传统CICD流水线采用“推”模式，将变更推送到不同的环境，推送后也不对目标环境状态进行管理和监控，容易导致推送后部署失败。随着云原生容器化已经成为行业事实标准，GitOps成为一种更可靠的应用部署和基础设置运维的范式。

技术选型

ArgoCD遵循GitOps的核心原则，已经成为落地GitOps的默认技术选型，因此此处不探讨其他选型方案。

原理

Git 作为唯一事实来源（GitOps思想）
自动化和持续同步：Argo CD会持续监控 Git 仓库的状态，并自动将集群的实际状态与 Git 中声明的目标状态进行同步。
- 利用K8s的Kustomize能力完成对集群apply操作

实践

环境

安装ArgoCD

$ kubectl create namespace argocd
$ kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

检查安装结果

查看argo cd登录密码

kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d; echo

设计代码库结构

落地ArgoCD通常有以下几种仓库设计范式

单仓库模式
多仓库模式
App of Apps模式

本次采用App of Apps的单仓库模式，仓库结构如下

.
├── apps
│ ├── app-one
│ ├── app-two
├── cluster-infra
│ ├── argocd
│ │ ├── base
│ │ └── overlays
│ ├── ingress
│ │ ├── base
│ │ └── overlays
│ │ └── dev
│ └── prometheus-stack
│ ├── base
│ └── overlays
│ └── dev
└── environments
 └── dev
 └── root.yaml

该结构特点如下

基础设施和应用部署分离（apps和cluster-infra）
环境基础配置和环境定制配置分离（base和overlays）
环境根节点配置（environments控制git仓库配置、应用apps注册）

本次将该代码库存放到阿里云代码仓库中。

环境配置迁移

目前的k3s环境中已经手动运维安装了argocd，现在需要将已安装的配置迁移到git库中

将官方安装配置https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml写入到cluster-infra/argocd/base/argocd.yaml中

配置``cluster-infra/argocd/base/kustomization.yaml`文件，注册同目录的安装文件

resources:
 - argocd.yaml

本次依赖阿里云代码仓库，通过ssh连接，需要覆盖默认的argocd配置，编写cluster-infra/argocd/overlays/with-known-hosts/ssh-known-hosts.patch.yaml

apiVersion: v1 
kind: ConfigMap 
metadata: 
 name: argocd-ssh-known-hosts-cm 
data: 
 ssh_known_hosts: |- 
 # BEGIN >>> 自定义条目（全局生效） 
 codeup.aliyun.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD6WbValczSDKC4ktjRzDNC1qINDAViw/oGA8ud7qH5F9XFZ4QAWhZmFBsp+cppBIOM63exjc4O9vbzMrd7AUItXigymGfTqrGt5VykIiaNhjttxSwbZf3N9eDcAvX4gGm3fcm6D1ACvPxvxMsCwOpdz5tDHqDxGOLPAXN+FSAvjdcN4uMz0MXghGh9Q3zSpy8z1TLkAKT2QBnr4q2tsdZKwWON8bBHIPwP3RSAdSUL6bAyotyncMDAvG53kwRKJ3wJ5ZezXqZNNV7EzHds+pvwThtuWtWsBuuRpL/oZiviWlXy6hILoWbkRfgONPwT0faaOn1GLzjHLuqHHfALGOen 
 # END <<< 自定义条目 
 
 # This file was automatically generated by hack/update-ssh-known-hosts.sh. DO NOT EDIT 
 [ssh.github.com]:443 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSENjQEezOmxkZMy7opKgwFB9nkt5YRrYMjNuG5N87uRgg6CLrbo5wAdT/y6v0mKV0U2w0WZ2YB/++Tpockg= 
 [ssh.github.com]:443 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl 
 [ssh.github.com]:443 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj7ndNxQowgcQnjshcLrqPEiiphnt+VTTvDP6mHBL9j1aNUkY4Ue1gvwnGLVlOhGeYrnZaMgRK6+PKCUXaDbC7qtbW8gIkhL7aGCsOr/C56SJMy/BCZfxd1nWzAOxSDPgVsmerOBYfNqltV9/hWCqBywINIR+5dIg6JTJ72pcEpEjcYgXkE2YEFXV1JHnsKgbLWNlhScqb2UmyRkQyytRLtL+38TGxkxCflmO+5Z8CSSNY7GidjMIZ7Q4zMjA2n1nGrlTDkzwDCsw+wqFPGQA179cnfGWOWRVruj16z6XyvxvjJwbz0wQZ75XK5tKSb7FNyeIEs4TT4jk+S4dhPeAUC5y+bDYirYgM4GC7uEnztnZyaVWQ7B381AK4Qdrwt51ZqExKbQpTUNn+EjqoTwvqNj4kqx5QUCI0ThS/YkOxJCXmPUWZbhjpCg56i+2aB6CmK2JGhn57K5mj0MNdBXA4/WnwH6XoPWJzK5Nyu2zB3nAZp+S5hpQs+p1vN1/wsjk= 
 bitbucket.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPIQmuzMBuKdWeF4+a2sjSSpBK0iqitSQ+5BM9KhpexuGt20JpTVM7u5BDZngncgrqDMbWdxMWWOGtZ9UgbqgZE= 
 bitbucket.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIazEu89wgQZ4bqs3d63QSMzYVa0MuJ2e2gKTKqu+UUO 
 bitbucket.org ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQeJzhupRu0u0cdegZIa8e86EG2qOCsIsD1Xw0xSeiPDlCr7kq97NLmMbpKTX6Esc30NuoqEEHCuc7yWtwp8dI76EEEB1VqY9QJq6vk+aySyboD5QF61I/1WeTwu+deCbgKMGbUijeXhtfbxSxm6JwGrXrhBdofTsbKRUsrN1WoNgUa8uqN1Vx6WAJw1JHPhglEGGHea6QICwJOAr/6mrui/oB7pkaWKHj3z7d1IC4KWLtY47elvjbaTlkN04Kc/5LFEirorGYVbt15kAUlqGM65pk6ZBxtaO3+30LVlORZkxOh+LKL/BvbZ/iRNhItLqNyieoQj/uh/7Iv4uyH/cV/0b4WDSd3DptigWq84lJubb9t/DnZlrJazxyDCulTmKdOR7vs9gMTo+uoIrPSb8ScTtvw65+odKAlBj59dhnVp9zd7QUojOpXlL62Aw56U4oO+FALuevvMjiWeavKhJqlR7i5n9srYcrNV7ttmDw7kf/97P5zauIhxcjX+xHv4M= 
 github.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSENjQEezOmxkZMy7opKgwFB9nkt5YRrYMjNuG5N87uRgg6CLrbo5wAdT/y6v0mKV0U2w0WZ2YB/++Tpockg= 
 github.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl 
 github.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj7ndNxQowgcQnjshcLrqPEiiphnt+VTTvDP6mHBL9j1aNUkY4Ue1gvwnGLVlOhGeYrnZaMgRK6+PKCUXaDbC7qtbW8gIkhL7aGCsOr/C56SJMy/BCZfxd1nWzAOxSDPgVsmerOBYfNqltV9/hWCqBywINIR+5dIg6JTJ72pcEpEjcYgXkE2YEFXV1JHnsKgbLWNlhScqb2UmyRkQyytRLtL+38TGxkxCflmO+5Z8CSSNY7GidjMIZ7Q4zMjA2n1nGrlTDkzwDCsw+wqFPGQA179cnfGWOWRVruj16z6XyvxvjJwbz0wQZ75XK5tKSb7FNyeIEs4TT4jk+S4dhPeAUC5y+bDYirYgM4GC7uEnztnZyaVWQ7B381AK4Qdrwt51ZqExKbQpTUNn+EjqoTwvqNj4kqx5QUCI0ThS/YkOxJCXmPUWZbhjpCg56i+2aB6CmK2JGhn57K5mj0MNdBXA4/WnwH6XoPWJzK5Nyu2zB3nAZp+S5hpQs+p1vN1/wsjk= 
 gitlab.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFSMqzJeV9rUzU4kWitGjeR4PWSa29SPqJ1fVkhtj3Hw9xjLVXVYrU9QlYWrOLXBpQ6KWjbjTDTdDkoohFzgbEY= 
 gitlab.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAfuCHKVTjquxvt6CM6tdG4SLp1Btn/nOeHHE5UOzRdf 
 gitlab.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsj2bNKTBSpIYDEGk9KxsGh3mySTRgMtXL583qmBpzeQ+jqCMRgBqB98u3z++J1sKlXHWfM9dyhSevkMwSbhoR8XIq/U0tCNyokEi/ueaBMCvbcTHhO7FcwzY92WK4Yt0aGROY5qX2UKSeOvuP4D6TPqKF1onrSzH9bx9XUf2lEdWT/ia1NEKjunUqu1xOB/StKDHMoX4/OKyIzuS0q/T1zOATthvasJFoPrAjkohTyaDUz2LN5JoH839hViyEG82yB+MjcFV5MU3N1l1QL3cVUCh93xSaua1N85qivl+siMkPGbO5xR/En4iEY6K2XPASUEMaieWVNTRCtJ4S8H+9 
 ssh.dev.azure.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7Hr1oTWqNqOlzGJOfGJ4NakVyIzf1rXYd4d7wo6jBlkLvCA4odBlL0mDUyZ0/QUfTTqeu+tm22gOsv+VrVTMk6vwRU75gY/y9ut5Mb3bR5BV58dKXyq9A9UeB5Cakehn5Zgm6x1mKoVyf+FFn26iYqXJRgzIZZcZ5V6hrE0Qg39kZm4az48o0AUbf6Sp4SLdvnuMa2sVNwHBboS7EJkm57XQPVU3/QpyNLHbWDdzwtrlS+ez30S3AdYhLKEOxAG8weOnyrtLJAUen9mTkol8oII1edf7mWWbWVf0nBmly21+nZcmCTISQBtdcyPaEno7fFQMDD26/s0lfKob4Kw8H 
 vs-ssh.visualstudio.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7Hr1oTWqNqOlzGJOfGJ4NakVyIzf1rXYd4d7wo6jBlkLvCA4odBlL0mDUyZ0/QUfTTqeu+tm22gOsv+VrVTMk6vwRU75gY/y9ut5Mb3bR5BV58dKXyq9A9UeB5Cakehn5Zgm6x1mKoVyf+FFn26iYqXJRgzIZZcZ5V6hrE0Qg39kZm4az48o0AUbf6Sp4SLdvnuMa2sVNwHBboS7EJkm57XQPVU3/QpyNLHbWDdzwtrlS+ez30S3AdYhLKEOxAG8weOnyrtLJAUen9mTkol8oII1edf7mWWbWVf0nBmly21+nZcmCTISQBtdcyPaEno7fFQMDD26/s0lfKob4Kw8H

配置``cluster-infra/argocd/overlays/with-known-hosts/kustomization.yaml`文件，注册hosts patch文件

resources: 
 - ../../base 
 
patchesStrategicMerge: 
 - ssh-known-hosts.patch.yaml

按这样配置后，argocd会先执行base的安装文件apply基础环境，再使用ssh-known-hosts.patch.yaml覆盖ssh的配置，保证环境与git配置一致。

接下来，继续在apps根目录注册应用，配置environments/dev/kustomization.yaml

resources:
 - ../../cluster-infra/argocd/overlays/with-known-hosts

编写根目录git配置

apiVersion: argoproj.io/v1alpha1 
kind: Application 
metadata: 
 name: dev-root 
 namespace: argocd 
spec: 
 project: default 
 source: 
 repoURL: git@codeup.aliyun.com:xxxx/fjy8018/pve-k3s-gitops.git 
 targetRevision: master 
 path: environments/dev 
 kustomize: 
 enableHelm: true 
 destination: 
 server: https://kubernetes.default.svc 
 namespace: argocd 
 syncPolicy: 
 automated: 
 prune: true 
 selfHeal: true

到此，所有配置迁移完成，将git仓库提交到远程后，在master节点添加私钥

argocd repo add git@codeup.aliyun.com:xxxx/fjy8018/pve-k3s-gitops.git --ssh-private-key-path /root/.ssh/gitops_ed25519

同步dev根应用

kubectl apply -f dev-root.yaml

本地使用kubectl暴露临时端口，再使用argocd cli连接集群查看状态

kubectl port-forward svc/argocd-server -n argocd --address 0.0.0.0 30443:443
argocd login xxxx:30443 --username admin
argocd app list

部署ingress

一直使用临时端口+命令行查看应用状态也不方便，官方提供webui界面查看部署状态，考虑使用ingress暴露固定域名访问

ingress的部署也遵循相同步骤

配置官方安装文件
配置环境个性化
1. 端口暴露
根应用添加配置

将官方https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/cloud/deploy.yaml安装文件配置到cluster-infra/ingress/base/ingress-nginx.yaml

配置端口暴露cluster-infra/ingress/overlays/dev/ingress-nginx-controller-nodeport.yaml

apiVersion: v1 
kind: Service 
metadata: 
 name: ingress-nginx-controller 
 namespace: ingress-nginx 
spec: 
 type: NodePort 
 ports: 
 - name: http 
 port: 80 
 targetPort: http 
 protocol: TCP 
 nodePort: 30080 
 - name: https 
 port: 443 
 targetPort: https 
 protocol: TCP 
 nodePort: 30443

根应用添加配置

resources: 
 - ../../cluster-infra/ingress/overlays/dev 
 - ../../cluster-infra/argocd/overlays/with-known-hosts

在argocd环境目录下添加ingress域名配置

apiVersion: networking.k8s.io/v1 
kind: Ingress 
metadata: 
 name: argocd-ingress 
 namespace: argocd 
 annotations: 
 nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" 
spec: 
 ingressClassName: nginx 
 rules: 
 - host: xxxx 
 http: 
 paths: 
 - path: / 
 pathType: Prefix 
 backend: 
 service: 
 name: argocd-server 
 port: 
 number: 443

git提交后即可自动同步并创建ingress

访问域名查看argocd webui

部署Prometheus+Grafana

为了完善集群监控，argocd 支持通过helm charts部署应用

配置cluster-infra/prometheus-stack/base/prometheus-stack.yaml

apiVersion: argoproj.io/v1alpha1 
kind: Application 
metadata: 
 name: prometheus-stack 
 namespace: argocd 
 annotations: 
 argocd.argoproj.io/sync-options: ServerSideApply=true 
spec: 
 project: default 
 source: 
 repoURL: https://prometheus-community.github.io/helm-charts 
 chart: kube-prometheus-stack 
 targetRevision: 75.16.1 
 helm: 
 valueFiles: 
 - values.yaml 
 destination: 
 server: https://kubernetes.default.svc 
 namespace: monitoring 
 syncPolicy: 
 automated: 
 prune: true 
 selfHeal: true

注意，由于Helm仓库部署prometheus时伴随大量CRD资源文件apply，需要开启ServerSideApply模式避免通过接口apply导致字符超长报错。参考：https://argocd.devops.gold/user-guide/sync-options/#_9

使用helm模式安装的应用无法使用patchesStrategicMerge策略覆盖，目前覆盖个性化配置只能将所有内容写到一个资源文件内，因此dev环境的个性化配置values如下

prometheus: 
 prometheusSpec: 
 retention: 3d 
 retentionSize: "2Gi" 
 resources: 
 requests: 
 memory: 128Mi 
 limits: 
 memory: 1Gi 
 storageSpec: 
 volumeClaimTemplate: 
 spec: 
 resources: 
 requests: 
 storage: 5Gi 
 ingress: 
 enabled: true 
 ingressClassName: nginx 
 annotations: {} 
 hosts: 
 - prometheus.fjy8018.top 
 paths: 
 - / 
 tls: 
 - hosts: 
 - xxx.xxx
 secretName: prometheus-tls 
 prometheusSpec: 
 externalUrl: https://xxx.xxx 
 routePrefix: / 
 
grafana: 
 adminPassword: "2EvwJJ0NOLzhYa" 
 resources: 
 requests: 
 memory: 64Mi 
 limits: 
 memory: 512Mi 
 ingress: 
 enabled: true 
 ingressClassName: nginx 
 annotations: {} 
 hosts: 
 - xxx.xxx 
 tls: 
 - hosts: 
 - xxx.xxx
 secretName: grafana-tls

需要在dev中覆盖base的资源文件，通过multi repo模式合并helm配置

apiVersion: argoproj.io/v1alpha1 
kind: Application 
metadata: 
 name: prometheus-stack 
 namespace: argocd 
 annotations: 
 argocd.argoproj.io/sync-options: ServerSideApply=true 
spec: 
 project: default 
 sources: 
 - repoURL: https://prometheus-community.github.io/helm-charts 
 chart: kube-prometheus-stack 
 targetRevision: 75.16.1 
 helm: 
 valueFiles: 
 - $values/cluster-infra/prometheus-stack/overlays/dev/values-dev.yaml 
 - repoURL: 'git@codeup.aliyun.com:xxxx/fjy8018/pve-k3s-gitops.git' 
 targetRevision: master 
 ref: values 
 destination: 
 server: https://kubernetes.default.svc 
 namespace: monitoring 
 syncPolicy: 
 automated: 
 prune: true 
 selfHeal: true

prometheus依赖CRD提前部署的namespace需要注册为独立应用提前配置好

apiVersion: v1 
kind: Namespace 
metadata: 
 name: monitoring

命名空间的配置和普通应用一致

apiVersion: argoproj.io/v1alpha1 
kind: Application 
metadata: 
 name: common-ns 
 namespace: argocd 
spec: 
 project: default 
 source: 
 repoURL: git@codeup.aliyun.com:xxxx/fjy8018/pve-k3s-gitops.git 
 targetRevision: master 
 path: cluster-infra/common-ns 
 destination: 
 server: https://kubernetes.default.svc 
 namespace: argocd 
 syncPolicy: 
 automated: 
 prune: true 
 selfHeal: true

最终根目录应用配置如下

resources: 
 - ../../cluster-infra/ingress/overlays/dev 
 - ../../cluster-infra/argocd/overlays/with-known-hosts 
 - ../../cluster-infra/prometheus-stack/overlays/dev 
 - common-ns-app.yaml

同步后效果如下

查看监控部署拓扑

通过域名访问grafana

DeepSeek 本地部署开启联网搜索

Sun, 09 Feb 2025 17:28:00 +0800

背景

由于官方DeepSeek频繁不可用，第三方API站点部署的全量模型无法联网，因此有需要联网推理的场景只能使用本地部署解决。

环境

CPU：12700 内存：32G DDR4 3200 显卡：RTX 2060 Super

模型准备

受显存限制，使用ollama只能下载8b和14b模型。

8b模型可以完全加载到显存中，使用cuda推理。
14b模型需要利用CPU+GPU混合推理。

本地下载开源docker镜像open-webui

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

该镜像下载困难可以手动传到阿里云镜像仓库做一次代理，本次使用我同步的代理镜像启动

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --rm registry.cn-guangzhou.aliyuncs.com/fjy8018_proxy/open-webui:main

首次启动耗时较长，耐心等待，显示如下画面表示启动完成

联网API

进入http://localhost:3000/admin/settings 点击联网搜索配置搜索引擎

目前有3种免费的联网搜索API，各有优劣

Duckduckgo

优点：免费，无需API配置缺点：网络无法直接访问。

Bing Search

优点：免费，国内网络可直连。缺点：需要微软账户+信用卡认证。

Google Search

优点：免费，搜索内容可靠。缺点：需要配置，网络无法直连。

此处为保障最好的搜索质量，使用Google搜索API接入。

申请API和密钥

进入谷歌可编程搜索引擎官网：Programmable Search Engine | Google for Developers 创建搜索引擎

点击控制面板

创建搜索引擎，名称任意

创建后得到搜索引擎ID

选择程序化访问

获取密钥

配置联网搜索

选择google pse，填写密钥和搜索ID

验证

8b + 联网搜索

使用8b+联网搜索验证

容器日志可以看到搜索到的文本内容

8b联网+推理显存沾满，显卡满载，但不需要CPU参与推理，32token/s，速度较快。

14b+联网搜索

改成14b后，token速度下降到7token/s，勉强可用。

由于显存不足，需要CPU+GPU混合推理，显卡cuda性能无法完全利用，CPU占用率高。

ClassLoader类隔离

Wed, 13 Nov 2024 21:56:49 +0800

基本概念

参考oracle jdk 21中对ClassLoader定义

A class loader is an object that is responsible for loading classes. The class ClassLoader is an abstract class. Given the binary name of a class, a class loader should attempt to locate or generate data that constitutes a definition for the class. A typical strategy is to transform the name into a file name and then read a “class file” of that name from a file system.

在Java中，ClassLoader负责加载类，当给定一个类的二进制名称，ClassLoader会尝试定位或生成构成该类定义的数据。

层次结构

ClassLoader遵循双亲委派模型，当请求查找类或资源时， ClassLoader 实例通常会将类或资源的搜索委托给其父类加载器，然后再尝试查找类或资源本身

由于JDK9中模块系统（Jigsaw）的引入，jdk8和jdk9类加载器层次结构有区别，ExtClassLoader变成PlatformClassLoader负责加载--module-path（或MODPATH环境变量）指定的模块

graph TD
 Bootstrap[Bootstrap ClassLoader]
 Extension[Extension ClassLoader]
 Application[Application ClassLoader]

 Bootstrap --> Extension
 Extension --> Application

 subgraph "JDK 8"
 Bootstrap:::primary
 Extension:::secondary
 Application:::tertiary
 end

graph TD
 Bootstrap[Bootstrap ClassLoader]
 Platform[Platform ClassLoader]
 App[Application ClassLoader]

 Bootstrap --> Platform
 Platform --> App

 subgraph "JDK 9"
 Bootstrap:::primary
 Platform:::secondary
 App:::tertiary
 end

生命周期

类加载的过程可以分为几个阶段及常见异常

加载（Loading）
ClassNotFoundException（找不到目标类）
验证（Verification）
ClassFormatError（类文件格式不正确）
NoClassDefFoundError（已加载的类无法初始化）
准备（Preparation）
解析（Resolution）
LinkageError（类加载器无法找到或者无法链接到其他类或接口）
IncompatibleClassChangeError
AbstractMethodError
NoSuchMethodError
…
初始化（Initialization）

核心方法

ClassLoader类提供了几个关键的方法来支持类的加载过程，其中loadClass和findClass最常使用

loadClass
protected Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException
Loads the class with the specified binary name. The default implementation of this method searches for classes in the following order:

Invoke findLoadedClass(String) to check if the class has already been loaded.

Invoke the loadClass method on the parent class loader. If the parent is null the class loader built into the virtual machine is used, instead.

Invoke the findClass(String) method to find the class.

If the class was found using the above steps, and the resolve flag is true, this method will then invoke the resolveClass(Class) method on the resulting Class object.

Subclasses of ClassLoader are encouraged to override findClass(String), rather than this method.

Unless overridden, this method synchronizes on the result of getClassLoadingLock method during the entire class loading process.

Parameters:

name - The binary name of the class

resolve - If true then resolve the class

Returns:

The resulting Class object

Throws:

ClassNotFoundException - If the class could not be found

官方文档给出了方法设计介绍，loadClass用于实现双亲委派职责，当父类无法加载目标class时，调用子类的findClass加载。

当需要打破双亲委派模型时，则直接覆盖loadClass

findClass
protected Class<?> findClass(String name) throws ClassNotFoundException
Finds the class with the specified binary name. This method should be overridden by class loader implementations that follow the delegation model for loading classes, and will be invoked by the loadClass method after checking the parent class loader for the requested class.

Implementation Requirements:

The default implementation throws ClassNotFoundException.

Parameters:

name - The binary name of the class

Returns:

The resulting Class object

Throws:

ClassNotFoundException - If the class could not be found

Since:

1.2

在findClass用于实现自定义类加载逻辑

以常用的URLClassLoader为例，findClass将传入的类名转换成路径格式匹配在URLClassPath类路径下实际存在的文件结构，如果找到了对应的资源，则继续处理；否则返回null。

public class URLClassLoader extends SecureClassLoader implements Closeable {
 protected Class<?> findClass(final String name)
 throws ClassNotFoundException
 {
 final Class<?> result;
 try {
 result = AccessController.doPrivileged(
 new PrivilegedExceptionAction<Class<?>>() {
 public Class<?> run() throws ClassNotFoundException {
 String path = name.replace('.', '/').concat(".class");
 Resource res = ucp.getResource(path, false);
 if (res != null) {
 try {
 return defineClass(name, res);
 } catch (IOException e) {
 throw new ClassNotFoundException(name, e);
 }
 } else {
 return null;
 }
 }
 }, acc);
 } catch (java.security.PrivilegedActionException pae) {
 throw (ClassNotFoundException) pae.getException();
 }
 if (result == null) {
 throw new ClassNotFoundException(name);
 }
 return result;
 }
}

应用

tomcat

为了支持应用隔离和模块化部署，tomcat扩展了已有的双亲委派模型

以tomcat9为例，其层次结构如下

graph TD
 Bootstrap[Bootstrap ClassLoader]
 System[System ClassLoader]
 Common[Common ClassLoader]
 Webapp1[Webapp1 ClassLoader]
 Webapp2[Webapp2 ClassLoader]
 Webapp3[Webapp3 ClassLoader]

 Bootstrap --> System
 System --> Common
 Common --> Webapp1
 Common --> Webapp2
 Common --> Webapp3

Bootstrap（所有应用可见，JVM核心依赖）
$JAVA_HOME/jre/lib/ext
System（所有应用可见，tomcat核心依赖）
$CATALINA_HOME/bin
Common（需要对所有应用可见的扩展类）
Webapp（每个应用隔离，打破双亲委派，本地优先）

As mentioned above, the web application class loader diverges from the default Java delegation model (in accordance with the recommendations in the Servlet Specification, version 2.4, section 9.7.2 Web Application Classloader). When a request to load a class from the web application’s WebappX class loader is processed, this class loader will look in the local repositories first, instead of delegating before looking.

重点在于webapp classloader，其打破双亲委派机制，当从 Web 应用程序的 WebappX 类加载器加载类的请求被处理时，这个类加载器会首先查看本地存储库，而不是先进行委托。

在应用程序视角，类查找顺序如下

Bootstrap ClassLoader
WebApp ClassLoader
/WEB-INF/classes
/WEB-INF/lib/*.jar
System ClassLoader
Common ClassLoader

当应用配置<Loader delegate="true"/>时，可切换为标准双亲委派模型

OSGi

OSGi是一个 Java 模块化系统和服务平台，它提供了一种动态管理模块（Bundle）的方法，支持不停止应用程序的情况下动态更新、添加或删除模块。

以Apache Felix实现的OSGi为例，其查找逻辑如下

循环检测：防止在查找过程中出现无限递归。
委托给父类加载器：如果目标类或资源属于需要委托给父类加载器处理的包，则先尝试使用父类加载器进行查找。
从导入列表中查找：在当前修订版本的导入列表中搜索类或资源。
从本地类路径查找：如果前面步骤未能找到，则尝试从当前修订版本自己的类路径中查找。
从动态导入中查找：最后，如果还没有找到，则会尝试从动态导入中查找。

private Object findClassOrResourceByDelegation(String name, boolean isClass)
 throws ClassNotFoundException, ResourceNotFoundException
 {
 // 初始化结果为null，表示尚未找到类或资源
 Object result = null;

 // 用于循环检测的集合，避免在查找过程中出现无限递归的情况
 Set requestSet = (Set) m_cycleCheck.get();
 if (requestSet == null)
 {
 // 如果当前线程中没有设置循环检测集合，则创建一个新的并设置到当前线程
 requestSet = new HashSet();
 m_cycleCheck.set(requestSet);
 }
 // 将要查找的名字添加到循环检测集合中，如果已经存在说明出现了循环依赖
 if (requestSet.add(name))
 {
 try
 {
 // 根据是查找类还是资源来获取目标的包名
 String pkgName = (isClass)
 ? Util.getClassPackage(name)
 : Util.getResourcePackage(name);

 // 检查是否应该将该包委托给父类加载器
 if (shouldBootDelegate(pkgName))
 {
 try
 {
 // 获取适当的类加载器进行委托
 ClassLoader bdcl = getBootDelegationClassLoader();
 // 根据是要查找类还是资源，调用相应的方法
 result = (isClass)
 ? (Object) bdcl.loadClass(name)
 : (Object) bdcl.getResource(name);
 // 如果是java.*包下的内容或者已经找到了结果，则直接返回
 if (pkgName.startsWith("java.") || (result != null))
 {
 return result;
 }
 }
 catch (ClassNotFoundException ex)
 {
 // 如果是java.*包下的类且未找到，则抛出异常
 if (pkgName.startsWith("java."))
 {
 throw ex;
 }
 }
 }

 // 在导入列表中查找
 result = searchImports(pkgName, name, isClass);

 // 如果在导入列表中未找到，则尝试从修订版本自己的类路径中查找
 if (result == null)
 {
 if (isClass)
 {
 // 获取内部类加载器，并尝试加载类
 ClassLoader cl = getClassLoaderInternal();
 if (cl == null)
 {
 // 如果类加载器无效，则抛出异常
 throw new ClassNotFoundException(
 "Unable to load class '"
 + name
 + "' because the bundle wiring for "
 + m_revision.getSymbolicName()
 + " is no longer valid.");
 }
 result = ((BundleClassLoader) cl).findClass(name);
 }
 else
 {
 // 如果不是查找类而是资源，则使用修订版本的方法来查找资源
 result = m_revision.getResourceLocal(name);
 }

 // 如果还是没找到，则尝试从动态导入中查找
 if (result == null)
 {
 result = searchDynamicImports(pkgName, name, isClass);
 }
 }
 }
 finally
 {
 // 清理：从循环检测集合中移除当前查找的名字
 requestSet.remove(name);
 }
 }
 else
 {
 // 如果检测到循环依赖，则返回null以打破循环
 return null;
 }

 // 如果最终结果仍为null，根据查找类型抛出相应的异常
 if (result == null)
 {
 if (isClass)
 {
 throw new ClassNotFoundException(
 name + " not found by " + this.getBundle());
 }
 else
 {
 throw new ResourceNotFoundException(
 name + " not found by " + this.getBundle());
 }
 }

 // 返回最终找到的结果
 return result;
 }

SofaArk

SofaArk是蚂蚁集团开源轻量类隔离框架

其定义3个概念：Ark Container，Ark Plugin，Ark Biz。

sofaark通过maven插件在打包时将插件启动器、导出类、导入类、导出资源、导入资源、优先级等元信息保存在 Ark Plugin 中的 META-INF/MANIFEST.MF 中，每个ark plugin都有独立的类加载器读取插件元信息和类资源导入导出关系。插件之间可以互相委托加载类。
ark biz只能加载自己模块的类信息，也可以委托plugin加载所需的类。

graph TD
 Bootstrap[BootstrapClassLoader]
 Extension[ExtensionClassLoader]
 Application[ApplicationClassLoader]
 URLClassLoader[URLClassLoader]
 Container[ContainerClassLoader]
 Agent[AgentClassLoader]
		Biz[BizClassLoader]
		Plugin[PluginClassLoader]

 Bootstrap --> Extension
 Extension --> Application
 Application --> URLClassLoader
	 URLClassLoader --> Container
		URLClassLoader --> Biz
		URLClassLoader --> Plugin
		URLClassLoader --> Agent

源码

AbstractClasspathClassLoader提供了类加载查找模式的抽象实现，loadClass会从以下顺序查找

本地类加载缓存
子类实现的loadClassInternal方法

BizClassLoader实现如下

package com.alipay.sofa.ark.container.service.classloader;

public class BizClassLoader extends AbstractClasspathClassLoader {

 
 @Override
 protected Class<?> loadClassInternal(String name, boolean resolve) throws ArkLoaderException {
 Class<?> clazz = null;

 // 0. sun reflect related class throw exception directly
 if (classloaderService.isSunReflectClass(name)) {
 throw new ArkLoaderException(
 String
 .format(
 "[ArkBiz Loader] %s : can not load class: %s, this class can only be loaded by sun.reflect.DelegatingClassLoader",
 bizIdentity, name));
 }

 // 1. findLoadedClass
 if (clazz == null) {
 clazz = findLoadedClass(name);
 }

 // 2. JDK related class
 if (clazz == null) {
 clazz = resolveJDKClass(name);
 }

 // 3. Ark Spi class
 if (clazz == null) {
 clazz = resolveArkClass(name);
 }

 // 4. pre find class
 if (clazz == null) {
 clazz = preLoadClass(name);
 }

 // 5. Plugin Export class
 if (clazz == null) {
 clazz = resolveExportClass(name);
 }

 // 6. Biz classpath class
 if (clazz == null) {
 clazz = resolveLocalClass(name);
 }

 // 7. Java Agent ClassLoader for agent problem
 if (clazz == null) {
 clazz = resolveJavaAgentClass(name);
 }

 // 8. post find class
 if (clazz == null) {
 clazz = postLoadClass(name);
 }

 if (clazz != null) {
 if (resolve) {
 super.resolveClass(clazz);
 }
 return clazz;
 }

 throw new ArkLoaderException(String.format("[ArkBiz Loader] %s : can not load class: %s",
 bizIdentity, name));
 }
}

PluginClassLoader实现如下

package com.alipay.sofa.ark.container.service.classloader;

public class PluginClassLoader extends AbstractClasspathClassLoader {

	@Override
 protected Class<?> loadClassInternal(String name, boolean resolve) throws ArkLoaderException {
 Class<?> clazz = null;

 // 0. sun reflect related class throw exception directly
 if (classloaderService.isSunReflectClass(name)) {
 throw new ArkLoaderException(
 String
 .format(
 "[ArkPlugin Loader] %s : can not load class: %s, this class can only be loaded by sun.reflect.DelegatingClassLoader",
 pluginName, name));
 }

 // 1. findLoadedClass
 if (clazz == null) {
 clazz = findLoadedClass(name);
 }

 // 2. JDK related class
 if (clazz == null) {
 clazz = resolveJDKClass(name);
 }

 // 3. Ark Spi class
 if (clazz == null) {
 clazz = resolveArkClass(name);
 }

 // 4. pre find class
 if (clazz == null) {
 clazz = preLoadClass(name);
 }

 // 5. Import class export by other plugins
 if (clazz == null) {
 clazz = resolveExportClass(name);
 }

 // 6. Plugin classpath class
 if (clazz == null) {
 clazz = resolveLocalClass(name);
 }

 // 7. Java Agent ClassLoader for agent problem
 if (clazz == null) {
 clazz = resolveJavaAgentClass(name);
 }

 // 8. Post find class
 if (clazz == null) {
 clazz = postLoadClass(name);
 }

 if (clazz != null) {
 if (resolve) {
 super.resolveClass(clazz);
 }
 return clazz;
 }

 throw new ArkLoaderException(String.format(
 "[ArkPlugin Loader] %s : can not load class: %s", pluginName, name));
 }
}

流程基本一致，主要在于plugin和biz读取到的import类元信息不同

参考资料

GraphQL实践10——Netflix Dgs Graphql异步订阅

Fri, 28 Oct 2022 21:59:23 +0800

介绍

GraphQL订阅模式支持服务端主动向客户端推送数据通知，避免客户端轮训

DGS Subscriptions

新增依赖

此处使用的是SpringBoot Web组件，故引入websockets依赖

如果使用的是WebFlux则要引入对应的dgs-webflux

		 <dependency>
 <groupId>com.netflix.graphql.dgs</groupId>
 <artifactId>graphql-dgs-subscriptions-websockets-autoconfigure</artifactId>
 </dependency>

异步通知解析

此处定义为当新增一个Actor就给所有订阅者发异步通知

@Slf4j
@DgsComponent
@RequiredArgsConstructor
public class ActorDataFetcher {

	private final ActorAssembler actorAssembler;

	private FluxSink<Actor> actorStream;

	private ConnectableFlux<Actor> actorPublisher;

	@PostConstruct
	private void createActorPublisher() {
		Flux<Actor> publisher = Flux.create(emitter -> {
			actorStream = emitter;
		});

		actorPublisher = publisher.publish();
		actorPublisher.connect();
	}

	@DgsMutation
	public Actor addActor(@InputArgument SubmitActor actor) {
		Actor actorEntity = actorAssembler.convert(actor);
		actorEntity.setActorId(10);
		actorEntity.setLastUpdate(DateUtil.format(new Date(), DatePattern.NORM_DATETIME_PATTERN));
		actorStream.next(actorEntity);
		log.info("服务端创建演员：{}", actor);
		return actorEntity;
	}

	/**
	 * 异步通知
	 *
	 * @return
	 */
	@DgsSubscription
	public Publisher<Actor> actorAdded() {
		return actorPublisher;
	}
}

单元测试

由于异步通知无法用浏览器自带的控制台测试，只能通过GraphQL客户端订阅后接收异步通知，此处参考官方最佳实践，使用单元测试模拟

@Slf4j
@SpringBootTest
class ActorDataFetcherTest {

	@Autowired
	private DgsQueryExecutor dgsQueryExecutor;

	@Test
	void addActor() {

		GraphQLQueryRequest graphQLQueryRequest = new GraphQLQueryRequest(
				AddActorGraphQLQuery.newRequest()
						.actor(
								SubmitActor.newBuilder()
										.firstName("fu")
										.lastName("jiayang")
										.build())
						.build(),
				new AddActorProjectionRoot()
						.actorId()
						.firstName()
						.lastUpdate());
		dgsQueryExecutor.execute(graphQLQueryRequest.serialize());
	}

	@Test
	void actorAdded() {
		ExecutionResult executionResult = dgsQueryExecutor.execute("""
				subscription {
				 actorAdded {
				 actorId
				 firstName
				 lastName
				 lastUpdate
				 }
				}
				""");
		Publisher<ExecutionResult> reviewPublisher = executionResult.getData();
		List<Actor> actors = new CopyOnWriteArrayList<>();

		reviewPublisher.subscribe(new Subscriber<>() {
			@Override
			public void onSubscribe(Subscription s) {
				s.request(2);
			}

			@Override
			public void onNext(ExecutionResult executionResult) {
				if (executionResult.getErrors().size() > 0) {
					executionResult.getErrors().forEach(error -> log.error(error.toString()));
				}
				Map<String, Object> actorResult = executionResult.getData();
				Actor actor = new ObjectMapper().convertValue(actorResult.get("actorAdded"), Actor.class);
 log.info("客户端监听到演员新增：{}",actor);
				actors.add(actor);
			}

			@Override
			public void onError(Throwable t) {
				t.printStackTrace();
			}

			@Override
			public void onComplete() {
			}
		});

		addActor();
		addActor();

		assertThat(actors.size()).isEqualTo(2);
	}
}

从日志输出可以看到客户端成功接收异步通知

总结

订阅模式属于GraphQL特色功能，类似消息队列实现方式，减少客户端对普通查询接口轮训性能开销

GraphQL实践9——Netflix Dgs Graphql数据更新

Fri, 28 Oct 2022 21:58:54 +0800

功能介绍

graphql-java提供了Mutations类型支持数据更新

NGS Mutations

类型定义

type Mutation {
 addActor(actor: SubmitActor!): Actor
}

type Subscription {
 actorAdded: Actor
}

input SubmitActor {
 firstName: String!
 lastName: String!
}

SubmitActor类型可以交给代码生成器自动生成

类型转换器

@Mapper(componentModel = SPRING)
public interface ActorAssembler {

	Actor convert(SubmitActor submit);
}

定义解析器

	@DgsMutation
	public Actor addActor(@InputArgument SubmitActor actor) {
		Actor actorEntity = actorAssembler.convert(actor);
		actorEntity.setActorId(10);
		actorEntity.setLastUpdate(new Date());
		return actorEntity;
	}

测试

访问http://localhost:8080/graphiql即可看到在线查询页面

GraphQL实践8——Netflix Dgs Graphql代码生成

Fri, 28 Oct 2022 21:57:00 +0800

功能介绍

DGS官方开发了识别schema配置生成对应类型、测试类的代码生成器，而Maven插件则由社区维护

Maven代码生成插件

新增依赖

<dependency> 
 <groupId>com.netflix.graphql.dgs.codegen</groupId> 
 <artifactId>graphql-dgs-codegen-client-core</artifactId> 
 <version>5.1.17</version> 
</dependency>

还需要新增构建插件

	 <plugin>
 <groupId>io.github.deweyjose</groupId>
 <artifactId>graphqlcodegen-maven-plugin</artifactId>
 <version>1.24</version>
 <executions>
 <execution>
 <goals>
 <goal>generate</goal>
 </goals>
 </execution>
 </executions>
 <configuration>
 <addGeneratedAnnotation>true</addGeneratedAnnotation>
 <packageName>top.fjy8018.graphsqldemo.generated</packageName>
 <schemaPaths>
 <param>src/main/resources/graphql/actor.graphqls</param>
 </schemaPaths>
 <!-- 生成测试客户端 -->
 <generateClient>true</generateClient>
 </configuration>
 </plugin>
 <plugin>
 <groupId>org.codehaus.mojo</groupId>
 <artifactId>build-helper-maven-plugin</artifactId>
 <executions>
 <execution>
 <phase>generate-sources</phase>
 <goals>
 <goal>add-source</goal>
 </goals>
 <configuration>
 <sources>
 <source>${project.build.directory}/generated-sources</source>
 </sources>
 </configuration>
 </execution>
 </executions>
 </plugin>

插件配置指引：deweyjose/graphqlcodegen: Maven port of the Netflix Gradle code generation plugin for graphql.

编译

执行

mvn clean package

即可生成对应的类

存在问题

IDEA默认不识别target目录下的资源文件，生成后会导致无法引用

解决方法

需要手动将对应的生成目录配置为Source Root即可

总结

DGS官方推荐开发者优先使用代码生成器生成标准类库后再开发，可以减少开发量

GraphQL实践7——Netflix Dgs Graphql分页查询

Fri, 28 Oct 2022 16:06:31 +0800

问题背景

上一篇文章提到，使用List查询每次都返回全量数据，而实际场景更多使用分页查询，graphql-java提供Connection实现游标分页，在Dgs也有对应功能扩展Relay Pagination

集成Relay Pagination

新增依赖

 <dependency>
 <groupId>com.netflix.graphql.dgs</groupId>
 <artifactId>graphql-dgs-pagination</artifactId>
 </dependency>

Mybatis Plus 配置分页插件

@Configuration
@MapperScan("top.fjy8018.graphsqldemo.mapper")
public class MybatisPlusConfig {

 /**
 * 分页插件配置
 *
 * @return
 */
 @Bean
 public MybatisPlusInterceptor mybatisPlusInterceptor() {
 MybatisPlusInterceptor interceptor = new MybatisPlusInterceptor();
 PaginationInnerInterceptor pageConfig = new PaginationInnerInterceptor(DbType.MYSQL);
 interceptor.addInnerInterceptor(pageConfig);
 return interceptor;
 }
}

Relay Pagination配置

type Query {
 actorList: [Actor]
 findOneActor(id : Int!): Actor
 filmList(current: Int!,size: Int!): FilmConnection
 findOneFilm(id : Int!): Film
}

type Actor {
 actorId: Int!
 firstName: String!
 lastName: String!
 lastUpdate: String
}

type Film @connection {
 filmId: Int!
 title: String!
 description: String
 releaseYear: Int
 languageId: Int!
 originalLanguageId: Int
 rentalDuration: Int
 rentalRate: Float
 length: Int
 replacementCost: Float
 rating: String
 specialFeatures: String
 lastUpdate: String
 actors: [Actor]
}

@connection和XXXConection已经由graphql-java底层框架自动生成，无需手动定义

类型转换器

public interface ConnectionAssembler {

 /**
 * mybatis plus分页对象转为Connection
 *
 * @param page 分页对象
 * @param <T> 泛型
 * @return Connection
 */
 static <T> Connection<T> convert(IPage<T> page) {
 AtomicInteger index = new AtomicInteger();
 List<Edge<T>> defaultEdges = Stream.ofNullable(page.getRecords())
 .flatMap(Collection::stream)
 .map(e -> (Edge<T>) new DefaultEdge<>(e, new DefaultConnectionCursor(String.valueOf(index.incrementAndGet()))))
 .toList();
 return new DefaultConnection<>(defaultEdges,
 new DefaultPageInfo(new DefaultConnectionCursor(String.valueOf(page.getCurrent())),
 new DefaultConnectionCursor(String.valueOf(page.getPages())),
 page.getCurrent() > 1,
 page.getCurrent() < page.getPages()));
 }
}

Graphql解析配置

	@DgsQuery
	public Connection<Film> filmList(@InputArgument("current") Integer current,
									 @InputArgument("size") Integer size,
									 DgsDataFetchingEnvironment dfe) {
		IPage<Film> page = new PageDTO<>(current, size);
		IPage<Film> pageResult = filmRepository.page(page);
		return ConnectionAssembler.convert(pageResult);
	}

将List查询改为返回Connection即可

测试

访问http://localhost:8080/graphiql即可看到在线查询页面

查看查询数量，确认是按照分页查询

总结

到此，DSG核心功能尝试基本完成，基本实现常用的Rest功能，同时集成方便，而且官方文档清晰，但目前开源社区还没有太多的基于DSG的系统集成样例，还需要在实践中评估摸索。

thoughtworks也从2016年开始关注graphql，目前给出的建议也是“评估”，可以进行小规模尝试并在合适的时候推广

We’ve seen many successful GraphQL implementations on our projects. We’ve seen some interesting patterns of use too, including GraphQL for server-side resource aggregation. That said, we’ve concerns about misuse of this framework and some of the problems that can occur. Examples include performance gotchas around N+1 queries and lots of boilerplate code needed when adding new models, leading to complexity. There are workarounds to these gotchas such as query caching. Even though it’s not a silver bullet, we still think it’s worth assessing as part of your architecture.

GraphQL实践6——Netflix Dgs Graphql N+1问题

Fri, 28 Oct 2022 16:05:51 +0800

N+1问题介绍

对于上一篇文章样例，如果要获取每个电影的演员名单，要执行如下动作

查询所有电影清单
遍历N个电影，查询对应电影的演员名单

总查询开销为N+1次查询，代价非常大，效率低

优化方案 DataLoader

对于一对一表关联的情况且每个关联对象只有一个值，可以直接使用BatchLoader

但实际上，对于大部分表关联情况，通常为一对多或者多对多，不保证每个关联都有值，此时需要使用MappedBatchLoader维护关联关系

维护多对多实体关系Map

public interface ActorService {
	Map<Integer, List<Actor>> listByFilmId(Collection<Integer> filmIds);
}

@Slf4j
@Service
@RequiredArgsConstructor
public class ActorServiceImpl implements ActorService {

	private final ActorRepository actorRepository;

	private final FilmActorRepository filmActorRepository;

	@Override
	public Map<Integer, List<Actor>> listByFilmId(Collection<Integer> filmIds) {
		List<FilmActor> filmActors = filmActorRepository.listByFilmId(filmIds);
		if (CollectionUtils.isEmpty(filmActors)) {
			return Collections.emptyMap();
		}
		Map<Integer, List<FilmActor>> filmActorMap = filmActors.stream().collect(Collectors.groupingBy(FilmActor::getFilmId));
		List<Integer> actorIds = filmActors.stream().map(FilmActor::getActorId).distinct().toList();
		List<Actor> actors = actorRepository.listByIds(actorIds);
		Map<Integer, Actor> actorsMap = Stream.ofNullable(actors).flatMap(Collection::stream).collect(Collectors.toMap(Actor::getActorId, Function.identity(), (e1, e2) -> e1));
		Map<Integer, List<Actor>> result = Maps.newHashMapWithExpectedSize(filmIds.size());
		filmActorMap.forEach((k, v) -> {
			List<Actor> actorList = Stream.ofNullable(filmActorMap.get(k)).flatMap(Collection::stream).map(e -> actorsMap.get(e.getActorId())).toList();
			result.put(k, actorList);
		});
		return result;
	}
}

Dgs MappedBatchLoader 配置

@DgsDataLoader(name = "actors")
@RequiredArgsConstructor
public class FilmActorsDataLoader implements MappedBatchLoader<Integer, List<Actor>> {

	private final ActorService actorService;

	@Override
	public CompletionStage<Map<Integer, List<Actor>>> load(Set<Integer> keys) {
		return CompletableFuture.supplyAsync(() -> actorService.listByFilmId(keys));
	}
}

测试

访问http://localhost:8080/graphiql即可看到在线查询页面

此时执行嵌套查询只会查询2次，一次查电影列表，一次查所有电影的Actor列表，非常快速

总结

使用BatchLoader实现对N+1问题优化，但还有一个潜在问题，即大数据分页，该样例中，有1000个电影，对应5000+演员，查询一次获取全量数据，对数据库压力较大，也不符合实际场景，还需要进行分页优化

GraphQL实践5——Netflix Dgs Graphql懒加载

Fri, 28 Oct 2022 16:04:43 +0800

问题背景

在graphql中，需要查询的字段由调用方传递，如果此时存在关联表才能获取的字段，但前端不需要该字段的时候，默认查询所有字段会带来较大开销

例如

type Actor {
 actorId: Int!
 firstName: String!
 lastName: String!
 lastUpdate: String
}

type Film {
 filmId: Int!
 title: String!
 description: String
 actors: [Actor]
}

type Query {
 filmList: [Film]
}

解决方案——懒加载

DGS框架中支持配置调用方传递了指定字段才读取对应的数据

配置懒加载

增加实体定义，模拟嵌套实体

Mybatis Plus配置

扫描配置

@Configuration
@MapperScan("top.fjy8018.graphsqldemo.mapper")
public class MybatisPlusConfig {
}

实体

表结构还是使用sakila样例数据库的表

@Data
public class Actor implements Serializable {

 @Serial
 private static final long serialVersionUID = 1L;

 /**
 * 联系人ID
 */
 @TableId(type = IdType.AUTO)
 private Integer actorId;

 private String firstName;

 private String lastName;

 private Date lastUpdate;

}

@Data
public class Film implements Serializable {

	@Serial
	private static final long serialVersionUID = 1L;

	/**
	 * 联系人ID
	 */
	@TableId(type = IdType.AUTO)
	private Integer filmId;

	private String title;

	private String description;

	private Integer releaseYear;

	private Integer languageId;

	private Integer originalLanguageId;

	private Integer rentalDuration;

	private BigDecimal rentalRate;

	private Integer length;

	private BigDecimal replacementCost;

	private String rating;

	private String specialFeatures;

	private Date lastUpdate;
}

@Data
public class FilmActor implements Serializable {

	@Serial
	private static final long serialVersionUID = 1L;

	/**
	 * 联系人ID
	 */
	@TableId(type = IdType.AUTO)
	private Integer actorId;

	private Integer filmId;

	private Date lastUpdate;
}

Mapper

public interface ActorMapper extends BaseMapper<Actor> {}
public interface FilmActorMapper extends BaseMapper<FilmActor> {}
public interface FilmMapper extends BaseMapper<Film> {}

DAO

public interface ActorRepository extends IService<Actor> {
}
public interface FilmActorRepository extends IService<FilmActor> {
	List<FilmActor> listByFilmId(@Size.List({@Size(min = 1, message = "至少要有一个元素")}) List<Integer> filmIds);
}
public interface FilmRepository extends IService<Film> {
}

@Validated
@Repository
@RequiredArgsConstructor
public class ActorRepositoryImpl extends ServiceImpl<ActorMapper, Actor> implements ActorRepository {
}
@Validated
@Repository
@RequiredArgsConstructor
public class FilmActorRepositoryImpl extends ServiceImpl<FilmActorMapper, FilmActor> implements FilmActorRepository {
	@Override
	public List<FilmActor> listByFilmId(List<Integer> filmIds) {
		return lambdaQuery().in(FilmActor::getFilmId, filmIds).list();
	}
}
@Validated
@Repository
@RequiredArgsConstructor
public class FilmRepositoryImpl extends ServiceImpl<FilmMapper, Film> implements FilmRepository {
}

Service

多表查询聚合逻辑

public interface ActorService {
	List<Actor> listByFilmId(Integer filmId);
}

@Slf4j
@Service
@RequiredArgsConstructor
public class ActorServiceImpl implements ActorService {

	private final ActorRepository actorRepository;

	private final FilmActorRepository filmActorRepository;

	@Override
	public List<Actor> listByFilmId(Integer filmId) {
		List<FilmActor> filmActors = filmActorRepository.listByFilmId(List.of(filmId));
		if (CollectionUtils.isEmpty(filmActors)) {
			return Collections.emptyList();
		}
		List<Integer> actorIds = filmActors.stream().map(FilmActor::getActorId).toList();
		return actorRepository.listByIds(actorIds);
	}
}

Graphql 查询解析器

@DgsComponent
@RequiredArgsConstructor
public class FilmDataFetcher {

	private final FilmRepository filmRepository;

	private final ActorService actorService;

	@DgsQuery
	public Collection<Film> filmList() {
		return filmRepository.list();
	}

	@DgsData(parentType = "Film")
	public List<Actor> actors(DgsDataFetchingEnvironment dfe) {
		Film film = dfe.getSource();
		return actorService.listByFilmId(film.getFilmId());
	}

	@DgsQuery
	public Film findOneFilm(@InputArgument Integer id) {
		return filmRepository.getById(id);
	}
}

增加DgsDataFetchingEnvironment入参，DGS会自动获取请求上下文参数

此处定义即为传入了actors参数才执行该方法获取对应的Actor集合，实现懒加载

测试

访问http://localhost:8080/graphiql即可看到在线查询页面

不访问Actor

此时可以看到日志输出，没有查询Actor表

访问Actor

此时DSG会执行N+1查询Actor表，性能损耗非常大

总结

对于表关联，开销较大字段，都可以使用DSG懒加载实现性能优化，但此时还是存在N+1查询性能问题，将在后面使用DataLoader优化

GraphQL实践4——Netflix Dgs Graphql集成Mybatis Plus

Fri, 28 Oct 2022 16:03:00 +0800

介绍

通过JPA或者Mybatis Plus集成Graphql框架已经很简便，但其实使用graphql-java集成过于底层，很多扩展内容实现并不方便，例如：

分页查询
资源懒加载
N+1问题

Netflix 基于长期的Graphql实践，2020年开源了DGS Framework，让解决上述问题更加简单，大量开源系统集成Graphql也是通过集成DGS组件

The DGS framework project started at Netflix in 2019 as internal teams began developing multiple GraphQL services. As 2020 wrapped up, Netflix decided to open source the framework and build a community around it.

集成Netflix DGS

官网：Home - DGS Framework (netflix.github.io)

引入依赖

技术栈：

SpringBoot 2.7.1
Mybatis Plus
JDK 17
Dgs SpringBoot

本次和上一篇文章一样，选择原因不在赘述

	<dependencyManagement>
 <dependencies>
 <dependency>
 <groupId>com.netflix.graphql.dgs</groupId>
 <artifactId>graphql-dgs-platform-dependencies</artifactId>
 <!-- The DGS BOM/platform dependency. This is the only place you set version of DGS -->
 <version>4.9.16</version>
 <type>pom</type>
 <scope>import</scope>
 </dependency>
 </dependencies>
 </dependencyManagement>

 <dependencies>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-web</artifactId>
 </dependency>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-validation</artifactId>
 </dependency>

 <dependency>
 <groupId>com.netflix.graphql.dgs</groupId>
 <artifactId>graphql-dgs-spring-boot-starter</artifactId>
 </dependency>

 <dependency>
 <groupId>org.projectlombok</groupId>
 <artifactId>lombok</artifactId>
 <optional>true</optional>
 </dependency>

 <dependency>
 <groupId>mysql</groupId>
 <artifactId>mysql-connector-java</artifactId>
 <scope>runtime</scope>
 </dependency>
 <!-- mybatis-plus-boot-starter -->
 <dependency>
 <groupId>com.baomidou</groupId>
 <artifactId>mybatis-plus-boot-starter</artifactId>
 <version>3.5.2</version>
 </dependency>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-test</artifactId>
 <scope>test</scope>
 </dependency>
 </dependencies>

配置文件

spring:
 datasource:
 driver-class-name: com.mysql.cj.jdbc.Driver
 username: root
 password: ****
 url: jdbc:mysql://localhost:3306/sakila?useSSL=false&useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai
server:
 port: 8080
logging:
 level:
 top.fjy8018.graphsqldemo.mapper: DEBUG

dgs:
 graphql:
 schema-locations:
 - classpath*:graphql/*.graphqls

由于DGS默认扫描路径为classpath*:schema/**/*.graphql*，本次基于上一篇文章的工程配置，需要手动指定一下扫描文件目录

Mybatis Plus配置

扫描配置

@Configuration
@MapperScan("top.fjy8018.graphsqldemo.mapper")
public class MybatisPlusConfig {
}

实体

表结构还是使用sakila样例数据库的表

@Data
public class Actor implements Serializable {

 @Serial
 private static final long serialVersionUID = 1L;

 /**
 * 联系人ID
 */
 @TableId(type = IdType.AUTO)
 private Integer actorId;

 private String firstName;

 private String lastName;

 private Date lastUpdate;

}

Mapper

public interface ActorMapper extends BaseMapper<Actor> {
}

DAO

从分层模型定义上，MP自带的CRUD接口本质上还是一个DAO，故定义为Repository更加合适，不与业务逻辑Serice冲突

同时也建议QueryWrapper仅在对应的Repository使用，防止查询条件散落在各处

public interface ActorRepository extends IService<Actor> {
}

@Validated
@Repository
@RequiredArgsConstructor
public class ActorRepositoryImpl extends ServiceImpl<ActorMapper, Actor> implements ActorRepository {
}

Graphql 查询解析器

@DgsComponent
@RequiredArgsConstructor
public class ActorDataFetcher {

	private final ActorRepository actorRepository;

	@DgsQuery
	public Collection<Actor> actorList() {
		return actorRepository.list();
	}

	@DgsQuery
	public Actor findOneActor(@InputArgument Integer id) {
		return actorRepository.getById(id);
	}
}

type Query {
 actorList: [Actor]
 findOneActor(id : Int!): Actor
}

type Actor {
 actorId: Int!
 firstName: String!
 lastName: String!
 lastUpdate: String
}

此处集成和graphql-java-kickstart类似，也是十分简便，不指定别名默认将方法名映射到文件方法名

测试

访问http://localhost:8080/graphiql即可看到在线查询页面

总结

此处还看不出DGS与其他graphql-java组件的区别，都是集成简便，可以和任何数据源、ORM框架集成，后续将会逐步实践DGS特性，包括懒加载、N+1问题、分页组件等

GraphQL实践3——Spring-Graphql集成Mybatis Plus

Thu, 27 Oct 2022 15:37:00 +0800

介绍

考虑到国内大部分场景对接的是Mybatis或者Mybatis Plus生态，而不是JPA，同时Graphql版本也进一步迭代，故计划使用Mybatis Plus对接Graphql

集成过程

引入依赖

由于JDK已经发展到19，目前最新的LTS为17，我所在的公司也将生产升级到了JDK 17，未来更多新应用会直接基于17构建，故此处直接基于JDK17对接
SpringBoot版本为2.7.1，目前已知3.0.0 RC1还不兼容第三方graphql
此处不使用spring官方的starter，改为使用graphql-java-kickstart的starter，编写更加简单


 <dependencies>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-web</artifactId>
 </dependency>

 <dependency>
 <groupId>org.projectlombok</groupId>
 <artifactId>lombok</artifactId>
 <optional>true</optional>
 </dependency>

 <!-- graphql-spring-boot-starter -->
 <dependency>
 <groupId>com.graphql-java-kickstart</groupId>
 <artifactId>graphql-spring-boot-starter</artifactId>
 <version>12.0.0</version>
 </dependency>

 <dependency>
 <groupId>mysql</groupId>
 <artifactId>mysql-connector-java</artifactId>
 <scope>runtime</scope>
 </dependency>
 <!-- mybatis-plus-boot-starter -->
 <dependency>
 <groupId>com.baomidou</groupId>
 <artifactId>mybatis-plus-boot-starter</artifactId>
 <version>3.5.2</version>
 </dependency>
 <!-- querydsl-core -->
 <dependency>
 <groupId>com.querydsl</groupId>
 <artifactId>querydsl-core</artifactId>
 </dependency>
 
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-test</artifactId>
 <scope>test</scope>
 </dependency>
 </dependencies>

配置文件

spring:
 datasource:
 driver-class-name: com.mysql.cj.jdbc.Driver
 username: root
 password: ****
 url: jdbc:mysql://localhost:3306/sakila?useSSL=false&useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai
server:
 port: 8080
graphql:
 tools:
 schema-location-pattern: graphql/*.graphqls
 graphiql:
 # 开启可交互页面，用于调试
 enabled: true
logging:
 level:
 top.fjy8018.graphsqldemo.mapper: DEBUG

Mybatis Plus配置

扫描配置

@Configuration
@MapperScan("top.fjy8018.graphsqldemo.mapper")
public class MybatisPlusConfig {
}

实体

@Data
public class Actor implements Serializable {

 @Serial
 private static final long serialVersionUID = 1L;

 /**
 * 联系人ID
 */
 @TableId(type = IdType.AUTO)
 private Integer actorId;

 private String firstName;

 private String lastName;

 private Date lastUpdate;

}

Mapper

public interface ActorMapper extends BaseMapper<Actor> {
}

DAO

public interface ActorRepository extends IService<Actor> {
}

@Validated
@Repository
@RequiredArgsConstructor
public class ActorRepositoryImpl extends ServiceImpl<ActorMapper, Actor> implements ActorRepository {
}

Graphql 查询解析器

@Component
@AllArgsConstructor
public class ActorGraphQLQueryResolver implements GraphQLQueryResolver {

 private final ActorRepository actorRepository;

 public Collection<Actor> actorList() {
 return actorRepository.list();
 }

 public Actor findOneActor(Integer id) {
 return actorRepository.getById(id);
 }
}

查询文件定义在resources/graphql/actor.graphqls

type Query {
 actorList: [ActorEntity]
 findOneActor(id : ID!): ActorEntity
}

type ActorEntity {
 actorId: ID!
 firstName: String!
 lastName: String!
 lastUpdate: String
}

测试

访问http://localhost:8080/graphiql即可看到在线查询页面

总结

依托Mybatis Plus强大的自带CRUD方法，可以实现和JPA类似的快速对接

Java 17 生产实践

Fri, 06 May 2022 11:04:00 +0800

背景

版本发布

2021年3月16日 Java 17 作为继Java 11的下一个LTS版本正式发布，JDK 17 作为下一代 LTS 将提供至少到 2026 年的支持。

同业动态

2022年ThoughtWorks 26期技术雷达也提到，建议各个组织将Java 17纳入评估。

评估 82. Java 17

我们通常不会专门介绍某一语言的新版本，但我们还是想关注一下 Java 新的长期支持（LTS）版本——Java 17。尽管出现了前景不错的新特性，比如模式匹配等预览特性，但其实向新 LTS 版本升级的方案更应该吸引各个组织的兴趣。我们建议各个组织在 Java 有新的发布时对其进行评估，确保能够恰当地适配这些新特性和版本。尽管定期更新有利于简化开发并且方便管理，但许多组织却意外地并不经常更新语言的版本。希望通过 LTS 版本的升级以及开发团队对语言的定期更新，能够使生产软件免于因为“更新成本太高”而一直困在 Java 的过时版本上。

框架发展情况

Spring Framework 6和SpringBoot 3.0 将在2022年底正式发布，将基于Java 17构建。

Spring Boot 3.0.0 M1 Release Notes · spring-projects/spring-boot Wiki (github.com)

Minimum Requirements Changes
Spring Boot 3.0 makes the following changes to its minimum supported versions:

Gradle 7.3

Jakarta EE 9

Java 17

Kotlin 1.6

Spring Framework 6

其他团队落地情况

在此之前，我所在的团队使用的Java版本一直停留在Java8，今年经过其他开发团队的尝试，我们团队在4月份也正式决定在生产环境使用Java 17。

实践情况

开发组件版本

本次计划在内部运营管理系统重构中使用Java 17作为基础版本，其他核心组件版本如下

SpringBoot 2.6.6
mapstruct 1.4.2.Final
mybatis-plus 3.5.1
jdk 17.0.1

开发时SpringBoot 3.0还处于M2里程碑阶段，暂时不适合带入生产环境，等未来3.0 GA后再考虑升级

基础镜像

原本使用的基于jdk8 的Docker image也需要替换成基于jdk 17的image

带来的问题：

原本基于jdk8构建并注入到镜像中的opentelemetry-javaagent暂时不可用，等待后续基于jdk17 构建后才能重新启用

JVM参数

由于从Java11开始G1替代CMS作为默认的垃圾收集器，Java14开始移除了CMS GC。

故团队内定义的系统框架配置的GC调优参数基本废弃，并且可以使用Java 14新引入的JVM参数，例如-XX:+ShowCodeDetailsInExceptionMessages

开启前

Exception in thread "main" java.lang.NullPointerException
 at Book.main(Book.java:5)

开启后

Exception in thread "main" java.lang.NullPointerException: 
 Cannot assign field "book" because "shoopingcart.buy" is null
 at Book.main(Book.java:5)

语法特性

从Java8到Java17期间新增的语法特性很多，此处提及几个比较常用的

快速创建不可变集合

不可变对象具有许多优点，包括：

可安全使用不受信任的库。
线程安全。
不需要支持变化判断，所有不可变的集合实现都比其可变的同级实现更节省内存。

Java9开始新增List、Map、Set等of方法快速创建不可变集合

List.of(prodStrategy, instStrategy)

实现源码

 static <E> List<E> of(E e1, E e2) {
 return new ImmutableCollections.List12<>(e1, e2);
 }

Java16开始，Stream类中也新增toList()方法，快速将流转成不可变集合

List<Long> ids = idList.stream().map(ContactId::id).toList();

实现源码

 default List<T> toList() {
 return (List<T>) Collections.unmodifiableList(new ArrayList<>(Arrays.asList(this.toArray())));
 }

注意：如果需要将流转成可变集合，还是需要使用collect(Collectors.toList())方法

record类型

Java14开始新增record类型，用于表示不可变class，实践中可以方便的表示值对象，无需使用@Value注解

public record ContactId(@NotNull(message = "ID不能为空") Long id) {
}

instanceof模式匹配

Java14开始支持模式匹配，无需做强制类型转换

	private void configMessageConverters(List<HttpMessageConverter<?>> converters) {
		converters.removeIf(x -> x instanceof StringHttpMessageConverter || x instanceof MappingJackson2HttpMessageConverter);
		converters.add(new StringHttpMessageConverter(Charsets.UTF_8));
		converters.add(new MappingJackson2HttpMessageConverter(objectMapper));
	}

创建类型安全的流

Java9 开始，新增ofNullable方法创建类型安全的流对象，避免在调用stream方法时报NPE

// query.getDescColumns()可能为null
List<OrderItem> descItems = Stream.ofNullable(query.getDescColumns())
				.flatMap(Collection::stream)
				.map(OrderItem::desc)
				.toList();

性能和安全提升

从同一段功能逻辑中测试发现，Java8和Java17在并发场景下性能差距达到2倍以上

并且同样的逻辑（使用了并发安全的工具）在并发量达到50000的时候在Java8中依然有线程安全问题，而Java17中不会有线程安全问题

外部测评结果

从规划调度引擎 OptaPlanner 项目（原文 (opens new window)）对 JDK 17和 JDK 11 的性能基准测试进行了对比来看：

对于 G1GC（默认），Java 17 比 Java 11 快 8.66%；

对于 ParallelGC，Java 17 比 Java 11 快 6.54%；

Parallel GC 整体比 G1 GC 快 16.39%

简而言之，JDK17 更快，高吞吐量垃圾回收器比低延迟垃圾回收器更快。

GraphQL实践2——Spring-GraphQL集成JPA与MySQL

Mon, 15 Nov 2021 15:38:00 +0800

介绍

上一篇介绍到用第三方库集成GraphQL，目前spring-graphql项目已经出到1.0.0-M3版本，属于内部预览版，此处尝鲜验证

引入依赖

由于SpringBoot 2.6.0还未发布，因此需要引入较多依赖

<dependencies>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-web</artifactId>
 </dependency>
 <dependency>
 <groupId>org.projectlombok</groupId>
 <artifactId>lombok</artifactId>
 <optional>true</optional>
 </dependency>
 <dependency>
 <groupId>org.springframework.experimental</groupId>
 <artifactId>graphql-spring-boot-starter</artifactId>
 <version>1.0.0-M3</version>
 </dependency>
 <!--升级到Spring Boot 2.6后可以移除-->
 <dependency>
 <groupId>org.springframework.data</groupId>
 <artifactId>spring-data-commons</artifactId>
 <version>2.6.0-RC1</version>
 </dependency>
 <dependency>
 <groupId>org.springframework.data</groupId>
 <artifactId>spring-data-jpa</artifactId>
 <version>2.6.0-RC1</version>
 </dependency>
 <!-- https://mvnrepository.com/artifact/org.hibernate.javax.persistence/hibernate-jpa-2.1-api -->
 <dependency>
 <groupId>org.hibernate.javax.persistence</groupId>
 <artifactId>hibernate-jpa-2.1-api</artifactId>
 <version>1.0.2.Final</version>
 </dependency>
 <!-- https://mvnrepository.com/artifact/com.querydsl/querydsl-jpa -->
 <dependency>
 <groupId>com.querydsl</groupId>
 <artifactId>querydsl-jpa</artifactId>
 <version>4.4.0</version>
 </dependency>
 <!-- https://mvnrepository.com/artifact/com.querydsl/querydsl-apt -->
 <dependency>
 <groupId>com.querydsl</groupId>
 <artifactId>querydsl-apt</artifactId>
 <version>4.4.0</version>
 </dependency>
 <!-- https://mvnrepository.com/artifact/javax.annotation/javax.annotation-api -->
 <dependency>
 <groupId>javax.annotation</groupId>
 <artifactId>javax.annotation-api</artifactId>
 </dependency>
 <dependency>
 <groupId>com.querydsl</groupId>
 <artifactId>querydsl-core</artifactId>
 </dependency>
 <dependency>
 <groupId>com.querydsl</groupId>
 <artifactId>querydsl-jpa</artifactId>
 </dependency>
 <dependency>
 <groupId>mysql</groupId>
 <artifactId>mysql-connector-java</artifactId>
 <scope>runtime</scope>
 </dependency>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-data-jpa</artifactId>
 </dependency>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-test</artifactId>
 <scope>test</scope>
 </dependency>
</dependencies>

引入里程碑仓库

由于还没有GA，所以要引入里程碑仓库

<repositories>
 <repository>
 <id>spring-milestones</id>
 <name>Spring Milestones</name>
 <url>https://repo.spring.io/milestone</url>
 </repository>
 <repository>
 <id>spring-snapshots</id>
 <name>Spring Snapshots</name>
 <url>https://repo.spring.io/snapshot</url>
 <snapshots>
 <enabled>true</enabled>
 </snapshots>
 </repository>
</repositories>

引入插件

由于使用了dsl动态生成，还要引入注解处理器

<build>
 <plugins>
 <plugin>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-maven-plugin</artifactId>
 <configuration>
 <excludes>
 <exclude>
 <groupId>org.projectlombok</groupId>
 <artifactId>lombok</artifactId>
 </exclude>
 </excludes>
 </configuration>
 </plugin>
 <!-- added for QueryDsl-->
 <plugin>
 <groupId>com.mysema.maven</groupId>
 <artifactId>apt-maven-plugin</artifactId>
 <version>1.1.3</version>
 <executions>
 <execution>
 <goals>
 <goal>process</goal>
 </goals>
 <configuration>
 <outputDirectory>target/generated-sources/java</outputDirectory>
 <processor>com.querydsl.apt.jpa.JPAAnnotationProcessor</processor>
 </configuration>
 </execution>
 </executions>
 </plugin>
 </plugins>
</build>

SpringBoot 配置

编写配置文件

spring:
 datasource:
 driver-class-name: com.mysql.cj.jdbc.Driver
 username: root
 password: "****"
 url: jdbc:mysql://localhost:3306/sakila?useSSL=false&useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai
 jpa:
 hibernate:
 ddl-auto: none
 show-sql: true
 database-platform: org.hibernate.dialect.MySQL55Dialect
 properties:
 hibernate:
 format_sql: true
 use_sql_comments: true
 graphql:
 path: /graphql
 schema:
 locations: classpath:graphql/
 fileExtensions: .graphqls, .gqls
 printer:
 enabled: true
server:
 port: 8080

实体类

package top.fjy8018.graphsqldemo.entity;

import lombok.Data;

import javax.persistence.*;
import java.sql.Timestamp;
import java.util.Objects;

/**
 * 演员表实体类
 *
 * @author F嘉阳
 * @date 2021/11/5 10:34
 */
@Data
@Entity
@Table(name = "actor", schema = "sakila")
public class ActorEntity {
 @Id
 @Column(name = "actor_id", nullable = false)
 private Integer actorId;

 @Column(name = "first_name", nullable = false, length = 45)
 private String firstName;

 @Column(name = "last_name", nullable = false, length = 45)
 private String lastName;

 @Column(name = "last_update", nullable = false)
 private Date lastUpdate;

}

DAO

DAO直接继承dsl相关处理器，可自动配对增删改查方法，相比第三方库更加简便

package top.fjy8018.graphsqldemo.repository;

import org.springframework.data.querydsl.QuerydslPredicateExecutor;
import org.springframework.data.repository.CrudRepository;
import org.springframework.graphql.data.GraphQlRepository;
import top.fjy8018.graphsqldemo.entity.ActorEntity;

/**
 * 演员表DAO
 *
 * @author F嘉阳
 * @date 2021/11/5 10:35
 */
@GraphQlRepository
public interface ActorEntityRepository extends
 CrudRepository<ActorEntity, Integer>, QuerydslPredicateExecutor<ActorEntity> {
}

Graphql资源定义

在resources/graphql下定义接口和实体文件

接口声明和实体定义schema.graphqls

type Query {
 findOneActor(actorId : ID!): ActorEntity
 actorList: [ActorEntity]
}

type ActorEntity {
 actorId: ID!
 firstName: String!
 lastName: String!
 lastUpdate: String
}

接口定义acotrEntity.graphql

query findOneActor($id: ID!) {
 findOneActor(actorId: $id) {
 actorId
 firstName
 lastName
 lastUpdate
 }
}

query actorList {
 actorList{
 actorId
 }
}

启动测试

Spring-Graphql默认不包含可视化界面，此处使用postman进行测试

唯一查询

列表查询

总结

使用Spring官方组件好处在于和Spring生态集成度很高，如果本身就采用Jpa方式进行业务开发，迁移更加方便，需要开发的代码也很少。

样例源码地址

FJiayang/graphql-demo

GraphQL实践1——集成JPA与MySQL

Fri, 05 Nov 2021 17:41:00 +0800

介绍

官方定义：

GraphQL 既是一种用于 API 的查询语言也是一个满足你数据查询的运行时。 GraphQL 对你的 API 中的数据提供了一套易于理解的完整描述，使得客户端能够准确地获得它需要的数据，而且没有任何冗余，也让 API 更容易地随着时间推移而演进，还能用于构建强大的开发者工具。

中文官网：

GraphQL — A query language for your API

集成方案选择

GraphQL经过近几年的发展，SpringBoot官方、第三方都提供了starter

SpringBoot官方提供的starter

Spring GraphQL

第三方提供的starter

graphql-spring-boot

两者活跃度都还不错，但考虑到SpringBoot官方的依赖库还未GA，而且使用起来与Spring家族耦合过大，最终决定采用第三方的starter

集成过程

数据库配置

此处采用MySQL数据库，数据内容采用MySQL官方样例sakila数据，表结构和数据地址如下

sakila database下载地址

导入后如下

工程配置

引入最新的SpringBoot框架，依赖如下

<dependencies>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-web</artifactId>
 </dependency>

 <dependency>
 <groupId>org.projectlombok</groupId>
 <artifactId>lombok</artifactId>
 <optional>true</optional>
 </dependency>

 <!-- https://mvnrepository.com/artifact/com.graphql-java-kickstart/graphql-spring-boot-starter -->
 <dependency>
 <groupId>com.graphql-java-kickstart</groupId>
 <artifactId>graphql-spring-boot-starter</artifactId>
 <version>12.0.0</version>
 </dependency>

 <dependency>
 <groupId>mysql</groupId>
 <artifactId>mysql-connector-java</artifactId>
 <scope>runtime</scope>
 </dependency>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-data-jpa</artifactId>
 </dependency>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-test</artifactId>
 <scope>test</scope>
 </dependency>
</dependencies>

增加数据库配置

spring:
 datasource:
 driver-class-name: com.mysql.cj.jdbc.Driver
 username: root
 password: "****"
 url: jdbc:mysql://localhost:3306/sakila?useSSL=false&useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai
 jpa:
 hibernate:
 ddl-auto: update
 show-sql: true
 database-platform: org.hibernate.dialect.MySQL55Dialect
 properties:
 hibernate:
 format_sql: true
 use_sql_comments: true
server:
 port: 8080
graphql:
 tools:
 schema-location-pattern: graphql/*.graphqls
 graphiql:
 # 开启可交互页面，用于调试
 enabled: true

实体映射

新增实体映射，此处只映射Actor表作为样例

package top.fjy8018.graphsqldemo.entity;

import lombok.Data;

import javax.persistence.*;
import java.sql.Timestamp;
import java.util.Objects;

/**
 * 演员表实体映射
 *
 * @author F嘉阳
 * @date 2021/11/5 10:34
 */
@Data
@Entity
@Table(name = "actor", schema = "sakila")
public class ActorEntity {
 @Id
 @Column(name = "actor_id", nullable = false)
 private Integer actorId;

 @Column(name = "first_name", nullable = false, length = 45)
 private String firstName;

 @Column(name = "last_name", nullable = false, length = 45)
 private String lastName;

 @Column(name = "last_update", nullable = false)
 private Timestamp lastUpdate;
}

新增DAO

package top.fjy8018.graphsqldemo.repository;

import org.springframework.data.jpa.repository.JpaRepository;
import top.fjy8018.graphsqldemo.entity.ActorEntity;

/**
 * 演员表DAO
 *
 * @author F嘉阳
 * @date 2021/11/5 10:35
 */
public interface ActorEntityRepository extends JpaRepository<ActorEntity, Integer> {
}

新增查询接口配置

在resources/graphql下新增文件actor.graphqls

type Query {
 actorList: [ActorEntity]
 findOneActor(id : ID!): ActorEntity
}

type ActorEntity {
 actorId: ID!
 firstName: String!
 lastName: String!
 lastUpdate: String
}

新增查询解析器

package top.fjy8018.graphsqldemo.resolver;

import graphql.kickstart.tools.GraphQLQueryResolver;
import lombok.AllArgsConstructor;
import org.springframework.stereotype.Component;
import top.fjy8018.graphsqldemo.entity.ActorEntity;
import top.fjy8018.graphsqldemo.repository.ActorEntityRepository;

import java.util.Collection;

/**
 * 演员表GraphQL查询解析器
 *
 * @author F嘉阳
 * @date 2021/11/5 16:27
 */
@Component
@AllArgsConstructor
public class ActorGraphQLQueryResolver implements GraphQLQueryResolver {

 private final ActorEntityRepository actorRepository;

 public Collection<ActorEntity> actorList() {
 return actorRepository.findAll();
 }

 public ActorEntity findOneActor(Integer id) {
 return actorRepository.findById(id).orElse(null);
 }
}

启动验证

启动工程后，浏览器打开http://localhost:8080/graphiql

列表查询

由于采用交互式页面，输入也会有语法提示，比较方便

唯一查询

优缺点分析

优点

接口定义清晰，配置即接口
前端按需获取字段，减少数据传输量
接口整合、演进更加便利

缺点

减少字段个数对后端实现没有变化，并不能提升数据库性能

样例源码地址

FJiayang/graphql-demo

Kubesphere安装——以GCP为例

Sat, 19 Jun 2021 22:15:00 +0800

平台准备

创建工作空间

启用计算引擎API

准备虚拟机

作为演示环境，此处使用Google Cloud Platform创建3台虚拟机，配置如下

CPU	内存	磁盘
4	16G	40G
4	16G	40G
4	16G	40G

创建实例模板

创建需要的实例模板

配置操作系统、资源等，注意放行防火墙

创建实例

基于实例模板创建3台虚拟机，选择地区为香港

此处用于演示用途，可选择无状态机器，机器删除后不会保留磁盘内容

注意关闭自动扩缩容

等待实例创建

由于资源调度问题，最终只成功分配两台虚拟机，不过对于演示环境安装过程一样。

配置连接秘钥

添加本地生成好的rsa秘钥即可

本地SSH工具配置私钥即可连接

前置环境准备

kubesphere建议前置项

节点要求

所有节点必须都能通过 SSH 访问。

所有节点时间同步。

所有节点都应使用 sudo/curl/openssl。

依赖项要求
KubeKey 可以一同安装 Kubernetes 和 KubeSphere。根据要安装的 Kubernetes 版本，需要安装的依赖项可能会不同。您可以参考下表，查看是否需要提前在节点上安装相关依赖项。

依赖项 Kubernetes 版本 ≥ 1.18 Kubernetes 版本 < 1.18

socat 必须可选，但建议安装

conntrack 必须可选，但建议安装

ebtables 可选，但建议安装可选，但建议安装

ipset 可选，但建议安装可选，但建议安装

依赖项	Kubernetes 版本 ≥ 1.18	Kubernetes 版本 < 1.18
`socat`	必须	可选，但建议安装
`conntrack`	必须	可选，但建议安装
`ebtables`	可选，但建议安装	可选，但建议安装
`ipset`	可选，但建议安装	可选，但建议安装

以下内容所有节点都需要执行

$ yum install -y socat curl openssl conntrack ebtables ipset

主节点下载kubekey，由于使用google虚拟机，能正常访问google和github，故直接下载即可

$ curl -sfL https://get-kk.kubesphere.io | VERSION=v1.1.0 sh -
$ chmod +x kk

放行VPC防火墙

创建集群

服务器免密登陆

主节点生成免密登陆配置

$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:yxln4RFEXhsvIMcL0/EzF8CLUpo/hMlkM8KBDcNEjbU root@kubesphere-instance-group-1-bx5d
The key's randomart image is:
+---[RSA 2048]----+
| +=Oo..=Bo+.. |
| +.=.B++=.+ . |
| E= @+o*.o |
| B.+o.= |
| S++ |
| . *o |
| + . |
| |
| |
+----[SHA256]-----+

所有主机允许公钥免密登陆

$ vi /etc/ssh/sshd_config
# 将下面配置改为yes并去掉注释
PubkeyAuthentication yes
PermitRootLogin yes

$ service sshd restart

控制台添加主节点公钥，则所有新建主机都能被主节点访问

创建示例配置文件

$./kk create config --with-kubernetes v1.20.4 --with-kubesphere v3.1.0 -f gcp-config

按默认配置只安装控制台和基础服务

apiVersion: kubekey.kubesphere.io/v1alpha1
kind: Cluster
metadata:
 name: sample
spec:
 hosts:
 - {name: node1, address: 34.92.90.181, internalAddress: 10.170.0.5, privateKeyPath: "/root/.ssh/id_rsa"}
 - {name: node2, address: 35.220.138.230, internalAddress: 10.170.0.6, privateKeyPath: "/root/.ssh/id_rsa"}
 roleGroups:
 etcd:
 - node1
 master: 
 - node1
 worker:
 - node1
 - node2
 controlPlaneEndpoint:
 domain: lb.kubesphere.local
 address: ""
 port: 6443
 kubernetes:
 version: v1.20.4
 imageRepo: kubesphere
 clusterName: cluster.local
 network:
 plugin: calico
 kubePodsCIDR: 10.233.64.0/18
 kubeServiceCIDR: 10.233.0.0/18
 registry:
 registryMirrors: []
 insecureRegistries: []
 addons: []


---
apiVersion: installer.kubesphere.io/v1alpha1
kind: ClusterConfiguration
metadata:
 name: ks-installer
 namespace: kubesphere-system
 labels:
 version: v3.1.0
spec:
 persistence:
 storageClass: "" 
 authentication:
 jwtSecret: ""
 zone: ""
 local_registry: "" 
 etcd:
 monitoring: false 
 endpointIps: localhost 
 port: 2379 
 tlsEnable: true
 common:
 redis:
 enabled: false
 redisVolumSize: 2Gi 
 openldap:
 enabled: false
 openldapVolumeSize: 2Gi 
 minioVolumeSize: 20Gi
 monitoring:
 endpoint: http://prometheus-operated.kubesphere-monitoring-system.svc:9090
 es: 
 elasticsearchMasterVolumeSize: 4Gi 
 elasticsearchDataVolumeSize: 20Gi 
 logMaxAge: 7 
 elkPrefix: logstash
 basicAuth:
 enabled: false
 username: ""
 password: ""
 externalElasticsearchUrl: ""
 externalElasticsearchPort: "" 
 console:
 enableMultiLogin: true 
 port: 30880
 alerting: 
 enabled: false
 # thanosruler:
 # replicas: 1
 # resources: {}
 auditing: 
 enabled: false
 devops: 
 enabled: false
 jenkinsMemoryLim: 2Gi 
 jenkinsMemoryReq: 1500Mi 
 jenkinsVolumeSize: 8Gi 
 jenkinsJavaOpts_Xms: 512m 
 jenkinsJavaOpts_Xmx: 512m
 jenkinsJavaOpts_MaxRAM: 2g
 events: 
 enabled: false
 ruler:
 enabled: true
 replicas: 2
 logging: 
 enabled: false
 logsidecar:
 enabled: true
 replicas: 2
 metrics_server: 
 enabled: false
 monitoring:
 storageClass: ""
 prometheusMemoryRequest: 400Mi 
 prometheusVolumeSize: 20Gi 
 multicluster:
 clusterRole: none 
 network:
 networkpolicy:
 enabled: false
 ippool:
 type: none
 topology:
 type: none
 notification: 
 enabled: false
 openpitrix:
 store:
 enabled: false
 servicemesh: 
 enabled: false 
 kubeedge:
 enabled: false
 cloudCore:
 nodeSelector: {"node-role.kubernetes.io/worker": ""}
 tolerations: []
 cloudhubPort: "10000"
 cloudhubQuicPort: "10001"
 cloudhubHttpsPort: "10002"
 cloudstreamPort: "10003"
 tunnelPort: "10004"
 cloudHub:
 advertiseAddress: 
 - "" 
 nodeLimit: "100"
 service:
 cloudhubNodePort: "30000"
 cloudhubQuicNodePort: "30001"
 cloudhubHttpsNodePort: "30002"
 cloudstreamNodePort: "30003"
 tunnelNodePort: "30004"
 edgeWatcher:
 nodeSelector: {"node-role.kubernetes.io/worker": ""}
 tolerations: []
 edgeWatcherAgent:
 nodeSelector: {"node-role.kubernetes.io/worker": ""}
 tolerations: []

安装集群

此时，在虚拟机安装代理即可查看虚拟机监控

创建完成

访问成功

集群正常，不安装组件时内存占用率并不高

安装kubectl

$ yum install -y kubectl

安装可插拔组件

点击自定义资源 CRD，然后在搜索栏中输入 clusterconfiguration，点击搜索结果进入其详情页面。

按照自身需要启用可插拔组件，此处打开所有组件，即修改为如下配置

apiVersion: installer.kubesphere.io/v1alpha1
kind: ClusterConfiguration
metadata:
 labels:
 version: v3.1.0
 name: ks-installer
 namespace: kubesphere-system
spec:
 alerting:
 enabled: true
 auditing:
 enabled: true
 authentication:
 jwtSecret: ''
 common:
 es:
 basicAuth:
 enabled: false
 password: ''
 username: ''
 elasticsearchDataVolumeSize: 20Gi
 elasticsearchMasterVolumeSize: 4Gi
 elkPrefix: logstash
 externalElasticsearchPort: ''
 externalElasticsearchUrl: ''
 logMaxAge: 7
 minioVolumeSize: 2Gi
 monitoring:
 endpoint: 'http://prometheus-operated.kubesphere-monitoring-system.svc:9090'
 openldap:
 enabled: true
 openldapVolumeSize: 2Gi
 redis:
 enabled: true
 redisVolumSize: 2Gi
 console:
 enableMultiLogin: true
 port: 30880
 devops:
 enabled: true
 jenkinsJavaOpts_MaxRAM: 2g
 jenkinsJavaOpts_Xms: 512m
 jenkinsJavaOpts_Xmx: 512m
 jenkinsMemoryLim: 2Gi
 jenkinsMemoryReq: 1500Mi
 jenkinsVolumeSize: 8Gi
 etcd:
 endpointIps: 10.170.0.5
 monitoring: true
 port: 2379
 tlsEnable: true
 events:
 enabled: true
 ruler:
 enabled: true
 replicas: 2
 kubeedge:
 cloudCore:
 cloudHub:
 advertiseAddress:
 - ''
 nodeLimit: '100'
 cloudhubHttpsPort: '10002'
 cloudhubPort: '10000'
 cloudhubQuicPort: '10001'
 cloudstreamPort: '10003'
 nodeSelector:
 node-role.kubernetes.io/worker: ''
 service:
 cloudhubHttpsNodePort: '30002'
 cloudhubNodePort: '30000'
 cloudhubQuicNodePort: '30001'
 cloudstreamNodePort: '30003'
 tunnelNodePort: '30004'
 tolerations: []
 tunnelPort: '10004'
 edgeWatcher:
 edgeWatcherAgent:
 nodeSelector:
 node-role.kubernetes.io/worker: ''
 tolerations: []
 nodeSelector:
 node-role.kubernetes.io/worker: ''
 tolerations: []
 enabled: true
 logging:
 enabled: true
 logsidecar:
 enabled: true
 replicas: 2
 metrics_server:
 enabled: true
 monitoring:
 prometheusMemoryRequest: 400Mi
 prometheusVolumeSize: 5Gi
 storageClass: ''
 multicluster:
 clusterRole: none
 network:
 ippool:
 type: calico
 networkpolicy:
 enabled: true
 topology:
 type: weave-scope
 openpitrix:
 store:
 enabled: true
 persistence:
 storageClass: ''
 servicemesh:
 enabled: true

通过在线控制台，查看安装进度

执行以下命令

$ kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

安装组件过程中，对CPU负载较大

所有组件安装完成

资源占用情况

Kubesphere运维Kubernetes集群

Sun, 23 May 2021 22:58:00 +0800

背景

使用原生Kubernetes-dashboard和Grafana+Prometheus管理较大规模集群比较困难，有如下缺陷

每次执行命令都要登陆服务器
容器日志采集方案、监控方案搭建比较繁琐
添加节点不方便
管理存储插件、秘钥、配置不方便

解决方案

目前国内流行的k8s管理平台主要有rancher、kubesphere、kubeboard

rancher主打原生集群管理，侧重运维人员
kubesphere主打开发运维工作流，侧重研发人员
kubeboard与kubesphere类似，侧重微服务管理，但功能上较少，对多节点管理需要收费

经过评估，最终决定采用kubesphere进行集群管理

Rancher — 多云混合云多集群Kubernetes管理平台

[post url=“https://kubesphere.io/zh/” title=“Kubesphere” intro=“面向云原生应用的容器混合云” cover=“https://kubesphere.io/images/logo.svg” /]

Kuboard — 快速在 Kubernetes 上落地微服务

安装步骤

目标是面对小规模集群管理，节点数最多为5个，故不考虑高可用集群配置，采用单主节点+4个worker节点配置

采用纯净CentOS 7 Linux系统，需要做以下前置准备

依赖包安装

安装官方要求的依赖包，由于计划安装NFS存储插件，还需要安装nfs工具包

$ yum install -y socat conntrack ebtables ipset nfs-utils

配置DNS解析

将各节点DNS配置为私有DNS服务器

$ tee /etc/resolv.conf <<-'EOF'
# Generated by NetworkManager
nameserver 192.168.1.13
EOF

私有DNS搭建可以参考文章

[[利用Docker自建Dns服务器]]

按照官方教程安装多节点kubernetes

强烈建议先用KubeKey安装纯净k8s集群+控制面板，关闭所有插件。等安装完毕再一个个插件打开，防止因为内存不够而无法打开kubesphere

安装步骤参考官方文档一键安装方式

多节点安装 — Kubesphere

启动成功

启动成功后按照官方指南打开插件

启用可插拔组件 — Kubesphere

所有插件打开后如下

使用

节点状态

存储卷管理

监控告警

日志采集

日志查询

操作审计

工具箱

企业空间权限控制

内置helm应用商店，支持上线审批，在线编辑等

支持关联外部仓库

应用流量治理

链路追踪

灰度发布

费用统计

更多的使用场景包括：DevOps、边缘节点、多集群管理还在探索中

SpringCloudStream函数式编程模型

Sun, 21 Feb 2021 12:22:00 +0800

背景

SpringCloud 3.1版本之后，@EnableBinding、@Output等StreamApi注解都标记为废弃

/**
 * Indicates that an output binding target will be created by the framework.
 *
 * @author Dave Syer
 * @author Marius Bogoevici
 * @author Artem Bilan
 *
 * @deprecated as of 3.1 in favor of functional programming model
 */

@Qualifier
@Target({ ElementType.FIELD, ElementType.METHOD, ElementType.ANNOTATION_TYPE,
		ElementType.PARAMETER })
@Retention(RetentionPolicy.RUNTIME)
@Inherited
@Documented
@Deprecated
public @interface Output {

	/**
	 * Specify the binding target name; used as a bean name for binding target and as a
	 * destination name by default.
	 * @return the binding target name
	 */
	String value() default "";

}

/**
 * Enables the binding of targets annotated with {@link Input} and {@link Output} to a
 * broker, according to the list of interfaces passed as value to the annotation.
 *
 * @author Dave Syer
 * @author Marius Bogoevici
 * @author David Turanski
 * @author Soby Chacko
 *
 * @deprecated as of 3.1 in favor of functional programming model
 */
@Target({ ElementType.TYPE, ElementType.ANNOTATION_TYPE })
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
@Configuration
@Import({ BindingBeansRegistrar.class, BinderFactoryAutoConfiguration.class })
@EnableIntegration
@Deprecated
public @interface EnableBinding {

	/**
	 * A list of interfaces having methods annotated with {@link Input} and/or
	 * {@link Output} to indicate binding targets.
	 * @return list of interfaces
	 */
	Class<?>[] value() default {};

}

Producing and Consuming Messages — SpringCloud 官方文档

@EnableBinding @deprecated as of 3.1 in favor of functional programming model — stackoverflow

The type EnableBinding is deprecated — stackoverflow

因此，所有采用命令式编程的API，在升级到新版本时，官方推荐使用更加简单的函数式编程API

完整代码样例

FJiayang/spring-cloud-stream-rabbit-example — GitHub项目地址

升级方式

命令式编程模型

以简单多Topic消息为例，原本的编程方式如下

生产者

@SpringBootApplication
public class ProducerApplication {

 public static void main(String[] args) {
 SpringApplication.run(ProducerApplication.class, args);
 }
}

/**
 * @author F嘉阳
 * @date 2018-10-08 17:57
 */
@RestController
@EnableBinding(MySource.class)
public class Producer {

 @Autowired
 private MySource channel;

 @RequestMapping("/send")
 public String send() {
 channel.output().send(MessageBuilder.withPayload(new Date()).build());
 return "success";
 }
}

/**
 * @author F嘉阳
 * @date 2018-10-08 18:01
 */
public interface MySource {
 String OUTPUT = "output";

 @Output(MySource.OUTPUT)
 MessageChannel output();
}

配置文件

spring:
 rabbitmq:
 host: 192.168.163.128
 username: cms
 password: cms-mq-admin
 cloud:
 stream:
 bindings:
 output:
 destination: my-test-channel
server:
 port: 8082

消费者

@SpringBootApplication
public class ConsumerApplication {

 public static void main(String[] args) {
 SpringApplication.run(ConsumerApplication.class, args);
 }
}

/**
 * @author F嘉阳
 * @date 2018-10-08 18:10
 */
@EnableBinding(MySink.class)
public class Consumer {
 @StreamListener(MySink.INPUT)
 public void receive(Message<String> message) {
 System.out.println("接收到MQ消息:" + message.getPayload());
 }
}

/**
 * @author F嘉阳
 * @date 2018-10-08 18:07
 */
public interface MySink {
 String INPUT = "input";

 @Input(MySink.INPUT)
 SubscribableChannel input();
}

配置文件

spring:
 rabbitmq:
 host: 192.168.163.128
 username: cms
 password: cms-mq-admin
 cloud:
 stream:
 bindings:
 input:
 destination: my-test-channel
server:
 port: 8081

函数式编程模型

从命令式升级到函数式编程，首先需要升级SpringCloud版本至3.1以上，此处升级到最新版2020.0.1

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
 <modelVersion>4.0.0</modelVersion>

 <groupId>top.fjy8018</groupId>
 <artifactId>cloud-stream</artifactId>
 <version>0.0.1-SNAPSHOT</version>
 <packaging>pom</packaging>

 <name>cloud-stream</name>
 <description>Demo project for Spring Boot</description>

 <modules>
 <module>producer</module>
 <module>consumer</module>
 </modules>

 <parent>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-parent</artifactId>
 <version>2.4.2</version>
 <relativePath/> <!-- lookup parent from repository -->
 </parent>

 <properties>
 <spring-cloud.version>2020.0.1</spring-cloud.version>
 </properties>

 <dependencyManagement>
 <dependencies>
 <dependency>
 <groupId>org.springframework.cloud</groupId>
 <artifactId>spring-cloud-dependencies</artifactId>
 <version>${spring-cloud.version}</version>
 <type>pom</type>
 <scope>import</scope>
 </dependency>
 </dependencies>
 </dependencyManagement>
</project>

目前国内有清晰的升级指引的还比较少，目前只能参考官方文档材料

Programming Model — SpringCloud官方文档

模型1——系统定时驱动的消息队列

官方文档描述

Suppliers (Sources)
Function and Consumer are pretty straightforward when it comes to how their invocation is triggered. They are triggered based on data (events) sent to the destination they are bound to. In other words, they are classic event-driven components.

However, Supplier is in its own category when it comes to triggering. Since it is, by definition, the source (the origin) of the data, it does not subscribe to any in-bound destination and, therefore, has to be triggered by some other mechanism(s). There is also a question of Supplier implementation, which could be imperative or reactive and which directly relates to the triggering of such suppliers.

Consider the following sample:
@SpringBootApplication
public static class SupplierConfiguration {

	@Bean
	public Supplier<String> stringSupplier() {
		return () -> "Hello from Supplier";
	}
}
The preceding Supplier bean produces a string whenever its get() method is invoked. However, who invokes this method and how often? The framework provides a default polling mechanism (answering the question of “Who?”) that will trigger the invocation of the supplier and by default it will do so every second (answering the question of “How often?”). In other words, the above configuration produces a single message every second and each message is sent to an output destination that is exposed by the binder.

其大意为，若采用默认的函数式编程模型，会每秒钟发送一条消息到消费者，定时发送参数可以配置

按照文档升级后如下

生产者

@SpringBootApplication
public class ProducerApplication {

 public static void main(String[] args) {
 SpringApplication.run(ProducerApplication.class, args);
 }

 @Bean
 public Supplier<Date> source1() {
 return () -> new Date();
 }
}

函数式编程不需要在代码中定义监听队列，只需要按照约定写在配置文件中即可

官方文档描述

Functional binding names
Unlike the explicit naming required by annotation-based support (legacy) used in the previous versions of spring-cloud-stream, the functional programming model defaults to a simple convention when it comes to binding names, thus greatly simplifying application configuration. Let’s look at the first example:
@SpringBootApplication
public class SampleApplication {

	@Bean
	public Function<String, String> uppercase() {
	 return value -> value.toUpperCase();
	}
}
In the preceding example we have an application with a single function which acts as message handler. As a Function it has an input and output. The naming convention used to name input and output bindings is as follows:

input - <functionName> + -in- + <index>

output - <functionName> + -out- + <index>

The in and out corresponds to the type of binding (such as input or output). The index is the index of the input or output binding. It is always 0 for typical single input/output function, so it’s only relevant for Functions with multiple input and output arguments.

So if for example you would want to map the input of this function to a remote destination (e.g., topic, queue etc) called “my-topic” you would do so with the following property:
--spring.cloud.stream.bindings.uppercase-in-0.destination=my-topic
Note how uppercase-in-0 is used as a segment in property name. The same goes for uppercase-out-0.

配置文件

spring:
 rabbitmq:
 host: 192.168.163.128
 username: cms
 password: cms-mq-admin

 cloud:
 stream:
 bindings:
 source1-out-0:
 destination: test1
 function:
 definition: source1

server:
 port: 8083

由此可见，整体配置简化了很多，多为规约大于配置的方式进行定义，特别注意function配置的位置，是在stream配置的同级

消费者

@SpringBootApplication
public class ConsumerApplication {

 public static void main(String[] args) {
 SpringApplication.run(ConsumerApplication.class, args);
 }

 @Bean
 public Consumer<Date> sink1() {
 return System.out::println;
 }
}

配置文件同理

spring:
 rabbitmq:
 host: 192.168.163.128
 username: cms
 password: cms-mq-admin

 cloud:
 stream:
 bindings:
 sink1-in-0:
 destination: test1
 function:
 definition: sink1


server:
 port: 8081

运行结果

生产者每隔一秒发送一次消息

模型2——`StreamBridge`

但实际生产中，更多是由业务场景触发，故不能使用该模式进行配置，SpringCloud还提供了另外一种StreamBridge模式

官方文档介绍

Sending arbitrary data to an output (e.g. Foreign event-driven sources)
There are cases where the actual source of data may be coming from the external (foreign) system that is not a binder. For example, the source of the data may be a classic REST endpoint. How do we bridge such source with the functional mechanism used by spring-cloud-stream?

Spring Cloud Stream provides two mechanisms, so let’s look at them in more details

Here, for both samples we’ll use a standard MVC endpoint method called delegateToSupplier bound to the root web context, delegating incoming requests to stream via two different mechanisms - imperative (via StreamBridge) and reactive (via EmitterProcessor).

大意是，实际的数据源可以是外部事件驱动，例如通过Rest接口

官方样例

Using StreamBridge
@SpringBootApplication
@Controller
public class WebSourceApplication {

	public static void main(String[] args) {
		SpringApplication.run(WebSourceApplication.class, "--spring.cloud.stream.source=toStream");
	}

	@Autowired
	private StreamBridge streamBridge;

	@RequestMapping
	@ResponseStatus(HttpStatus.ACCEPTED)
	public void delegateToSupplier(@RequestBody String body) {
		System.out.println("Sending " + body);
		streamBridge.send("toStream-out-0", body);
	}
}
Here we autowire a StreamBridge bean which allows us to send data to an output binding effectively bridging non-stream application with spring-cloud-stream. Note that preceding example does not have any source functions defined (e.g., Supplier bean) leaving the framework with no trigger to create source bindings, which would be typical for cases where configuration contains function beans. So to trigger the creation of source binding we use spring.cloud.stream.source property where you can declare the name of your sources. The provided name will be used as a trigger to create a source binding. So in the preceding example the name of the output binding will be toStream-out-0 which is consistent with the binding naming convention used by functions (see Binding and Binding names). You can use ; to signify multiple sources (e.g., --spring.cloud.stream.source=foo;bar)

参考官方文档改造后

生产者

这里直接贴出多主题的消费模式，实际生产中多数也是订阅多个主题

@SpringBootApplication
public class ProducerApplication {

 public static void main(String[] args) {
 SpringApplication.run(ProducerApplication.class, args);
 }
}

/**
 * @author F嘉阳
 * @date 2018-10-08 17:57
 */
@RestController
public class Producer {

 @Autowired
 private StreamBridge streamBridge;

 @RequestMapping("/send1")
 public String send1() {
 streamBridge.send("source1-out-0", new Date());
 return "success1";
 }

 @RequestMapping("/send2")
 public String send2() {
 streamBridge.send("source2-out-0", LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_DATE_TIME));
 return "success2";
 }

}

配置文件

spring:
 rabbitmq:
 host: 192.168.133.128
 username: dev-user
 password: devpassword

 cloud:
 stream:
 bindings:
 source1-out-0:
 destination: test2
 source2-out-0:
 destination: test3
 function:
 definition: source1;source2


server:
 port: 8083

消费者

@SpringBootApplication
public class ConsumerApplication {

 public static void main(String[] args) {
 SpringApplication.run(ConsumerApplication.class, args);
 }

 @Bean
 public Consumer<Date> sink1() {
 return System.out::println;
 }

 @Bean
 public Consumer<String> sink2() {
 return System.out::println;
 }
}

配置文件

spring:
 rabbitmq:
 host: 192.168.133.128
 username: dev-user
 password: devpassword

 cloud:
 stream:
 bindings:
 sink1-in-0:
 destination: test2
 sink2-in-0:
 destination: test3
 function:
 definition: sink1;sink2

server:
 port: 8081

运行结果

触发/send1

触发/send2

可见消息被正确的路由

总结

整体来看，函数式编程比命令式编程更加简洁，而且加上SpringCloud规约大于配置的特点，可以明显减少配置的代码量，是未来发展的趋势。

Mapstruct爬坑记录

Sat, 20 Feb 2021 22:45:11 +0800

背景

众所周知，Spring框架提供了著名的类拷贝工具BeanUtils，其使用简单、方便。

但是其内部实现基于运行时反射进行同名属性拷贝，对于需要进行字段映射和高性能运行时场景都无法满足要求，因此，开源社区衍生出了以下的类拷贝框架

使用

目前第二代密码管理平台采用的是mica重写后的BeanUtil类拷贝方法，整体来说性能可以满足要求，但为了进一步提升查询时类转换性能，同时考虑到应用中本身就包含了swagger依赖，而swagger中本身就引入了mapstruct依赖包，故无需引入新依赖。

因此，最终考虑将业务场景下的类拷贝改成Mapstruct实现，由此开始mapstruct爬坑之路。。

软件包版本

SpringBoot:2.4.2

SpringCloud:2020.0.1

lombok: 1.18.16

mapstruct: 1.3.1.Final

新增依赖

由于需要由mapstruct根据定义的接口自动生成类转换实现，故需要引入mapstruct接口处理包，有两种方式，一种通过maven插件的形式，一种通过maven依赖的形式

maven插件的形式（官方推荐）

<build>
 <plugins>
 <plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-compiler-plugin</artifactId>
 <version>3.5.1</version>
 <configuration>
 <source>1.8</source>
 <target>1.8</target>
 <annotationProcessorPaths>
 <path>
 <groupId>org.mapstruct</groupId>
 <artifactId>mapstruct-processor</artifactId>
 <version>${org.mapstruct.version}</version>
 </path>
 </annotationProcessorPaths>
 </configuration>
 </plugin>
 </plugins>
</build>

maven依赖的形式

<dependency>
 <groupId>org.mapstruct</groupId>
 <artifactId>mapstruct-processor</artifactId>
 <version>${mapstruct.version}</version>
</dependency>

考虑到平台的maven模块数量较大，互相依赖关系比较复杂，遗漏插件依赖容易导致难以排查的bug，最终采用第二种方式，减少编码量。

新增映射类

/**
 * @author F嘉阳
 * @date 2021/2/18 12:06
 */
@Mapper(componentModel = "spring", unmappedTargetPolicy = ReportingPolicy.IGNORE)
public interface OssEntityMapper {

	OssVO ossToOssVO(Oss oss);

	OssDTO ossToOssDto(Oss oss);

	/**
	 * bladeFile转Oss
	 *
	 * @param bladeFile
	 * @return
	 */
	@Mapping(target = "fileName", source = "name")
	Oss bladeFileToOss(BladeFile bladeFile);
}

新增单元测试

由于首次使用mapstruct，编写单元测试检测代码是否成功

/**
 * @author F嘉阳
 * @date 2021/2/18 14:23
 */
@Slf4j
@ExtendWith(BladeSpringExtension.class)
@SpringBootTest(classes = ResourceApplication.class)
@BladeBootTest(appName = AppConstant.APPLICATION_RESOURCE_NAME, profile = "test", enableLoader = true)
class OssEntityMapperTest {

	@Autowired
	private OssEntityMapper mapper;

	@Test
	void ossToOssVO() {
		Oss oss = new Oss();
		oss.setFileName("filename");
		oss.setLink("link");
		oss.setOriginalName("origin");
		oss.setOssType(OssType.QI_NIU_OSS);
		oss.setIsDeleted(BladeConstant.DB_NOT_DELETED);
		OssVO ossVO = mapper.ossToOssVO(oss);
		assertEquals("filename", ossVO.getFileName());
		assertEquals("link", ossVO.getLink());
		assertEquals("origin", ossVO.getOriginalName());
	}

	@Test
	void ossToOssDto() {
		Oss oss = new Oss();
		oss.setFileName("filename");
		oss.setLink("link");
		oss.setOriginalName("origin");
		oss.setOssType(OssType.QI_NIU_OSS);
		oss.setIsDeleted(BladeConstant.DB_NOT_DELETED);
		OssDTO ossDTO = mapper.ossToOssDto(oss);
		assertEquals("filename", ossDTO.getFileName());
		assertEquals("link", ossDTO.getLink());
		assertEquals("origin", ossDTO.getOriginalName());
	}

	@Test
	void bladeFileToOss() {
		BladeFile file = new BladeFile();
		file.setName("file");
		file.setLink("link");
		file.setOriginalName("origin");
		Oss oss = mapper.bladeFileToOss(file);
		assertEquals("file", oss.getFileName());
		assertEquals("link", oss.getLink());
		assertEquals("origin", oss.getOriginalName());
	}
}

编译结果

编译结果，报错

这就很奇怪了，这是按官网最简单的方式写的，而且IDEA也安装了mapstruct插件，也没报错，能正确识别属性，怎么编译就不通过了

查看生成的代码，确实没有属性映射

爬坑之路

由于编译报错，考虑以下方法解决

移除spring容器依赖，采用类引用的方式编写。——无效

移除属性映射，尝试编译，可以编译通过，但生成的代码没有同名属性映射，只会返回空值。——无效

@Override
public Oss bladeFileToOss(BladeFile bladeFile) {
 if ( bladeFile == null ) {
 return null;
 }

 Oss oss = new Oss();

 return oss;
}

更换类名后测试通过，多次测试仍然有不通过的情况。——可能和编译环境有关？
相同代码放在单独的工程（非maven多模块）测试通过。——可能和maven多模块依赖有关？
根据网上检索的结果，普遍反映的lombok版本太低导致的，经过检查，当前的lombok版本应该是满足最低版本要求的。——无效
考虑是否和spring某些组件冲突，单独新开maven多模块工程测试，依然测试不通过。——果然和maven有关？

解决方法

最终通过给官方仓库提交issue咨询得到答案

原来是lombok版本太高导致的。。

最终降低lombok版本问题解决。

Sentinel持久化存储和镜像构建

Sun, 07 Feb 2021 12:06:58 +0800

背景

Sentinel默认只能对限流规则存储到内存中

在生产环境中使用 Sentinel — Sentinel中文文档

官方文档对于数据持久化描述如下

我们提供了 ZooKeeper, Apollo, Nacos 等的动态数据源实现。以 ZooKeeper 为例子，如果要使用第三方的配置中心作为配置管理，您需要做下面的几件事情:

实现一个公共的 ZooKeeper 客户端用于推送规则，在 Sentinel 控制台配置项中需要指定 ZooKeeper 的地址，启动时即创建 ZooKeeper Client。

我们需要针对每个应用（appName），每种规则设置不同的 path（可随时修改）；或者约定大于配置（如 path 的模式统一为 /sentinel_rules/{appName}/{ruleType}，e.g. sentinel_rules/appA/flowRule）。

规则配置页需要进行相应的改造，直接针对应用维度进行规则配置；修改同个应用多个资源的规则时可以批量进行推送，也可以分别推送。Sentinel 控制台将规则缓存在内存中（如 InMemFlowRuleStore），可以对其进行改造使其支持应用维度的规则缓存（key 为 appName），每次添加/修改/删除规则都先更新内存中的规则缓存，然后需要推送的时候从规则缓存中获取全量规则，然后通过上面实现的 Client 将规则推送到 ZooKeeper 即可。

应用客户端需要注册对应的读数据源以监听变更，可以参考相关文档。

由于alibaba/Sentinel官方并没有实现对具体数据源的写入支持，只提供的实现的接口，故无法做到开箱即用

解决方法

已经有贡献者提供了对三种动态数据源的实现PR，但还没合并进主版本中

添加持久化支持，可选持久化选项 — Fixes Issues #1696 添加持久化支持，可选持久化选项：nacos、apollo、zookeeper ; 修改 README.md 说明文档

进一步完善

本次基于jiajiangnan/Sentinelfork出自己的工程FJiayang/Sentinel，实现docker镜像打包，并基于GitHub Actions完成自动化构建和上传制品。

jiajiangnan/Sentinel — jiajiangnan的Sentinel仓库源码

FJiayang/Sentinel — FJiayang的Sentinel仓库源码

添加Dockerfile

基于最小jdk镜像制作

FROM openjdk:8-jdk-alpine3.7

MAINTAINER fjy8018@gmail.com

ENV VERSION 1.8.0

RUN mkdir -p /sentinel

WORKDIR /sentinel

ADD ./sentinel-dashboard/target/sentinel-dashboard.jar ./app-${VERSION}.jar

# 设置时区，默认为UTC
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
RUN echo 'Asia/Shanghai' > /etc/timezone

CMD java ${PARAM} -jar app-${VERSION}.jar

添加GitHub流水线配置文件

name: Publish Docker image

on: [push]

jobs:
 build:

 runs-on: ubuntu-latest

 steps:
 - uses: actions/checkout@v2
 - name: Set up Maven Central Repository
 uses: actions/setup-java@v1
 with:
 java-version: 1.8

 - name: Publish package
 run: mvn --batch-mode clean package -Dmaven.test.skip=true

 - name: Push to Docker Hub
 uses: docker/build-push-action@v1
 with:
 username: ${{ secrets.DOCKER_USERNAME }}
 password: ${{ secrets.DOCKER_PASSWORD }}
 repository: fjy8018/sentinel
 tag_with_ref: true

具体的流水线编写可参考文章

使用方法

具体参数文档

项	类型	默认值	最小值	描述
sentinel.dashboard.auth.username	String	sentinel	无	登录控制台的用户名，默认为 `sentinel`
sentinel.dashboard.auth.password	String	sentinel	无	登录控制台的密码，默认为 `sentinel`
sentinel.dashboard.app.hideAppNoMachineMillis	Integer	0	60000	是否隐藏无健康节点的应用，距离最近一次主机心跳时间的毫秒数，默认关闭
sentinel.dashboard.removeAppNoMachineMillis	Integer	0	120000	是否自动删除无健康节点的应用，距离最近一次其下节点的心跳时间毫秒数，默认关闭
sentinel.dashboard.unhealthyMachineMillis	Integer	60000	30000	主机失联判定，不可关闭
sentinel.dashboard.autoRemoveMachineMillis	Integer	0	300000	距离最近心跳时间超过指定时间是否自动删除失联节点，默认关闭
datasource.provider	String	memory	无	默认为 `memory`, 可选持久化配置 `nacos`、`apollo`、`zookeeper`
datasource.provider.nacos.server-addr	String	localhost:8848	无	nacos 注册中心地址
datasource.provider.nacos.username	String		无	nacos 用户名，默认为空
datasource.provider.nacos.password	String		无	nacos 密码，默认为空
datasource.provider.nacos.namespace	String		无	nacos 名臣空间，默认为空
datasource.provider.nacos.group-id	String	SENTINEL_GROUP	无	nacos 分组，默认为 `SENTINEL_GROUP`
datasource.provider.apollo.server-addr	String	http://localhost:10034	无	apollo 注册中心地址，必须有前缀 `http://` 或 `https://`
datasource.provider.apollo.token	String	token	无	apollo 登录 token，默认为 `token`
datasource.provider.zookeeper.server-addr	String	localhost:2181	无	zookeeper 注册中心地址
datasource.provider.zookeeper.session-timeout	Integer	60000	0	zookeeper session超时时间，默认 `60000`
datasource.provider.zookeeper.connection-timeout	Integer	15000	0	zookeeper connection超时时间，默认 `15000`
datasource.provider.zookeeper.retry.max-retries	Integer	3	0	zookeeper 最大重试次数，默认 `3`
datasource.provider.zookeeper.retry.base-sleep-time	Integer	1000	1000	zookeeper 重试间隔最小时长，默认 `1000`
datasource.provider.zookeeper.retry.max-sleep-time	Integer	2147483647	0	zookeeper 重试间隔最大时长，默认 `2147483647`

对于单条Docker命令启动

$ docker run --rm -e PARAM="-Dserver.port=8858 -Dcsp.sentinel.dashboard.server=localhost:8858 -Dproject.name=sentinel-dashboard -Ddatasource.provider=nacos -Ddatasource.provider.nacos.server-addr=192.168.133.128:8848 -Ddatasource.provider.nacos.username=nacos -Ddatasource.provider.nacos.password=nacos -Ddatasource.provider.nacos.group-id=SENTINEL_GROUP" fjy8018/sentinel:1.8.0

对于docker-compose启动，此处结合nacos与mysql基础设施给出示例，mysql需要提前写入nacos相关初始化表结构

version: '3.7'
services:

 mysql57:
 image: mysql:5.7.29
 restart: always
 command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
 ports:
 - 3306:3306
 networks:
 - dev
 volumes:
 - ./mysql57/data:/var/lib/mysql
 - ./mysql57/custom-master.cnf:/etc/mysql/conf.d/custom.cnf
 - ./mysql57/log:/var/log/mysql
 environment:
 MYSQL_ROOT_PASSWORD: root

 nacos-standalone:
 image: nacos/nacos-server:1.1.4
 networks:
 - dev
 environment:
 - PREFER_HOST_MODE=hostname
 - MODE=standalone
 - SPRING_DATASOURCE_PLATFORM=mysql
 - MYSQL_MASTER_SERVICE_HOST=mysql57
 - MYSQL_MASTER_SERVICE_DB_NAME=nacos
 - MYSQL_MASTER_SERVICE_PORT=3306
 - MYSQL_SLAVE_SERVICE_HOST=mysql57
 - MYSQL_SLAVE_SERVICE_PORT=3306
 - MYSQL_MASTER_SERVICE_USER=root
 - MYSQL_MASTER_SERVICE_PASSWORD=root
 ports:
 - 8848:8848
 depends_on:
 - mysql57
 volumes:
 - ./standalone-logs/:/home/nacos/logs

 sentinel:
 image: fjy8018/sentinel:1.8.0
 environment:
 - PARAM=-Dserver.port=8858 -Dcsp.sentinel.dashboard.server=localhost:8858 -Dproject.name=sentinel-dashboard -Ddatasource.provider=nacos -Ddatasource.provider.nacos.server-addr=nacos-standalone:8848 -Ddatasource.provider.nacos.username=nacos -Ddatasource.provider.nacos.password=nacos -Ddatasource.provider.nacos.group-id=SENTINEL_GROUP
 networks:
 - dev
 ports:
 - 8858:8858
 restart: on-failure

networks:
 dev:

启动成功后可以看到nacos上成功注册了sentinel-dashboard

GitHub DevOps工作流构建Java项目并上传到DockerHub

Sun, 07 Feb 2021 11:58:43 +0800

工程准备

此处使用fork自alibaba/Sentinel的FJiayang/Sentinel项目实践

工程地址

Sentinel — Sentinel仓库源码

准备Dockerfile

该工程为标准Java项目，故准备好的Dockerfile文件如下

FROM anapsix/alpine-java:8_server-jre_unlimited

MAINTAINER fjy8018@gmail.com

ENV PARAM ""
ENV VERSION 1.8.0

RUN mkdir -p /sentinel

WORKDIR /sentinel

ADD ./sentinel-dashboard/target/sentinel-dashboard.jar ./app-${VERSION}.jar

# 设置时区，默认为UTC
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
RUN echo 'Asia/Shanghai' > /etc/timezone

ENTRYPOINT ["java", "${PARAM}", "-jar", "app-${VERSION}.jar"]

编写GitHub工作流

GitHub DevOps工作流又叫GitHub Actions 是2019年末推出的自动化CICD持续集成工作流，有详细的样例文档

此处参考以下文档编写

使用 Maven 构建和测试 Java — GitHub Actions 指南

安全配置

由于需要建构建后的制品推送到DockerHub中，但又不能在工作流中直接使用用户名和密码登陆，故还需要配置环境变量

在DockerHub中新增秘钥

在构建工程的设置中新增用户名和密码变量

编写后的工作流如下

name: Publish Docker image

on: [push]

jobs:
 build:

 runs-on: ubuntu-latest

 steps:
 - uses: actions/checkout@v2
 - name: Set up Maven Central Repository
 uses: actions/setup-java@v1
 with:
 java-version: 1.8

 - name: Publish package
 run: mvn --batch-mode clean package -Dmaven.test.skip=true

 - name: Push to Docker Hub
 uses: docker/build-push-action@v1
 with:
 username: ${{ secrets.DOCKER_USERNAME }}
 password: ${{ secrets.DOCKER_PASSWORD }}
 repository: fjy8018/sentinel
 tag_with_ref: true

触发构建

当前的触发方式配置为PUSH，修改好文件后会自动触发流水线工作

Maven打包，速度很快

打包完成

检查结果

登录到DockerHub中检查推送结果，可见成功推送

总结

整体来看，GitHub工作流效率高，文档齐全，编码上手速度快，相比GitLab CICD和DockerHub CICD来说，GitHub CICD配置更加简单、友好和高效。

DevOps实践心得二

Sun, 27 Dec 2020 12:25:47 +0800

背景

上一篇DevOps实践心得编写于2019年08月16日

文章最后提到未来探索方向是

全面容器化服务治理方向

国内对容器化有非常多的实践

kubernetes作为容器调度成为大多数服务治理的选择

容器隔离、迁移和伸缩带来的巨大优势

容器与传统应用部署结合方式

以半全自动方式落地DevOps，采用Mesos+marathon方式同时兼容传统服务部署和容器化部署

兼容分布式服务治理

国外有大量的实践案例

日志收集体系完备

经过一年半的时间，完成对完整DevOps的探索

Mesos+Marathon的DevOps分布式环境探索

参考文章

该架构更加适合还未进行容器化改造的项目，其支持较为通用，但各组件都是由开源组件协调而成，故搭建较为繁琐，而且后续版本升级并不方便

全面容器化服务探索

探索历程

最终完成了DevOps容器化自动部署实践

采用kuberneties方案，其统一性更加一致，依赖的基础设施也多支持容器化部署，机器利用率较高，版本升级、维护较为方便，适用于云原生项目部署

遇到的坑

最终生产环境采用kuberneties集群方案，5个节点，但由于实施经验不足，依然遇到不少坑

后续发展展望

云原生技术还在不断发展中，不少公司也研发并开源关于kuberneties集群管理方案，例如kubesphere

kubesphere — The Kubernetes platform tailored for hybrid multicloud

经过测试，其2.x版本资源占用较高，还处于刚起步阶段，暂时不适用于小规模集群

k8s证书失效问题排查

Sun, 27 Dec 2020 12:00:58 +0800

问题背景

生产环境kuberneties集群搭建于2019年11月7日，使用kubeadm结合国内镜像搭建，参考文章

2020年11月8日突然发现生产环境集群服务全都失效并且不可用，进行紧急排查处理

处理过程

1. 排查服务是否启动

[root@k8smaster1 ~]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
65dff767f422 b95b1efa0436 "kube-controller-m..." 20 hours ago Up 20 hours k8s_kube-controller-manager_kube-controller-manager-k8smaster1_kube-system_7a524400c16990c1a69c237c9da4c7f2_11
1b26acecbcea registry.cn-hangzhou.aliyuncs.com/imooc/pause:3.1 "/pause" 20 hours ago Up 20 hours k8s_POD_kube-controller-manager-k8smaster1_kube-system_7a524400c16990c1a69c237c9da4c7f2_1
9c6449df0a18 00638a24688b "kube-scheduler --..." 20 hours ago Up 20 hours k8s_kube-scheduler_kube-scheduler-k8smaster1_kube-system_baebad22afce708e52c6d3f886ff9424_10
028a1177cc9b registry.cn-hangzhou.aliyuncs.com/imooc/pause:3.1 "/pause" 20 hours ago Up 20 hours k8s_POD_kube-scheduler-k8smaster1_kube-system_baebad22afce708e52c6d3f886ff9424_1
6c6bdc2c2644 registry.cn-hangzhou.aliyuncs.com/imooc/pause:3.1 "/pause" 20 hours ago Up 20 hours k8s_POD_kube-apiserver-k8smaster1_kube-system_0e954d211e160f48f5ca9fd42d295c10_1
fe4a12daa7d7 registry.cn-hangzhou.aliyuncs.com/imooc/pause:3.1 "/pause" 20 hours ago Up 20 hours k8s_POD_etcd-k8smaster1_kube-system_6119961323d801d05a7dd23e429cda3f_1

容器服务正常，但kubelet服务有报错信息，报连接不上apiserver

[root@k8smaster1 ~]$ systemctl status kubelet -l
● kubelet.service - kubelet: The Kubernetes Node Agent
 Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
 Drop-In: /usr/lib/systemd/system/kubelet.service.d
 └─10-kubeadm.conf
 Active: active (running) since 日 2020-11-08 22:16:07 CST; 20h ago
 Docs: https://kubernetes.io/docs/
 Main PID: 15313 (kubelet)
 Tasks: 16
 Memory: 68.4M
 CGroup: /system.slice/kubelet.service
 └─15313 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --cgroup-driver=systemd --network-plugin=cni --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/imooc/pause:3.1

11月 09 18:43:06 k8smaster1 kubelet[15313]: E1109 18:43:06.967416 15313 reflector.go:126] k8s.io/kubernetes/pkg/kubelet/kubelet.go:442: Failed to list *v1.Service: Get https://192.168.1.14:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 192.168.1.14:6443: connect: connection refused
11月 09 18:43:07 k8smaster1 kubelet[15313]: E1109 18:43:07.009411 15313 kubelet.go:2244] node "k8smaster1" not found
11月 09 18:43:07 k8smaster1 kubelet[15313]: E1109 18:43:07.109665 15313 kubelet.go:2244] node "k8smaster1" not found
11月 09 18:43:07 k8smaster1 kubelet[15313]: I1109 18:43:07.165755 15313 kubelet_node_status.go:283] Setting node annotation to enable volume controller attach/detach
11月 09 18:43:07 k8smaster1 kubelet[15313]: E1109 18:43:07.167106 15313 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1beta1.CSIDriver: Get https://192.168.1.14:6443/apis/storage.k8s.io/v1beta1/csidrivers?limit=500&resourceVersion=0: dial tcp 192.168.1.14:6443: connect: connection refused
11月 09 18:43:07 k8smaster1 kubelet[15313]: I1109 18:43:07.169000 15313 kubelet_node_status.go:72] Attempting to register node k8smaster1
11月 09 18:43:07 k8smaster1 kubelet[15313]: E1109 18:43:07.209831 15313 kubelet.go:2244] node "k8smaster1" not found
11月 09 18:43:07 k8smaster1 kubelet[15313]: E1109 18:43:07.310013 15313 kubelet.go:2244] node "k8smaster1" not found
11月 09 18:43:07 k8smaster1 kubelet[15313]: E1109 18:43:07.362049 15313 kubelet_node_status.go:94] Unable to register node "k8smaster1" with API server: Post https://192.168.1.14:6443/api/v1/nodes: dial tcp 192.168.1.14:6443: connect: connection refused
11月 09 18:43:07 k8smaster1 kubelet[15313]: E1109 18:43:07.410174 15313 kubelet.go:2244] node "k8smaster1" not found

2. 检查APIServer

apiserver 端口连接正常

[root@k8smaster1 ~]$ telnet 192.168.1.14 6443
Trying 192.168.1.14...
telnet: connect to address 192.168.1.14: Connection refused

重设防火墙，并重新检查，连接正常

[root@k8smaster1 ~]$ iptables -F && iptables -X && iptables -F -t nat && iptables -X -t nat && iptables -P FORWARD ACCEPT
[root@k8smaster1 ~]$ telnet 192.168.1.14 6443
Trying 192.168.1.14...
Connected to 192.168.1.14.

但服务仍然无法访问

3. 重启kubelet

[root@k8smaster1 ~]$ systemctl restart kubelet
[root@k8smaster1 ~]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4383926ef6c9 ecf910f40d6e "kube-apiserver --..." 3 seconds ago Up 2 seconds k8s_kube-apiserver_kube-apiserver-k8smaster1_kube-system_0e954d211e160f48f5ca9fd42d295c10_1362
88eb6134fe3c 2c4adeb21b4f "etcd --advertise-..." 3 seconds ago Up 2 seconds k8s_etcd_etcd-k8smaster1_kube-system_6119961323d801d05a7dd23e429cda3f_1768
65dff767f422 b95b1efa0436 "kube-controller-m..." 20 hours ago Up 20 hours k8s_kube-controller-manager_kube-controller-manager-k8smaster1_kube-system_7a524400c16990c1a69c237c9da4c7f2_11
1b26acecbcea registry.cn-hangzhou.aliyuncs.com/imooc/pause:3.1 "/pause" 20 hours ago Up 20 hours k8s_POD_kube-controller-manager-k8smaster1_kube-system_7a524400c16990c1a69c237c9da4c7f2_1
9c6449df0a18 00638a24688b "kube-scheduler --..." 20 hours ago Up 20 hours k8s_kube-scheduler_kube-scheduler-k8smaster1_kube-system_baebad22afce708e52c6d3f886ff9424_10
028a1177cc9b registry.cn-hangzhou.aliyuncs.com/imooc/pause:3.1 "/pause" 20 hours ago Up 20 hours k8s_POD_kube-scheduler-k8smaster1_kube-system_baebad22afce708e52c6d3f886ff9424_1
6c6bdc2c2644 registry.cn-hangzhou.aliyuncs.com/imooc/pause:3.1 "/pause" 20 hours ago Up 20 hours k8s_POD_kube-apiserver-k8smaster1_kube-system_0e954d211e160f48f5ca9fd42d295c10_1
fe4a12daa7d7 registry.cn-hangzhou.aliyuncs.com/imooc/pause:3.1 "/pause" 20 hours ago Up 20 hours k8s_POD_etcd-k8smaster1_kube-system_6119961323d801d05a7dd23e429cda3f_1

检查kubelet状态，仍然报错

[root@k8smaster1 ~]$ systemctl status kubelet -l
● kubelet.service - kubelet: The Kubernetes Node Agent
 Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
 Drop-In: /usr/lib/systemd/system/kubelet.service.d
 └─10-kubeadm.conf
 Active: active (running) since 一 2020-11-09 18:52:24 CST; 24s ago
 Docs: https://kubernetes.io/docs/
 Main PID: 31759 (kubelet)
 Tasks: 15
 Memory: 30.1M
 CGroup: /system.slice/kubelet.service
 └─31759 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --cgroup-driver=systemd --network-plugin=cni --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/imooc/pause:3.1

11月 09 18:52:48 k8smaster1 kubelet[31759]: E1109 18:52:48.063832 31759 controller.go:115] failed to ensure node lease exists, will retry in 3.2s, error: Get https://192.168.1.14:6443/apis/coordination.k8s.io/v1beta1/namespaces/kube-node-lease/leases/k8smaster1?timeout=10s: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
11月 09 18:52:48 k8smaster1 kubelet[31759]: E1109 18:52:48.152723 31759 kubelet.go:2244] node "k8smaster1" not found
11月 09 18:52:48 k8smaster1 kubelet[31759]: E1109 18:52:48.252964 31759 kubelet.go:2244] node "k8smaster1" not found
11月 09 18:52:48 k8smaster1 kubelet[31759]: E1109 18:52:48.353226 31759 kubelet.go:2244] node "k8smaster1" not found
11月 09 18:52:48 k8smaster1 kubelet[31759]: E1109 18:52:48.453452 31759 kubelet.go:2244] node "k8smaster1" not found
11月 09 18:52:48 k8smaster1 kubelet[31759]: E1109 18:52:48.553682 31759 kubelet.go:2244] node "k8smaster1" not found
11月 09 18:52:48 k8smaster1 kubelet[31759]: E1109 18:52:48.653965 31759 kubelet.go:2244] node "k8smaster1" not found
11月 09 18:52:48 k8smaster1 kubelet[31759]: E1109 18:52:48.754181 31759 kubelet.go:2244] node "k8smaster1" not found
11月 09 18:52:48 k8smaster1 kubelet[31759]: E1109 18:52:48.854401 31759 kubelet.go:2244] node "k8smaster1" not found
11月 09 18:52:48 k8smaster1 kubelet[31759]: E1109 18:52:48.954576 31759 kubelet.go:2244] node "k8smaster1" not found

4.检查API Server日志

查容器ID

[root@k8smaster1 ~]$ docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
27a3b7787748 ecf910f40d6e "kube-apiserver --..." 18 seconds ago Up 18 seconds

查看日志

[root@k8smaster1 ~]$ docker logs 27a
Flag --insecure-port has been deprecated, This flag will be removed in a future version.
I1109 10:55:42.281078 1 server.go:559] external host was not specified, using 192.168.1.15
I1109 10:55:42.281264 1 server.go:146] Version: v1.14.0
I1109 10:55:42.759000 1 plugins.go:158] Loaded 9 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,MutatingAdmissionWebhook.
I1109 10:55:42.759031 1 plugins.go:161] Loaded 6 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,ValidatingAdmissionWebhook,ResourceQuota.
E1109 10:55:42.759943 1 prometheus.go:138] failed to register depth metric admission_quota_controller: duplicate metrics collector registration attempted
E1109 10:55:42.759976 1 prometheus.go:150] failed to register adds metric admission_quota_controller: duplicate metrics collector registration attempted
E1109 10:55:42.760007 1 prometheus.go:162] failed to register latency metric admission_quota_controller: duplicate metrics collector registration attempted
E1109 10:55:42.760038 1 prometheus.go:174] failed to register work_duration metric admission_quota_controller: duplicate metrics collector registration attempted
E1109 10:55:42.760065 1 prometheus.go:189] failed to register unfinished_work_seconds metric admission_quota_controller: duplicate metrics collector registration attempted
E1109 10:55:42.760086 1 prometheus.go:202] failed to register longest_running_processor_microseconds metric admission_quota_controller: duplicate metrics collector registration attempted
I1109 10:55:42.760102 1 plugins.go:158] Loaded 9 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,MutatingAdmissionWebhook.
I1109 10:55:42.760110 1 plugins.go:161] Loaded 6 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,ValidatingAdmissionWebhook,ResourceQuota.
I1109 10:55:42.763844 1 client.go:352] parsed scheme: ""
I1109 10:55:42.763866 1 client.go:352] scheme "" not registered, fallback to default scheme
I1109 10:55:42.764295 1 asm_amd64.s:1337] ccResolverWrapper: sending new addresses to cc: [{127.0.0.1:2379 0 <nil>}]
I1109 10:55:42.764449 1 asm_amd64.s:1337] balancerWrapper: got update addr from Notify: [{127.0.0.1:2379 <nil>}]
W1109 10:55:42.771517 1 clientconn.go:1251] grpc: addrConn.createTransport failed to connect to {127.0.0.1:2379 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate has expired or is not yet valid". Reconnecting...
I1109 10:55:43.756910 1 client.go:352] parsed scheme: ""
I1109 10:55:43.756939 1 client.go:352] scheme "" not registered, fallback to default scheme
I1109 10:55:43.756999 1 asm_amd64.s:1337] ccResolverWrapper: sending new addresses to cc: [{127.0.0.1:2379 0 <nil>}]
I1109 10:55:43.757077 1 asm_amd64.s:1337] balancerWrapper: got update addr from Notify: [{127.0.0.1:2379 <nil>}]
W1109 10:55:43.763355 1 clientconn.go:1251] grpc: addrConn.createTransport failed to connect to {127.0.0.1:2379 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate has expired or is not yet valid". Reconnecting...
W1109 10:55:43.770837 1 clientconn.go:1251] grpc: addrConn.createTransport failed to connect to {127.0.0.1:2379 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate has expired or is not yet valid". Reconnecting...
W1109 10:55:44.764994 1 clientconn.go:1251] grpc: addrConn.createTransport failed to connect to {127.0.0.1:2379 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate has expired or is not yet valid". Reconnecting...
W1109 10:55:45.541971 1 clientconn.go:1251] grpc: addrConn.createTransport failed to connect to {127.0.0.1:2379 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate has expired or is not yet valid". Reconnecting...
W1109 10:55:46.657328 1 clientconn.go:1251] grpc: addrConn.createTransport failed to connect to {127.0.0.1:2379 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate has expired or is not yet valid". Reconnecting...
W1109 10:55:47.938720 1 clientconn.go:1251] grpc: addrConn.createTransport failed to connect to {127.0.0.1:2379 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate has expired or is not yet valid". Reconnecting...
W1109 10:55:48.730961 1 clientconn.go:1251] grpc: addrConn.createTransport failed to connect to {127.0.0.1:2379 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate has expired or is not yet valid". Reconnecting...
W1109 10:55:51.899665 1 clientconn.go:1251] grpc: addrConn.createTransport failed to connect to {127.0.0.1:2379 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate has expired or is not yet valid". Reconnecting...
W1109 10:55:52.961615 1 clientconn.go:1251] grpc: addrConn.createTransport failed to connect to {127.0.0.1:2379 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate has expired or is not yet valid". Reconnecting...
W1109 10:55:59.193916 1 clientconn.go:1251] grpc: addrConn.createTransport failed to connect to {127.0.0.1:2379 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate has expired or is not yet valid". Reconnecting...
W1109 10:56:00.025333 1 clientconn.go:1251] grpc: addrConn.createTransport failed to connect to {127.0.0.1:2379 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate has expired or is not yet valid". Reconnecting...
F1109 10:56:02.764150 1 storage_decorator.go:57] Unable to create storage backend: config (&{ /registry {[https://127.0.0.1:2379] /etc/kubernetes/pki/apiserver-etcd-client.key /etc/kubernetes/pki/apiserver-etcd-client.crt /etc/kubernetes/pki/etcd/ca.crt} false true 0xc0007a94d0 apiextensions.k8s.io/v1beta1 <nil> 5m0s 1m0s}), err (context deadline exceeded)

日志最后提示：authentication handshake failed: x509: certificate has expired or is not yet valid

以此信息查询解决方案

解决方案

外部推荐方案

最终在该网站找到解决方案

Kubeadm证书过期时间调整 — Devops、小铁匠

采用方案

由于生产环境集群版本为1.14，是支持自动轮换证书的，故首先解决生产环境问题，先手动重新签发证书，再开启证书自动续签

重新签发证书

国内网络直接签发证书会因为无法连接Google而报错

[root@k8smaster1 ~]$ kubeadm alpha certs renew all
I1109 19:33:40.266778 22487 version.go:96] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get https://storage.googleapis.com/kubernetes-release/release/stable-1.txt: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
I1109 19:33:40.266978 22487 version.go:97] falling back to the local client version: v1.14.0
I1109 19:33:51.139051 22487 version.go:96] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get https://storage.googleapis.com/kubernetes-release/release/stable-1.txt: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
I1109 19:33:51.139111 22487 version.go:97] falling back to the local client version: v1.14.0
I1109 19:34:01.511909 22487 version.go:96] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get https://storage.googleapis.com/kubernetes-release/release/stable-1.txt: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
I1109 19:34:01.511967 22487 version.go:97] falling back to the local client version: v1.14.0
I1109 19:34:11.945411 22487 version.go:96] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get https://storage.googleapis.com/kubernetes-release/release/stable-1.txt: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
I1109 19:34:11.945504 22487 version.go:97] falling back to the local client version: v1.14.0
I1109 19:34:22.145905 22487 version.go:96] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get https://storage.googleapis.com/kubernetes-release/release/stable-1.txt: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
I1109 19:34:22.145975 22487 version.go:97] falling back to the local client version: v1.14.0
I1109 19:34:32.532514 22487 version.go:96] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get https://storage.googleapis.com/kubernetes-release/release/stable-1.txt: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
I1109 19:34:32.532561 22487 version.go:97] falling back to the local client version: v1.14.0
I1109 19:34:43.335574 22487 version.go:96] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get https://storage.googleapis.com/kubernetes-release/release/stable-1.txt: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
I1109 19:34:43.335628 22487 version.go:97] falling back to the local client version: v1.14.0

先准备一份连接国内镜像的kubeadm.conf文件

apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
kubernetesVersion: v1.14.0 #-->这里改成你集群对应的版本
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers 
#这里使用国内的镜像仓库，否则在重新签发的时候会报错：could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt"

重新签发，即可，并重启kubelet

[root@k8smaster1 ~]$ kubeadm alpha certs renew all --config=/root/kubeadm.conf
[root@k8smaster1 ~]$ systemctl restart kubelet

重新签发后若要使用kubectl命令，要重新生成~/.kube/config配置文件

[root@k8smaster1 kubernetes]$ kubeadm init phase kubeconfig all --config=/root/kubeadm.conf
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[root@k8smaster1 kubernetes]$ cp -i /etc/kubernetes/admin.conf ~/.kube/config
cp：是否覆盖"/root/.kube/config"？ y

命令正常使用

[root@k8smaster1 kubernetes]$ kubectl get pods
NAME READY STATUS RESTARTS AGE
glusterfs-hfdmh 1/1 Running 0 351d
glusterfs-k766z 1/1 Running 0 351d
glusterfs-rrc7x 1/1 Running 0 351d
heketi-68f9dfdfbf-2k58b 1/1 Running 0 351d
nginx 1/1 Running 1 370d
nginx-ds-29dbc 1/1 Running 5 366d
nginx-ds-4w6cn 1/1 Running 1 370d
nginx-ds-6lhsk 0/1 Evicted 0 216d
nginx-ds-xq4h7 1/1 Running 3 366d
tomcat-demo-6bc7d5b6f4-75rgc 0/1 Evicted 0 351d

重新启动集群上的所有服务，可见服务正常

证书自动续签

kubelet证书分为server和client两种， k8s 1.9默认启用了client证书的自动轮换，但server证书自动轮换需要用户开启

查询kubelet配置文件位置

[root@k8smaster1 ~]$ find / -name 10-kubeadm.conf
/usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
[root@k8smaster1 ~]$ vi /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf

添加配置如下

# 在/etc/systemd/system/kubelet.service.d/10-kubeadm.conf 增加如下参数
Environment="KUBELET_EXTRA_ARGS=--feature-gates=RotateKubeletServerCertificate=true"

添加Controller-Manager参数

apiVersion: v1
kind: Pod
metadata:
 creationTimestamp: null
 labels:
 component: kube-controller-manager
 tier: control-plane
 name: kube-controller-manager
 namespace: kube-system
spec:
 containers:
 - command:
 - kube-controller-manager
 - --allocate-node-cidrs=true
 # 证书有效期
 - --experimental-cluster-signing-duration=87600h0m0s
 # 证书自动签发
 - --feature-gates=RotateKubeletServerCertificate=true
 - --authentication-kubeconfig=/etc/kubernetes/controller-manager.conf
 - --authorization-kubeconfig=/etc/kubernetes/controller-manager.conf
 - --bind-address=127.0.0.1
 - --client-ca-file=/etc/kubernetes/pki/ca.crt
 - --cluster-cidr=172.22.0.0/16
 - --cluster-signing-cert-file=/etc/kubernetes/pki/ca.crt
 - --cluster-signing-key-file=/etc/kubernetes/pki/ca.key
 - --controllers=*,bootstrapsigner,tokencleaner
 - --kubeconfig=/etc/kubernetes/controller-manager.conf
 - --leader-elect=true
 - --node-cidr-mask-size=24
 - --requestheader-client-ca-file=/etc/k

创建RBAC对象

cat > ca-update.yaml << EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
 annotations:
 rbac.authorization.kubernetes.io/autoupdate: "true"
 labels:
 kubernetes.io/bootstrapping: rbac-defaults
 name: system:certificates.k8s.io:certificatesigningrequests:selfnodeserver
rules:
- apiGroups:
 - certificates.k8s.io
 resources:
 - certificatesigningrequests/selfnodeserver
 verbs:
 - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
 name: kubeadm:node-autoapprove-certificate-server
roleRef:
 apiGroup: rbac.authorization.k8s.io
 kind: ClusterRole
 name: system:certificates.k8s.io:certificatesigningrequests:selfnodeserver
subjects:
- apiGroup: rbac.authorization.k8s.io
 kind: Group
 name: system:nodes
EOF

[root@k8smaster1 ~]$ kubectl create -f ca-update.yaml
clusterrole.rbac.authorization.k8s.io/system:certificates.k8s.io:certificatesigningrequests:selfnodeserver created
clusterrolebinding.rbac.authorization.k8s.io/kubeadm:node-autoapprove-certificate-server created

金融企业会计笔记

Thu, 13 Aug 2020 11:20:00 +0800

会计科目分类

按资产性质划分

资产类科目

资产类科目是金融企业各项经济资源的分类名称，用来核算金融企业的各种资金运用结果。

此科目包括流动资产、固定资产、无形资产和其他资产等。

负债类科目

负债类科目是金融企业各项债务的分类名称用来核算金融企业的各种债务性资金来源状况。

此科目包括短期负债、长期负债和其他负债等。

所有者权益类科目

所有者权益类科目是金融企业资本项目的分类名称，用来核算所有者投入资金以及经营所得或亏损情况。

前者包括实收资本和资本公积，后者包括盈余公积和未分配利润。

损益类科目

损益类科目是金融企业各项经营收人和费用支出的分类名称用来核算金融企业的营业收支等情况。

此科目包括利息收入、利息支出、投资收益、营业费用等。(4)损益类科目。

资产负债共同类科目

资产负债共同类科目既能反映商业银行的资产业务，又能反映商业银行的负债业务。

此科目的余额在借方表现为资产，余额在贷方表现为负债。

在编制会计报表时应根据余额方向，分别纳入资产类或负债类科目反映。

按与资产负债表的关系分类

表内科目

表内科目是指反映金融企业资金实际增减变化而纳入资产负债表内的科目

表外科目

表外科目是指反映金融企业确已发生而尚未涉及资金的实际增减变化,或不涉及资金增减变化而不列人资产负债表的科目,如“有价单证”、“空白重要凭证”等

按反映经济业务内容详略程度分类

一级科目
二级科目

记账方法

复式记账法（借贷记账法）

借贷记账法的记账原理。借贷记账法以“资产=负债+所有者权益”的会计平衡公式为记账原理。这一平衡公式体现了资产总额与负债和所有者权益总额之间数量上的平衡关系。
借贷记账法的记账符号。借贷记账法的记账符号是“借”和“贷”。“借”在账户的左方，“贷“在账户的右方。
借贷记账法的记账规则。借贷记账法以“有借必有贷，借贷必相等”为记账规则。
- 资产和费用的增加记入借方
- 资产和费用的减少记入贷方
- 余额反映在借方
- 负债、权益和收入的增加记入贷方
- 负债、权益和收入的减少记入借方
- 余额反映在贷方。遵守记账规则
关键是在把经济业务记入账户之前，正确确定账户对应关系，编制会计分录。
借贷记账法的试算平衡
- 发生额平衡公式
  
  各科目借方本期发生额 = 各科目贷方本期发生额
- 余额平衡公式
  
  各科目借方余额合计=各科目贷方余额合计

单式记账法

单式记账法就是对发生的每一项经济业务只在一个账户中进行登记的记账方法。

单式记账设置收入、付出、余额三栏,业务发生或增加时记“收入”,销账或减少时记“付出”,“余额”表示结存。

账簿

序时账簿

序时账簿通常称为日记账,它是按照全部经济业务完成时间的先后顺序进行登记的账簿。

在实际工作中,按照会计部门收到凭证的先后顺序即按照凭证的编号顺序进行登记

分类账簿

分类账簿是全面、系统地对资金循环和收支活动所作的分类记录。

按照分类的概括程度不同,分类账簿又可分为

明细分类账
总分类账

明细分类账

明细分类账是按账户设置和登记的账簿。

金融企业的明细分类账按每一会计科目下的开户单位或具体核算对象分户立账,并根据记账凭证的业务顺序逐笔连续记载

总分类账

总分类账是按会计科目设置的，是汇总明细分类账的主要工具。

总分类账于每日营业终了，根据各科目日结单借、贷方发生额和今日余额进行登记。

备查账簿

备查账簿是对某些在日记账和分类账等主要账簿中未能记载的事项进行补充登记的账簿。

这种账簿属于辅助性登记，没有固定的格式，可根据需要加以设计。

账务组织

账务组织是指总的账务结构体系,它包括明细核算和综合核算两个系统。明细核算对综合核算起到补充作用,综合核算对明细核算起到统驭作用,二者构成了一套完整的、科学的、严密的账务组织体系

明细核算

明细核算是按账户核算，反映各项资金增减变化及其结果的详细情况的核算系统。

它的具体形式和所使用的账表包括分户账、登记簿、余额表等。

它的核算程序为：根据会计凭证登记分户账或登记簿，再根据分户账或登记簿编制余额表，最后同总账进行核对。

分户账

分户账是企业会计账簿中具体反映经济业务的明细分类账簿，是明细核算的主要形式。分户账的格式分为甲、乙、丙、丁四种。

登记簿

登记簿又称登记卡，是为了适应某些业务需要而设置的专用账簿。登记簿的格式根据业务需要可自行设计，一般都采用收入、付出、余额三栏的通用格式，根据表外科目传票采取单式记账方法进行登记处理。

余额表

余额表是核对总账与分户账余额和计算利息的重要工具。余额表分为计息余额表和一般余额表。

综合核算

综合核算是按会计科目进行核算,综合、概括地反映各科目的资金增减变化情况,由各种总分类账组成的核算系统。

综合核算所采用的主要形式包括科目日结单、总账、日计表等。

其记账程序为:通过业务传票的审核编制科目日结单,然后根据科目日结单登记各科目总账,最后编制日计表

科目日结单

科目日结单又称总账记账凭证，是每天会计科目的当日借贷方发生额和凭证、附件张数的汇总记录，是监督明细账发生额、轧平当日账务的重要工具，也是记载总账的依据。

每日营业终了，应按同一科目的凭证区分现金和转账凭证、借方或贷方发生额，各自加计总数填入科目日结单的有关栏目，并注明凭证张数。

总账

总账是综合核算与明细核算相互核对，统驭明细分户账的主要工具，是各科目的总括记录，也是编制日计表、月计表、业务状况报告表、资产负债表的依据。总账按科目设置，设有借、贷方发生额和借、贷方余额四栏，账页每月更换一次。

总账的发生额，应于每日营业终了，根据各科目日结单的借贷方发生额合计数分别填记；余额根据本科目的上日余额加减本日发生额后求出。

检验科目总账余额是否正确应视不同情况而定

如果上日借方余额大于贷方余额，可使用以下公式进行检验：

上日借方余额-上日贷方余额+本日借方发生额-本日贷方发生额=本日借方余额-本日贷方余额
如果上日贷方余额大于借方余额，可使用以下公式进行检验：

上日贷方余额-上日借方余额+本日贷方发生额-本日借方发生额=本日贷方余额-本日借方余额

日计表

日计表是反映当日业务活动和轧平全部账务的主要工具。日计表应每日编制，日计表上当日发生额和余额应根据总账各科目当日发生额和余额填记。

日计表上借、贷方发生额和借、贷方余额合计数必须各自平衡

存款业务

存款业务的分类

按存款期限划分

活期存款
定期存款

按存款对象划分

单位存款
储蓄存款

按资金性质划分

财政性存款
一般存款

按存款产生的来源划分

原始存款
派生存款

按存款币种划分

人民币存款
外币存款

存款账户的种类

基本存款账户

它是存款人办理日常转账结算和现金收付的账户。

一般存款账户

它是存款人在基本存款账户以外的银行借款转存、与基本存款账户不在同一地点的、附属非独立核算单位开立的账户。

临时存款账户

它是存款人因临时经营活动需要开立的账户。

专用存款账户

它是存款人因为特定用途需要开立的账户。

单位活期存款业务的核算

现金存入

支票户现金存入核算

分录：

借:现金贷:吸收存款——活期存款——XX单位户

存折户现金存入核算

分录：

借:现金贷:吸收存款——活期存款——XX单位户

现金支取

支票户现金支取核算

分录：

借:吸收存款——活期存款——XX单位户贷:现金

存折户现金支取核算

分录：

借:吸收存款一一活期存款一一XX单位户贷:现金

活期储蓄存款业务的核算

开户

分录：

借:现金贷:吸收存款——活期储蓄存款——XX存款人户

销户

分录：

借:吸收存款——活期储蓄存款——XX存款人户利息支出——活期储蓄存款利息支出户贷:现金

续存

分录：

借:现金贷:吸收存款——活期储蓄存款——XX存款人户

支取

分录：

借:吸收存款——活期储蓄存款——XX存款人户贷:现金

微服务平台开发总结（3）——第二代密码管理平台

Thu, 07 May 2020 21:38:00 +0800

背景

吸取前两代系统的失败经验，对第二代密码管理微服务进行重新规划，并且在探索过程中见识到了cloud-platform、SpringBlade、pigx等优秀的微服务项目，考虑基于开源项目二次开发

规划

为了尽快落地上线，第一期以实现最小功能为主，其他非必须功能尽可能往后放

第一期

SpringCloud微服务集成
Nacos、Sentinel、graylog、zipkin、 Turbine集成
前后端密码管理功能对接
操作日志记录
邮件通知
Excel导入导出
v1迁移到v2工具包

第二期

dble读写分离
支持忘记密码、密码锁定等
集成微信推送、微信注册功能
支持上传前端日志
页面排序、检索优化

第三期

增加用户反馈功能
接入Seata分布式事务
支持配置导入导出权限
支持录入标签
…

技术体系

前端

前后端分离-采用前后端分离模式，前端基于Vue
采用Vuex进行状态管理，采用ES6作为语法支持
前后端基于开源项目SpringBlade二次开发
前后端代码生成-定制针对两套前端与后端的代码生成模板，轻松生成整个模块的前后端代码，减少重复工作量
多租户顶部菜单配置-提供给每个租户独立的顶部菜单配置模块，可以自定义顶部菜单切换

后端

基于SpringCloud的分布式架构
采用Mybatis与MybatisPlus作为ORM持久层框架
多租户系统-完整的SaaS多租户架构
借鉴OAuth2，通过Jwt令牌实现了多终端认证系统，可控制子系统的token权限互相隔离
权限管理-精心设计的权限管理方案，角色权限精确到按钮
集成邮件服务，通过事件传播实现不依赖消息队列通知发送服务
集成微信推送服务，实现微信注册、敏感操作消息推送
动态聚合文档-实现基于Nacos的Swagger SpringCloud聚合文档
实现动态网关，不重启就能设置网关配置项
集成Turbine聚合熔断监控

数据架构

采用MySQL进行数据存储，底层配置为一主两从的高可用集群
采用ElasticSearch三节点集群进行日志数据存储

分布式支持

Nacos-集成阿里巴巴的Nacos完成统一的服务注册与配置
Sentinel-集成Sentinel从流量控制、熔断降级、系统负载等多个维度保护服务的稳定性
采用Docker打包，部署于Kubernetes集群上
Seata分布式事务-定制集成Seata，支持分布式事务，无代码侵入，不失灵活与简洁
使用graylog进行分布式日志采集，集中监控
通过zipkin进行分布式链路追踪，耗时统计和服务调用情况

DevOps

配置管理采用单仓库模式，集成docker打包插件，自动推送到harbor仓库
采用GitLab Runner CICD流水线自动部署到测试环境和生产环境
结合Maven私服和Harbor私服为内网自动化部署提供支持

官方架构图

截图

管理员

操作日志记录

错误日志系统内采集

Turbine集成熔断监控

通知发布

普通用户

主题切换

微信扫码注册

补全信息

主页资源权限控制

菜单资源权限控制

Excel导入导出

密码管理

运维

Nacos注册中心

Sentinel限流管控

SpringBoot Admin应用内指标监控

Harbor内网私服

Nexus内网私服，提供maven和npm包

graylog日志采集

三节点ES集群支持日志存储

耗时统计

新UI风格

动态服务依赖关系图

新UI依赖图

Prometheus指标采集

Grafana资源指标监控

Kuberneties服务部署监控

开发过程工具

Gitlab CICD流水线

看板

功能开发周期分析

gitlab集成k8s部署环境

Sonarqube代码质量检查

分支图

工程模块图

结果——成功

经过两个月的开发和调试，第一期工程符合预定计划，按期投产上线运行，并经过2周试运行和测试后，进入生产环境，迁移老版本数据，正式替换了运行一年半的第一代密码管理系统

时间线

2020年3月开始基于SpringBlade 2.6.2编码
2020年4月20日第一个测试版本进入k8s测试环境
2020年4月28日第一个稳定版进行试运行阶段
2020年5月4日试运行阶段结束，第一个稳定版1.1.2发布，正式进行数据迁移并投入使用

微服务平台开发总结（1）——第一代密码管理系统

Thu, 07 May 2020 21:34:00 +0800

背景

随着网站、工作使用需要，对密码存储的需求越来越大，初期采用本地数据库进行存储，但不适合多地访问，一旦离开了本地电脑，就无法查询密码信息，故计划搭建个人使用的密码信息管理系统

概要

时间线

密码管理信息系统于2018年12月立项，2019年1月9日开始第一次提交
2019年2月7日第一个版本上线试运行
2019年4月7日进入维护阶段，不再支持新功能
2019年11月16日停止维护

架构

后端

JDK8
SpringBoot 2.1.1.RELEASE
Jetty
SpringData Jpa
MySQL 5.7
Docker
Kuberneties 1.14
Redis
Graylog
GitLab CICD

前端

Vue 2.5
element 2.4.11
vuex 3
vue-axios 2.1.4
eslint 5.8.0
nodejs
nginx
https
Docker
GitLab CICD

原理

后端存储密码数据字段和网站相关信息
采用SQL全字段模糊查询匹配结果
密码字段存储采用AES128结合SHA随机盐值加密存储，即使数据库被破解也无法查看对应的密码信息
密码加密秘钥不由服务端存储，只在用户登陆时录入并临时通过AES128加密后存储在Redis中，用户登出或者登陆超时后销毁

优势

单体项目，部署简单
容器化部署，迁移便利
Kuberneties云平台部署，支持灰度发布
无状态后端服务，支持集群化部署

安全审计

支持简单的安全审计

登陆日志记录
登陆归属地记录

安全问题

通过实现的Spring拦截器进行登陆拦截，安全性不高
没有采用session会话方式，故多地登陆账号且加密秘钥未过期则不需要再次录入
支持所有跨域行为，没有对客户端进行识别

其他问题

独立用户控制，扩展性不高
没有基于RBAC模型的权限控制
没有管理后台
单模块、单体项目，不利于扩展

界面演示

登陆页

若用户不存在则自动新增

录入加密秘钥

主页

信息录入

支持全字段模糊检索

支持Excel模板信息导入

登出

效果

系统稳定运行一年，符合系统初期设计目的
同时兼容移动端渲染

Docker MySQL主从复制方案

Fri, 17 Apr 2020 15:37:02 +0800

背景

现有网上很多教程都是修改了源MySQL镜像，因此会存在安全隐患，也不容易提升版本。此处给出不修改源镜像的主从复制配置方案。

现状

现有一台生产上运行的MySQL5.7服务器，由docker镜像提供支持

原docker-compose文件

version: '3.7'
services:

 mysql57:
 image: mysql:5.7
 restart: always
 command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
 ports:
 - 3306:3306
 networks:
 - prod
 volumes:
 - ./mysql57/data:/var/lib/mysql
 environment:
 MYSQL_ROOT_PASSWORD: root
networks:
 prod:

改造过程

配置主服务器

新增同步用户并授权

##创建slave1用户
CREATE USER 'slave1'@'%' IDENTIFIED BY 'slavepass';

##为slave1赋予REPLICATION SLAVE权限。
GRANT REPLICATION SLAVE ON *.* TO 'slave1'@'%';

开启binlog日志

由于原主服务器没有开启binlog日志，故需要挂载配置文件并开启binlog日志

新增配置文件custom-master.cnf

[mysqld]
log-bin = mysql-bin
log-bin-index = mysql-bin.index
binlog_format = mixed
server-id = 1
sync-binlog = 1
character-set-server = utf8

修改docker-compose文件为

version: '3.7'
services:

 mysql57:
 image: mysql:5.7.29
 restart: always
 command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
 ports:
 - 3306:3306
 networks:
 - prod
 volumes:
 - ./mysql57/data:/var/lib/mysql
 - ./mysql57/custom-master.cnf:/etc/mysql/conf.d/custom.cnf
 - ./mysql57/log:/var/log/mysql
 environment:
 MYSQL_ROOT_PASSWORD: root
networks:
 prod:

更新服务（会短暂停机）

$ docker-compose up -d

配置内容解释

log-bin：设置二进制日志文件的基本名；
log-bin-index：设置二进制日志索引文件名；
binlog_format：控制二进制日志格式，进而控制了复制类型，三个可选值
 -STATEMENT：语句复制
 -ROW：行复制
 -MIXED：混和复制，默认选项
server-id：服务器设置唯一ID，默认为1，推荐取IP最后部分；
sync-binlog：默认为0，为保证不会丢失数据，需设置为1，用于强制每次提交事务时，同步二进制日志到磁盘上。

备份主数据库数据

该步骤可以等到两台从服务器的docker环境都配置完毕后操作，以减少停机时间

获取一个一致性的快照，需对所有表设置读锁：

flush tables with read lock;

获取二进制日志的坐标：

show master status;

File	Position	Binlog_Do_DB	Binlog_Ignore_DB	Executed_Gtid_Set
mysql-bin.000003	6165

备份数据

# 针对事务性引擎
mysqldump -u root -p --all-databases -e --single-transaction --flush-logs --max_allowed_packet=1048576 --net_buffer_length=16384 > /var/log/mysql/all_db.sql
# 针对 MyISAM 引擎，或多引擎混合的数据库
mysqldump -uroot --all-databases -e -l --flush-logs --max_allowed_packet=1048576 --net_buffer_length=16384 > /var/log/mysql/all_db.sql

由于挂载了日志目录，故在容器外部可以看到该备份的数据文件

配置从服务器

新增两台CentOS 7服务器，并安装docker和docker-compose服务
由于主从复制要求所有机器版本号包括小版本号都必须一致，故此处拉取mysql:5.7.29镜像
从主库拷贝数据文件all_db.sql到两台从库上

$ scp /root/mysql/log/all_db.sql root@192.168.1.33:/root/mysql/log/

新增配置文件custom-slave1.cnf

[mysqld]
log-bin = mysql-bin
binlog_format = mixed
log-slave-updates = 0
server-id = 2
relay-log = mysql-relay-bin
relay-log-index = mysql-relay-bin.index
read-only = 1
slave_net_timeout = 10

新增docker-compose文件

version: '3'
services:

 mysql57-master:
 image: mysql:5.7.29
 restart: always
 command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
 ports:
 - 3306:3306
 networks:
 - mysql-net
 volumes:
 - ./data:/var/lib/mysql
 - ./custom-slave1.cnf:/etc/mysql/conf.d/custom.cnf
 - ./log:/var/log/mysql
 environment:
 MYSQL_ROOT_PASSWORD: root

networks:
 mysql-net:

由于也挂载了log目录，启动后容器内就已经有从主库同步过来的数据文件
数据恢复

mysql -u root -p < /var/log/mysql/all_db.sql

使用工具连接数据库，配置主从连接

CHANGE MASTER TO
MASTER_HOST='10.168.1.30',
MASTER_USER='slave1',
MASTER_PASSWORD='slavepass',
MASTER_LOG_FILE='mysql-bin.000003',
MASTER_LOG_POS=6165;

启动slave同步进程

start slave;

关闭主服务器全局读锁

UNLOCK TABLES;

查看从服务器同步状态，主要是Slave_IO_Running， Slave_SQL_Running状态必须为Yes

Slave_IO_State	Master_Host	Master_User	Master_Port	Connect_Retry	Master_Log_File	Read_Master_Log_Pos	Relay_Log_File	Relay_Log_Pos	Relay_Master_Log_File	Slave_IO_Running	Slave_SQL_Running	Replicate_Do_DB	Replicate_Ignore_DB	Replicate_Do_Table	Replicate_Ignore_Table	Replicate_Wild_Do_Table	Replicate_Wild_Ignore_Table	Last_Errno	Last_Error	Skip_Counter	Exec_Master_Log_Pos	Relay_Log_Space	Until_Condition	Until_Log_File	Until_Log_Pos	Master_SSL_Allowed	Master_SSL_CA_File	Master_SSL_CA_Path	Master_SSL_Cert	Master_SSL_Cipher	Master_SSL_Key	Seconds_Behind_Master	Master_SSL_Verify_Server_Cert	Last_IO_Errno	Last_IO_Error	Last_SQL_Errno	Last_SQL_Error	Replicate_Ignore_Server_Ids	Master_Server_Id	Master_UUID	Master_Info_File	SQL_Delay	SQL_Remaining_Delay	Slave_SQL_Running_State	Master_Retry_Count	Master_Bind	Last_IO_Error_Timestamp	Last_SQL_Error_Timestamp	Master_SSL_Crl	Master_SSL_Crlpath	Retrieved_Gtid_Set	Executed_Gtid_Set	Auto_Position	Replicate_Rewrite_DB	Channel_Name	Master_TLS_Version
Waiting for master to send event	192.168.1.30	slave1	3306	60	mysql-bin.000003	3722724	mysql-relay-bin.000002	3717604	mysql-bin.000003	Yes	Yes							0		0	3722724	3717811	None		0	No						0	No	0		0			1	4c99130f-e664-11e9-987b-0242ac120003	/var/lib/mysql/master.info	0		Slave has read all relay log; waiting for more updates	86400								0

可能出现的问题

死锁——waiting for global read lock

原因在于flush tables with read lock语句和事务执行的过程都涉及到连个锁，而且不是原子的，再加上并行复制以及设置了从库binlog的顺序提交，最终导致多个线程形成死锁

解决方法：

重启主数据库
查看所有连接线程，分析死锁的原因，杀掉执行flush tables with read lock的线程

参考资料：

k8s环境下GitLab+Helm+GitLab Runner Java项目CICD落地实践

Wed, 27 Nov 2019 10:24:00 +0800

背景

目前使用5台服务器搭建了Kubernetes集群环境，监控、日志采集均已落地，业务也手工迁移到集群中顺利运行，故需要将原本基于原生docker环境的CICD流程迁移到Kubernetes集群中

优势

Kubernetes集群实现CICD有几个显著优势

Deployment 天然支持滚动部署、结合其他Kubernetes特性还能实现蓝绿部署、金丝雀部署等
新版本的GitLab与GitLab Runner天然支持Kubernetes集群，支持runner自动伸缩，减小资源占用

环境

Kubernetes版本：1.14

GitLab版本：12.2.5

GitLab-Runner版本：12.1.0

Docker环境版本：17.03.1

GitLab-Runner部署

配置介绍

原始环境的gitlab runner通过手动执行官网提供的注册命令和启动命令，分成两部部署，需要较多的手工操作，而在Kubernetes中，其支持使用Helm一键部署，官方文档如下

GitLab Runner Helm Chart

其实官方文档的指引并不清晰，许多配置在文档中没有介绍用法，推荐去其源码仓库查看详细的参数使用文档

The Kubernetes executor

其中介绍了几个关键配置，在后面修改工程的ci配置文件时会用到

使用DinD方式构建已经不再推荐

官方文档介绍

Use docker-in-docker workflow with Docker executor
The second approach is to use the special docker-in-docker (dind) Docker image with all tools installed (docker) and run the job script in context of that image in privileged mode.

Note: docker-compose is not part of docker-in-docker (dind). To use docker-compose in your CI builds, follow the docker-compose installation instructions.

Danger: By enabling --docker-privileged, you are effectively disabling all of the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. For more information, check out the official Docker documentation on Runtime privilege and Linux capabilities.

Docker-in-Docker works well, and is the recommended configuration, but it is not without its own challenges:

When using docker-in-docker, each job is in a clean environment without the past history. Concurrent jobs work fine because every build gets it’s own instance of Docker engine so they won’t conflict with each other. But this also means jobs can be slower because there’s no caching of layers.

By default, Docker 17.09 and higher uses --storage-driver overlay2 which is the recommended storage driver. See Using the overlayfs driver for details.

Since the docker:19.03.1-dind container and the Runner container don’t share their root filesystem, the job’s working directory can be used as a mount point for child containers. For example, if you have files you want to share with a child container, you may create a subdirectory under /builds/$CI_PROJECT_PATH and use it as your mount point (for a more thorough explanation, check issue #41227):

总之使用DinD进行容器构建并非不可行，但面临许多问题，例如使用overlay2网络需要Docker版本高于 17.09

Using docker:dind
Running the docker:dind also known as the docker-in-docker image is also possible but sadly needs the containers to be run in privileged mode. If you’re willing to take that risk other problems will arise that might not seem as straight forward at first glance. Because the docker daemon is started as a service usually in your .gitlab-ci.yaml it will be run as a separate container in your Pod. Basically containers in Pods only share volumes assigned to them and an IP address by which they can reach each other using localhost. /var/run/docker.sock is not shared by the docker:dind container and the docker binary tries to use it by default.

To overwrite this and make the client use TCP to contact the Docker daemon, in the other container, be sure to include the environment variables of the build container:

DOCKER_HOST=tcp://localhost:2375 for no TLS connection.

DOCKER_HOST=tcp://localhost:2376 for TLS connection.

Make sure to configure those properly. As of Docker 19.03, TLS is enabled by default but it requires mapping certificates to your client. You can enable non-TLS connection for DIND or mount certificates as described in Use Docker In Docker Workflow wiht Docker executor

在Docker 19.03.1版本之后默认开启了TLS配置，在构建的环境变量中需要声明，否则报连接不上docker的错误，并且使用DinD构建需要runner开启特权模式，以访问主机的资源，并且由于使用了特权模式，在Pod中对runner需要使用的资源限制将失效

使用Kaniko构建Docker镜像

目前官方提供另一种方式在docker容器中构建并推送镜像，实现更加优雅，可以实现无缝迁移，那就是kaniko

Building a Docker image with kaniko

其优势官网描述如下

在Kubernetes集群中构建Docker映像的另一种方法是使用kaniko。iko子

允许您构建没有特权访问权限的映像。

无需Docker守护程序即可工作。

在后面的实践中会使用两种方式构建Docker镜像，可根据实际情况选择

使用Helm部署

拉取Helm Gitlab-Runner仓库到本地，修改配置

GitLab Runner

将原有的gitlab-runner配置迁移到Helm中，迁移后如下

image: alpine-v12.1.0
imagePullPolicy: IfNotPresent
gitlabUrl: https://gitlab.fjy8018.top/
runnerRegistrationToken: "***"
unregisterRunners: true
terminationGracePeriodSeconds: 3600
concurrent: 10
checkInterval: 30
rbac:
 create: true
 clusterWideAccess: false
metrics:
 enabled: true
 listenPort: 9090
runners:
 image: ubuntu:16.04
 imagePullSecrets:
 - name: registry-secret
 locked: false
 tags: "k8s"
 runUntagged: true
 privileged: true
 pollTimeout: 180
 outputLimit: 4096
 cache: {}
 builds: {}
 services: {}
 helpers: {}
resources:
 limits:
 memory: 2048Mi
 cpu: 1500m
 requests:
 memory: 128Mi
 cpu: 200m
affinity: {}
nodeSelector: {}
tolerations: []
hostAliases:
 - ip: "192.168.1.13"
 hostnames:
 - "gitlab.fjy8018.top"
 - ip: "192.168.1.30"
 hostnames:
 - "harbor.fjy8018.top"
podAnnotations: {}

其中配置了私钥、内网harbor地址、harbor拉取资源私钥，资源限制策略

GitLab-Runner选择可能导致的坑

选择runner镜像为alpine-v12.1.0，这一点单独说一下，目前最新的runner版本为12.5.0，但其有许多问题，alpine新版镜像在Kubernetes中间断发生无法解析DNS的问题，反映到GitLab-Runner中就是Could not resolve host和server misbehaving

查阅解决方法

通过查询发现，其官方仓库还有多个相关issue没有关闭

官方gitlab：Kubernetes runner: Could not resolve host

stackoverflow：Gitlab Runner is not able to resolve DNS of Gitlab Server

给出的解决方案无一例外都是降级到alpine-v12.1.0

We had same issue for couple of days. We tried change CoreDNS config, move runners to different k8s cluster and so on. Finally today i checked my personal runner and found that i’m using different version. Runners in cluster had gitlab/gitlab-runner:alpine-v12.3.0, when mine had gitlab/gitlab-runner:alpine-v12.0.1. We added line
image: gitlab/gitlab-runner:alpine-v12.1.0
in values.yaml and this solved problem for us

其问题的根源应该在于alpine基础镜像对Kubernetes 集群支持有问题，

ndots breaks DNS resolving #64924

docker-alpine仓库对应也有未关闭的issue，其中就提到了关于DNS解析超时和异常的问题

DNS Issue #255

安装

一行命令安装即可

$ helm install /root/gitlab-runner/ --name k8s-gitlab-runner --namespace gitlab-runner

输出如下

NAME: k8s-gitlab-runner
LAST DEPLOYED: Tue Nov 26 21:51:57 2019
NAMESPACE: gitlab-runner
STATUS: DEPLOYED

RESOURCES:
==> v1/ConfigMap
NAME DATA AGE
k8s-gitlab-runner-gitlab-runner 5 0s

==> v1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
k8s-gitlab-runner-gitlab-runner 0/1 1 0 0s

==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
k8s-gitlab-runner-gitlab-runner-744d598997-xwh92 0/1 Pending 0 0s

==> v1/Role
NAME AGE
k8s-gitlab-runner-gitlab-runner 0s

==> v1/RoleBinding
NAME AGE
k8s-gitlab-runner-gitlab-runner 0s

==> v1/Secret
NAME TYPE DATA AGE
k8s-gitlab-runner-gitlab-runner Opaque 2 0s

==> v1/ServiceAccount
NAME SECRETS AGE
k8s-gitlab-runner-gitlab-runner 1 0s


NOTES:

Your GitLab Runner should now be registered against the GitLab instance reachable at: "https://gitlab.fjy8018.top/"

查看gitlab admin页面，发现已经有一个runner成功注册

工程配置

DinD方式构建所需配置

如果原本的ci文件是基于19.03 DinD镜像构建的则需要加上TLS相关配置

image: docker:19.03

variables:
 DOCKER_DRIVER: overlay
 DOCKER_HOST: tcp://localhost:2375
 DOCKER_TLS_CERTDIR: ""
...

其余配置保持不变，使用DinD构建

Kubectl和Kubernetes权限配置

由于使用k8s集群，而通过集群部署需要使用kubectl客户端，故手动创建了一个kubectl docker镜像，使用gitlab触发dockerhub构建，构建内容公开透明，可放心使用，如有其它版本的构建需求也可提pull request，会在后面补充，目前用到的只有1.14.0

fjy8018/kubectl

有kubectl客户端，还需要配置连接TLS和连接账户

为了保障安全，新建一个专门访问该工程命名空间的ServiceAccount

apiVersion: v1
kind: ServiceAccount
metadata:
 name: hmdt-gitlab-ci
 namespace: hmdt

利用集群提供的RBAC机制，为该账户授予该命名空间的admin权限

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
 name: hmdt-gitlab-role
 namespace: hmdt
subjects:
 - kind: ServiceAccount
 name: hmdt-gitlab-ci
 namespace: hmdt
roleRef:
 apiGroup: rbac.authorization.k8s.io
 kind: Role
 name: admin

创建后在查询其在k8s集群中生成的唯一名称，此处为hmdt-gitlab-ci-token-86n89

$ kubectl describe sa hmdt-gitlab-ci -n hmdt
Name: hmdt-gitlab-ci
Namespace: hmdt
Labels: <none>
Annotations: kubectl.`Kubernetes`.io/last-applied-configuration:
 {"apiVersion":"v1","kind":"ServiceAccount","metadata":{"annotations":{},"name":"hmdt-gitlab-ci","namespace":"hmdt"}}
Image pull secrets: <none>
Mountable secrets: hmdt-gitlab-ci-token-86n89
Tokens: hmdt-gitlab-ci-token-86n89
Events: <none>

然后根据上面的Secret找到CA证书

$ kubectl get secret hmdt-gitlab-ci-token-86n89 -n hmdt -o json | jq -r '.data["ca.crt"]' | base64 -d

再找到对应的 Token

$ kubectl get secret hmdt-gitlab-ci-token-86n89 -n hmdt -o json | jq -r '.data.token' | base64 -d

Kubernetes关联GitLab配置

进入gitlab Kubernetes集群配置页面，填写相关信息，让gitlab自动连接上集群环境

注意，需要将此处取消勾选，否则gitlab会自动创建新的用户账户，而不使用已经创建好的用户账户，在运行过程中会报无权限错误

不取消导致的报错如下，gitlab创建了新的用户账户hmdt-prod-service-account，但没有操作指定命名空间的权限

GitLab环境配置

创建环境

名称和url可以按需自定义

CI脚本配置

最终配置CI文件如下，该文件使用DinD方式构建Dockerfile

image: docker:19.03

variables:
 MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode -Dmaven.test.skip=true"
 MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
 DOCKER_DRIVER: overlay
 DOCKER_HOST: tcp://localhost:2375
 DOCKER_TLS_CERTDIR: ""
 SPRING_PROFILES_ACTIVE: docker
 IMAGE_VERSION: "1.8.6"
 DOCKER_REGISTRY_MIRROR: "https://XXX.mirror.aliyuncs.com"

stages:
 - test
 - package
 - review
 - deploy

maven-build:
 image: maven:3-jdk-8
 stage: test
 retry: 2
 script:
 - mvn $MAVEN_CLI_OPTS clean package -U -B -T 2C
 artifacts:
 expire_in: 1 week
 paths:
 - target/*.jar

maven-scan:
 stage: test
 retry: 2
 image: maven:3-jdk-8
 script:
 - mvn $MAVEN_CLI_OPTS verify sonar:sonar

maven-deploy:
 stage: deploy
 retry: 2
 image: maven:3-jdk-8
 script:
 - mvn $MAVEN_CLI_OPTS deploy

docker-harbor-build:
 image: docker:19.03
 stage: package
 retry: 2
 services:
 - name: docker:19.03-dind
 alias: docker
 before_script:
 - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
 script:
 - docker build --pull -t "$CI_REGISTRY_IMAGE:$IMAGE_VERSION" .
 - docker push "$CI_REGISTRY_IMAGE:$IMAGE_VERSION"
 - docker logout $CI_REGISTRY

deploy_live:
 image: fjy8018/kubectl:v1.14.0
 stage: deploy
 retry: 2
 environment:
 name: prod
 url: https://XXXX
 script:
 - kubectl version
 - kubectl get pods -n hmdt
 - cd manifests/
 - sed -i "s/__IMAGE_VERSION_SLUG__/${IMAGE_VERSION}/" deployment.yaml
 - kubectl apply -f deployment.yaml
 - kubectl rollout status -f deployment.yaml
 - kubectl get pods -n hmdt

若需要使用Kaniko构建Dockerfile，则配置如下

注意，其中依赖的镜像gcr.io/kaniko-project/executor:debug属于谷歌镜像仓库，可能存在无法拉取的情况

image: docker:19.03

variables:
 MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode -Dmaven.test.skip=true"
 MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
 DOCKER_DRIVER: overlay
 DOCKER_HOST: tcp://localhost:2375
 DOCKER_TLS_CERTDIR: ""
 SPRING_PROFILES_ACTIVE: docker
 IMAGE_VERSION: "1.8.6"
 DOCKER_REGISTRY_MIRROR: "https://XXX.mirror.aliyuncs.com"

cache:
 paths:
 - target/

stages:
 - test
 - package
 - review
 - deploy

maven-build:
 image: maven:3-jdk-8
 stage: test
 retry: 2
 script:
 - mvn $MAVEN_CLI_OPTS clean package -U -B -T 2C
 artifacts:
 expire_in: 1 week
 paths:
 - target/*.jar

maven-scan:
 stage: test
 retry: 2
 image: maven:3-jdk-8
 script:
 - mvn $MAVEN_CLI_OPTS verify sonar:sonar

maven-deploy:
 stage: deploy
 retry: 2
 image: maven:3-jdk-8
 script:
 - mvn $MAVEN_CLI_OPTS deploy


docker-harbor-build:
 stage: package
 retry: 2
 image:
 name: gcr.io/kaniko-project/executor:debug
 entrypoint: [""]
 script:
 - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
 - /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY_IMAGE:$IMAGE_VERSION

deploy_live:
 image: fjy8018/kubectl:v1.14.0
 stage: deploy
 retry: 2
 environment:
 name: prod
 url: https://XXXX
 script:
 - kubectl version
 - kubectl get pods -n hmdt
 - cd manifests/
 - sed -i "s/__IMAGE_VERSION_SLUG__/${IMAGE_VERSION}/" deployment.yaml
 - kubectl apply -f deployment.yaml
 - kubectl rollout status -f deployment.yaml
 - kubectl get pods -n hmdt

执行流水线

runner自动扩缩容

Kubernetes中的runner会根据任务多少自动扩缩容，目前配置的上限为10个

Grafana也能监控到集群在构建过程中的资源使用情况

使用`DinD`构建`Dockerfile`结果

使用`Kaniko`构建`Dockerfile`的结果

部署结果

执行部署时gitlab会自动注入配置好的kubectl config

构建结果

部署完成后可在环境配置页中查看部署结果，只有成功的部署才会被记录

GitLab Api使用

Wed, 06 Nov 2019 15:01:04 +0800

使用场景

为了进行规范化配置管理工作，基于gitlab规范分支管理流程，经常需要新建分支、设置保护分支和操作分支封板等，若只有几个工程则直接在gitlab web界面上操作即可，一旦工程数量增多，则会消耗大量时间，故考虑可通过gitlab 提供的web api编写脚本进行自动化操作

Api指南

此处摘要部分官网API文档指南，所有api操作都需要在gitlab上申请访问令牌，该令牌应当至少包含api访问权限

获取工程列表

支持多种参数过滤，非必传

Get a list of all visible projects across GitLab for the authenticated user. When accessed without authentication, only public projects with “simple” fields are returned.
GET /projects
Attribute Type Required Description

archived boolean no Limit by archived status

visibility string no Limit by visibility public, internal, or private

order_by string no Return projects ordered by id, name, path, created_at, updated_at, or last_activity_at fields. Default is created_at

sort string no Return projects sorted in asc or desc order. Default is desc

search string no Return list of projects matching the search criteria

simple boolean no Return only limited fields for each project. This is a no-op without authentication as then only simple fields are returned.

owned boolean no Limit by projects explicitly owned by the current user

membership boolean no Limit by projects that the current user is a member of

starred boolean no Limit by projects starred by the current user

statistics boolean no Include project statistics

with_custom_attributes boolean no Include custom attributes in response (admins only)

with_issues_enabled boolean no Limit by enabled issues feature

with_merge_requests_enabled boolean no Limit by enabled merge requests feature

with_programming_language string no Limit by projects which use the given programming language

wiki_checksum_failed boolean no Limit projects where the wiki checksum calculation has failed (Introduced in GitLab Premium 11.2)

repository_checksum_failed boolean no Limit projects where the repository checksum calculation has failed (Introduced in GitLab Premium 11.2)

min_access_level integer no Limit by current user minimal access level

Attribute	Type	Required	Description
`archived`	boolean	no	Limit by archived status
`visibility`	string	no	Limit by visibility `public`, `internal`, or `private`
`order_by`	string	no	Return projects ordered by `id`, `name`, `path`, `created_at`, `updated_at`, or `last_activity_at` fields. Default is `created_at`
`sort`	string	no	Return projects sorted in `asc` or `desc` order. Default is `desc`
`search`	string	no	Return list of projects matching the search criteria
`simple`	boolean	no	Return only limited fields for each project. This is a no-op without authentication as then only simple fields are returned.
`owned`	boolean	no	Limit by projects explicitly owned by the current user
`membership`	boolean	no	Limit by projects that the current user is a member of
`starred`	boolean	no	Limit by projects starred by the current user
`statistics`	boolean	no	Include project statistics
`with_custom_attributes`	boolean	no	Include custom attributes in response (admins only)
`with_issues_enabled`	boolean	no	Limit by enabled issues feature
`with_merge_requests_enabled`	boolean	no	Limit by enabled merge requests feature
`with_programming_language`	string	no	Limit by projects which use the given programming language
`wiki_checksum_failed`	boolean	no	Limit projects where the wiki checksum calculation has failed (Introduced in GitLab Premium 11.2)
`repository_checksum_failed`	boolean	no	Limit projects where the repository checksum calculation has failed (Introduced in GitLab Premium 11.2)
`min_access_level`	integer	no	Limit by current user minimal access level

分支操作

文档地址

创建分支

需要传递仓库ID、源分支名、新分支名称

Create a new branch in the repository.
POST /projects/:id/repository/branches
Parameters:

Attribute Type Required Description

id integer yes ID or URL-encoded path of the project owned by the authenticated user.

branch string yes Name of the branch.

ref string yes Branch name or commit SHA to create branch from.

Example request:
curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/repository/branches?branch=newbranch&ref=master

Attribute	Type	Required	Description
`id`	integer	yes	ID or URL-encoded path of the project owned by the authenticated user.
`branch`	string	yes	Name of the branch.
`ref`	string	yes	Branch name or commit SHA to create branch from.

保护分支

分支保护等级说明

Valid access levels

The access levels are defined in the ProtectedRefAccess.allowed_access_levels method. Currently, these levels are recognized:
0 => No access
30 => Developer access
40 => Maintainer access
60 => Admin access

需要传递分支名称、访问级别等信息

Protects a single repository branch or several project repository branches using a wildcard protected branch.
POST /projects/:id/protected_branches
curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" 'https://gitlab.example.com/api/v4/projects/5/protected_branches?name=*-stable&push_access_level=30&merge_access_level=30&unprotect_access_level=40'
Attribute Type Required Description

id integer/string yes The ID or URL-encoded path of the project owned by the authenticated user

name string yes The name of the branch or wildcard

push_access_level string no Access levels allowed to push (defaults: 40, maintainer access level)

merge_access_level string no Access levels allowed to merge (defaults: 40, maintainer access level)

unprotect_access_level string no Access levels allowed to unprotect (defaults: 40, maintainer access level)

allowed_to_push array no Array of access levels allowed to push, with each described by a hash

allowed_to_merge array no Array of access levels allowed to merge, with each described by a hash

allowed_to_unprotect array no Array of access levels allowed to unprotect, with each described by a hash

code_owner_approval_required boolean no Prevent pushes to this branch if it matches an item in the CODEOWNERS file. (defaults: false)

Attribute	Type	Required	Description
`id`	integer/string	yes	The ID or URL-encoded path of the project owned by the authenticated user
`name`	string	yes	The name of the branch or wildcard
`push_access_level`	string	no	Access levels allowed to push (defaults: `40`, maintainer access level)
`merge_access_level`	string	no	Access levels allowed to merge (defaults: `40`, maintainer access level)
`unprotect_access_level`	string	no	Access levels allowed to unprotect (defaults: `40`, maintainer access level)
`allowed_to_push`	array	no	Array of access levels allowed to push, with each described by a hash
`allowed_to_merge`	array	no	Array of access levels allowed to merge, with each described by a hash
`allowed_to_unprotect`	array	no	Array of access levels allowed to unprotect, with each described by a hash
`code_owner_approval_required`	boolean	no	Prevent pushes to this branch if it matches an item in the `CODEOWNERS` file. (defaults: false)

取消保护分支

需要传递仓库ID、分支名称

Unprotects the given protected branch or wildcard protected branch.
DELETE /projects/:id/protected_branches/:name
curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" 'https://gitlab.example.com/api/v4/projects/5/protected_branches/*-stable'
Attribute Type Required Description

id integer/string yes The ID or URL-encoded path of the project owned by the authenticated user

name string yes The name of the branch

Attribute	Type	Required	Description
`id`	integer/string	yes	The ID or URL-encoded path of the project owned by the authenticated user
`name`	string	yes	The name of the branch

删除分支

需要传递仓库ID和分支名称

Delete a branch from the repository.

Note: In the case of an error, an explanation message is provided.
DELETE /projects/:id/repository/branches/:branch
Parameters:

Attribute Type Required Description

id integer/string yes ID or URL-encoded path of the project owned by the authenticated user.

branch string yes Name of the branch.

Example request:
curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/repository/branches/newbranch

Attribute	Type	Required	Description
`id`	integer/string	yes	ID or URL-encoded path of the project owned by the authenticated user.
`branch`	string	yes	Name of the branch.

接口测试

使用postman对接口进行测试

获取工程列表

通过该接口可以查询所有工程的ID，注意，该接口默认使用了分页

脚本编写

批量创建分支

定义工程ID数组，对数组进行遍历，并设置基础分支变量，方便调整，此处默认写死基础分支，有需要也可以通过变量传递

#!/usr/bin/env bash

# 访问令牌
PRIVATE_TOKEN="XXXX"
# 分支名称
branch_name=$1

base_branch="BasicBranch"

# 项目ID
projectids=(47 48 50 51 52 54 55 56 58 59 60 62 63 64 65 67 68 69 70 71 74 75 89 90 91 92 93 95 103 104 113 114 115 143)

for id in "${projectids[@]}" ; do
	curl --request POST --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" "http://10.139.6.26:7077/api/v4/projects/$id/repository/branches?branch=$branch_name&ref=$base_branch"
done

批量保护分支

此处写死了分支访问级别，可参照上文的参数进行配置

#!/usr/bin/env bash

# 访问令牌
PRIVATE_TOKEN="XXXX"
# 分支名称
branch_name=$1

# 项目ID
projectids=(47 48 50 51 52 54 55 56 58 59 60 62 63 64 65 67 68 69 70 71 74 75 89 90 91 92 93 95 103 104 113 114 115 143)

for id in "${projectids[@]}" ; do
 # 访问权限均为 Maintainer access
 curl --request POST --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" "http://10.139.6.26:7077/api/v4/projects/$id/protected_branches?name=$branch_name&push_access_level=40&merge_access_level=40&unprotect_access_level=40"
done

批量取消保护分支

调用对应的删除接口即可

#!/usr/bin/env bash

# 访问令牌
PRIVATE_TOKEN="XXXX"
# 分支名称
branch_name=$1

# 项目ID
projectids=(47 48 50 51 52 54 55 56 58 59 60 62 63 64 65 67 68 69 70 71 74 75 89 90 91 92 93 95 103 104 113 114 115 143)

for id in "${projectids[@]}" ; do
 # 访问权限均为 Maintainer access
 curl --request DELETE --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" "http://10.139.6.26:7077/api/v4/projects/$id/protected_branches/$branch_name"
done

批量分支封板

分支封板即设置访问级别为no one

#!/usr/bin/env bash

# 访问令牌
PRIVATE_TOKEN="XXXX"
# 分支名称
branch_name=$1

# 项目ID
projectids=(47 48 50 51 52 54 55 56 58 59 60 62 63 64 65 67 68 69 70 71 74 75 89 90 91 92 93 95 103 104 113 114 115 143)

for id in "${projectids[@]}" ; do
 # 分支封板
 curl --request POST --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" "http://10.139.6.26:7077/api/v4/projects/$id/protected_branches?name=$branch_name&push_access_level=0&merge_access_level=0&unprotect_access_level=40"
done

批量删除分支

删除分支属于危险操作，操作需谨慎，因为不管是否为保护分支，都可以直接删除，而且通过api操作不会弹出确认操作

#!/usr/bin/env bash

# 访问令牌
PRIVATE_TOKEN="XXXX"
# 分支名称
branch_name=$1

# 项目ID
projectids=(47 48 50 51 52 54 55 56 58 59 60 62 63 64 65 67 68 69 70 71 74 75 89 90 91 92 93 95 103 104 113 114 115 143)

for id in "${projectids[@]}" ; do
 echo "删除项目ID：$id 的 $branch_name 分支"
 curl --request DELETE --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" "http://10.139.6.26:7077/api/v4/projects/$id/repository/branches/$branch_name"
done

Kubernetes基础命令学习

Sun, 20 Oct 2019 13:00:00 +0800

配置命令语法提示

执行命令查看指南

$ kubectl completion -h
Output shell completion code for the specified shell (bash or zsh). The shell code must be evaluated to provide
interactive completion of kubectl commands. This can be done by sourcing it from the .bash_profile.

 Detailed instructions on how to do this are available here:
https://kubernetes.io/docs/tasks/tools/install-kubectl/#enabling-shell-autocompletion

 Note for zsh users: [1] zsh completions are only supported in versions of zsh >= 5.2

Examples:
 # Installing bash completion on macOS using homebrew
 ## If running Bash 3.2 included with macOS
 brew install bash-completion
 ## or, if running Bash 4.1+
 brew install bash-completion@2
 ## If kubectl is installed via homebrew, this should start working immediately.
 ## If you've installed via other means, you may need add the completion to your completion directory
 kubectl completion bash > $(brew --prefix)/etc/bash_completion.d/kubectl


 # Installing bash completion on Linux
 ## If bash-completion is not installed on Linux, please install the 'bash-completion' package
 ## via your distribution's package manager.
 ## Load the kubectl completion code for bash into the current shell
 source <(kubectl completion bash)
 ## Write bash completion code to a file and source if from .bash_profile
 kubectl completion bash > ~/.kube/completion.bash.inc
 printf "
 # Kubectl shell completion
 source '$HOME/.kube/completion.bash.inc'
 " >> $HOME/.bash_profile
 source $HOME/.bash_profile

 # Load the kubectl completion code for zsh[1] into the current shell
 source <(kubectl completion zsh)
 # Set the kubectl completion code for zsh[1] to autoload on startup
 kubectl completion zsh > "${fpath[1]}/_kubectl"

Usage:
 kubectl completion SHELL [options]

Use "kubectl options" for a list of global command-line options (applies to all commands).

按照指南操作即可

配置多个上下文环境

查看当前上下文配置

$ kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* minikube minikube minikube

编辑~/.kube/config文件新增需要的上下文配置即可，配置完成后查看上下文信息

$ kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
 kubeadm kubeadm kubernetes-admin
* minikube minikube minikube

切换上下文

$ kubectl config use-context kubeadm
Switched to context "kubeadm".

再次查看，可见切换成功

$ kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* kubeadm kubeadm kubernetes-admin
 minikube minikube minikube

Pod相关操作

默认查看`default`命名空间的Pod

$ kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-deployment-54f57cf6bf-pznmh 1/1 Running 0 52s
nginx-deployment-54f57cf6bf-r9q89 1/1 Running 0 52s

查看所有命名空间的Pod

$ kubectl get pod --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-58cc8c89f4-c297b 1/1 Running 0 10m
kube-system coredns-58cc8c89f4-spzhj 1/1 Running 0 10m
kube-system etcd-k8s-master 1/1 Running 0 9m57s
kube-system kube-apiserver-k8s-master 1/1 Running 0 10m
kube-system kube-controller-manager-k8s-master 1/1 Running 0 9m58s
kube-system kube-flannel-ds-amd64-6fwg5 1/1 Running 0 5m
kube-system kube-flannel-ds-amd64-6rchg 1/1 Running 0 9m12s
kube-system kube-flannel-ds-amd64-z4plh 1/1 Running 0 4m54s
kube-system kube-proxy-cdcs4 1/1 Running 0 5m
kube-system kube-proxy-nlmbb 1/1 Running 0 4m54s
kube-system kube-proxy-xlb62 1/1 Running 0 10m
kube-system kube-scheduler-k8s-master 1/1 Running 0 10m

指定Pod对应的app名称

$ kubectl get pod -l app=nginx
NAME READY STATUS RESTARTS AGE
nginx-deployment-54f57cf6bf-pznmh 1/1 Running 0 58s
nginx-deployment-54f57cf6bf-r9q89 1/1 Running 0 58s

查看更多信息

$ kubectl get deployment -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx-deployment 2/2 2 2 64s nginx nginx:1.7.9 app=nginx

查看Pod详细信息

$ kubectl describe pod
Name: nginx-deployment-54f57cf6bf-67llv
Namespace: default
Priority: 0
Node: k8s-node1/10.0.2.15
Start Time: Sun, 20 Oct 2019 10:19:42 +0800
Labels: app=nginx
 pod-template-hash=54f57cf6bf
Annotations: <none>
Status: Running
IP: 172.100.1.3
IPs:
 IP: 172.100.1.3
Controlled By: ReplicaSet/nginx-deployment-54f57cf6bf
Containers:
 nginx:
 Container ID: docker://ffd4b4fa3fd4678a61797cfdcba31ea2b0b36459b65a95100484eda4be286b9f
 Image: nginx:1.7.9
 Image ID: docker-pullable://nginx@sha256:e3456c851a152494c3e4ff5fcc26f240206abac0c9d794affb40e0714846c451
 Port: 80/TCP
 Host Port: 0/TCP
 State: Running
 Started: Sun, 20 Oct 2019 10:19:45 +0800
 Ready: True
 Restart Count: 0
 Environment: <none>
 Mounts:
 /var/run/secrets/kubernetes.io/serviceaccount from default-token-zjpq9 (ro)
Conditions:
 Type Status
 Initialized True
 Ready True
 ContainersReady True
 PodScheduled True
Volumes:
 default-token-zjpq9:
 Type: Secret (a volume populated by a Secret)
 SecretName: default-token-zjpq9
 Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Normal Scheduled <unknown> default-scheduler Successfully assigned default/nginx-deployment-54f57cf6bf-67llv to k8s-node1
 Normal Pulled 46s kubelet, k8s-node1 Container image "nginx:1.7.9" already present on machine
 Normal Created 46s kubelet, k8s-node1 Created container nginx
 Normal Started 45s kubelet, k8s-node1 Started container nginx


Name: nginx-deployment-54f57cf6bf-r9q89
Namespace: default
Priority: 0
Node: k8s-node2/10.0.2.15
Start Time: Sun, 20 Oct 2019 10:17:24 +0800
Labels: app=nginx
 pod-template-hash=54f57cf6bf
Annotations: <none>
Status: Running
IP: 172.100.2.3
IPs:
 IP: 172.100.2.3
Controlled By: ReplicaSet/nginx-deployment-54f57cf6bf
Containers:
 nginx:
 Container ID: docker://61894b3762d723eabe2510284bb107098bdbcb8da4f841eea863a9c7a59c5806
 Image: nginx:1.7.9
 Image ID: docker-pullable://nginx@sha256:e3456c851a152494c3e4ff5fcc26f240206abac0c9d794affb40e0714846c451
 Port: 80/TCP
 Host Port: 0/TCP
 State: Running
 Started: Sun, 20 Oct 2019 10:18:08 +0800
 Ready: True
 Restart Count: 0
 Environment: <none>
 Mounts:
 /var/run/secrets/kubernetes.io/serviceaccount from default-token-zjpq9 (ro)
Conditions:
 Type Status
 Initialized True
 Ready True
 ContainersReady True
 PodScheduled True
Volumes:
 default-token-zjpq9:
 Type: Secret (a volume populated by a Secret)
 SecretName: default-token-zjpq9
 Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Normal Scheduled <unknown> default-scheduler Successfully assigned default/nginx-deployment-54f57cf6bf-r9q89 to k8s-node2
 Normal Pulling 3m4s kubelet, k8s-node2 Pulling image "nginx:1.7.9"
 Normal Pulled 2m24s kubelet, k8s-node2 Successfully pulled image "nginx:1.7.9"
 Normal Created 2m22s kubelet, k8s-node2 Created container nginx
 Normal Started 2m22s kubelet, k8s-node2 Started container nginx

Deployment相关操作

部署Deployment

$ kubectl create -f nginx_deployment.yml
deployment.apps/nginx-deployment created

升级Deployment

apply兼有create的语义，故也可直接使用apply创建\部署deployment

$ kubectl apply -f nginx_deployment_scale.yml
deployment.apps/nginx-deployment configured

查看Deployment更多信息

$ kubectl get deployment -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx-deployment 2/2 2 2 5m28s nginx nginx:1.8 app=nginx

查看详细信息

$ kubectl describe deployments.apps
Name: nginx-deployment
Namespace: default
CreationTimestamp: Sun, 20 Oct 2019 10:17:24 +0800
Labels: <none>
Annotations: deployment.kubernetes.io/revision: 3
 kubectl.kubernetes.io/last-applied-configuration:
 {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"name":"nginx-deployment","namespace":"default"},"spec":{"replica...
Selector: app=nginx
Replicas: 2 desired | 2 updated | 2 total | 2 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
 Labels: app=nginx
 Containers:
 nginx:
 Image: nginx:1.9.1
 Port: 80/TCP
 Host Port: 0/TCP
 Environment: <none>
 Mounts: <none>
 Volumes: <none>
Conditions:
 Type Status Reason
 ---- ------ ------
 Progressing True NewReplicaSetAvailable
 Available True MinimumReplicasAvailable
OldReplicaSets: <none>
NewReplicaSet: nginx-deployment-56f8998dbc (2/2 replicas created)
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Normal ScalingReplicaSet 22m deployment-controller Scaled up replica set nginx-deployment-54f57cf6bf to 2
 Normal ScalingReplicaSet 18m deployment-controller Scaled up replica set nginx-deployment-9f46bb5 to 1
 Normal ScalingReplicaSet 17m deployment-controller Scaled down replica set nginx-deployment-54f57cf6bf to 1
 Normal ScalingReplicaSet 17m deployment-controller Scaled up replica set nginx-deployment-9f46bb5 to 2
 Normal ScalingReplicaSet 16m deployment-controller Scaled down replica set nginx-deployment-54f57cf6bf to 0
 Normal ScalingReplicaSet 15m deployment-controller Scaled up replica set nginx-deployment-9f46bb5 to 4
 Normal ScalingReplicaSet 12m deployment-controller Scaled down replica set nginx-deployment-9f46bb5 to 3
 Normal ScalingReplicaSet 6m25s deployment-controller Scaled up replica set nginx-deployment-56f8998dbc to 1
 Normal ScalingReplicaSet 5m43s deployment-controller Scaled down replica set nginx-deployment-9f46bb5 to 2
 Normal ScalingReplicaSet 63s (x6 over 5m43s) deployment-controller (combined from similar events): Scaled down replica set nginx-deployment-56f8998dbc to 2

Node相关

查看Node

$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 24m v1.16.2
k8s-node1 Ready <none> 18m v1.16.2
k8s-node2 Ready <none> 18m v1.16.2

查看更多信息

$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k8s-master Ready master 24m v1.16.2 10.0.2.15 <none> CentOS Linux 7 (Core) 3.10.0-957.12.2.el7.x86_64 docker://18.9.8
k8s-node1 Ready <none> 18m v1.16.2 10.0.2.15 <none> CentOS Linux 7 (Core) 3.10.0-957.12.2.el7.x86_64 docker://18.9.8
k8s-node2 Ready <none> 18m v1.16.2 10.0.2.15 <none> CentOS Linux 7 (Core) 3.10.0-957.12.2.el7.x86_64 docker://18.9.8

查看标签

$ kubectl get nodes --show-labels
NAME STATUS ROLES AGE VERSION LABELS
k8s-master Ready master 24m v1.16.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-master,kubernetes.io/os=linux,node-role.kubernetes.io/master=
k8s-node1 Ready <none> 18m v1.16.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-node1,kubernetes.io/os=linux
k8s-node2 Ready <none> 18m v1.16.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-node2,kubernetes.io/os=linux

设置角色标签

$ kubectl label nodes k8s-node1 node-role.kubernetes.io/worker=
node/k8s-node1 labeled

查看详细信息

$ kubectl describe nodes
Name: k8s-master
Roles: master
Labels: beta.kubernetes.io/arch=amd64
 beta.kubernetes.io/os=linux
 kubernetes.io/arch=amd64
 kubernetes.io/hostname=k8s-master
 kubernetes.io/os=linux
 node-role.kubernetes.io/master=
Annotations: flannel.alpha.coreos.com/backend-data: {"VtepMAC":"32:38:c5:18:f0:ed"}
 flannel.alpha.coreos.com/backend-type: vxlan
 flannel.alpha.coreos.com/kube-subnet-manager: true
 flannel.alpha.coreos.com/public-ip: 10.0.2.15
 kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
 node.alpha.kubernetes.io/ttl: 0
 volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Sun, 20 Oct 2019 10:05:02 +0800
Taints: node-role.kubernetes.io/master:NoSchedule
Unschedulable: false
Conditions:
 Type Status LastHeartbeatTime LastTransitionTime Reason Message
 ---- ------ ----------------- ------------------ ------ -------
 MemoryPressure False Sun, 20 Oct 2019 10:27:46 +0800 Sun, 20 Oct 2019 10:05:02 +0800 KubeletHasSufficientMemory kubelet has sufficient memory available
 DiskPressure False Sun, 20 Oct 2019 10:27:46 +0800 Sun, 20 Oct 2019 10:05:02 +0800 KubeletHasNoDiskPressure kubelet has no disk pressure
 PIDPressure False Sun, 20 Oct 2019 10:27:46 +0800 Sun, 20 Oct 2019 10:05:02 +0800 KubeletHasSufficientPID kubelet has sufficient PID available
 Ready True Sun, 20 Oct 2019 10:27:46 +0800 Sun, 20 Oct 2019 10:07:42 +0800 KubeletReady kubelet is posting ready status
Addresses:
 InternalIP: 10.0.2.15
 Hostname: k8s-master
Capacity:
 cpu: 2
 ephemeral-storage: 41921540Ki
 hugepages-2Mi: 0
 memory: 1882144Ki
 pods: 110
Allocatable:
 cpu: 2
 ephemeral-storage: 38634891201
 hugepages-2Mi: 0
 memory: 1779744Ki
 pods: 110
System Info:
 Machine ID: d74c50274729456aad2a6198de9863b3
 System UUID: D74C5027-4729-456A-AD2A-6198DE9863B3
 Boot ID: fb2ea7bd-afb2-4051-afb7-78d9f59ec6f6
 Kernel Version: 3.10.0-957.12.2.el7.x86_64
 OS Image: CentOS Linux 7 (Core)
 Operating System: linux
 Architecture: amd64
 Container Runtime Version: docker://18.9.8
 Kubelet Version: v1.16.2
 Kube-Proxy Version: v1.16.2
PodCIDR: 172.100.0.0/24
PodCIDRs: 172.100.0.0/24
Non-terminated Pods: (8 in total)
 Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
 --------- ---- ------------ ---------- --------------- ------------- ---
 kube-system coredns-58cc8c89f4-c297b 100m (5%) 0 (0%) 70Mi (4%) 170Mi (9%) 23m
 kube-system coredns-58cc8c89f4-spzhj 100m (5%) 0 (0%) 70Mi (4%) 170Mi (9%) 23m
 kube-system etcd-k8s-master 0 (0%) 0 (0%) 0 (0%) 0 (0%) 22m
 kube-system kube-apiserver-k8s-master 250m (12%) 0 (0%) 0 (0%) 0 (0%) 22m
 kube-system kube-controller-manager-k8s-master 200m (10%) 0 (0%) 0 (0%) 0 (0%) 22m
 kube-system kube-flannel-ds-amd64-6rchg 100m (5%) 100m (5%) 50Mi (2%) 50Mi (2%) 21m
 kube-system kube-proxy-xlb62 0 (0%) 0 (0%) 0 (0%) 0 (0%) 23m
 kube-system kube-scheduler-k8s-master 100m (5%) 0 (0%) 0 (0%) 0 (0%) 22m
Allocated resources:
 (Total limits may be over 100 percent, i.e., overcommitted.)
 Resource Requests Limits
 -------- -------- ------
 cpu 850m (42%) 100m (5%)
 memory 190Mi (10%) 390Mi (22%)
 ephemeral-storage 0 (0%) 0 (0%)
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Normal NodeHasSufficientMemory 23m (x8 over 23m) kubelet, k8s-master Node k8s-master status is now: NodeHasSufficientMemory
 Normal NodeHasNoDiskPressure 23m (x8 over 23m) kubelet, k8s-master Node k8s-master status is now: NodeHasNoDiskPressure
 Normal NodeHasSufficientPID 23m (x7 over 23m) kubelet, k8s-master Node k8s-master status is now: NodeHasSufficientPID
 Normal Starting 23m kube-proxy, k8s-master Starting kube-proxy.


Name: k8s-node1
Roles: <none>
Labels: beta.kubernetes.io/arch=amd64
 beta.kubernetes.io/os=linux
 kubernetes.io/arch=amd64
 kubernetes.io/hostname=k8s-node1
 kubernetes.io/os=linux
Annotations: flannel.alpha.coreos.com/backend-data: {"VtepMAC":"16:5c:40:b1:14:b6"}
 flannel.alpha.coreos.com/backend-type: vxlan
 flannel.alpha.coreos.com/kube-subnet-manager: true
 flannel.alpha.coreos.com/public-ip: 10.0.2.15
 kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
 node.alpha.kubernetes.io/ttl: 0
 volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Sun, 20 Oct 2019 10:11:03 +0800
Taints: <none>
Unschedulable: false
Conditions:
 Type Status LastHeartbeatTime LastTransitionTime Reason Message
 ---- ------ ----------------- ------------------ ------ -------
 MemoryPressure False Sun, 20 Oct 2019 10:28:08 +0800 Sun, 20 Oct 2019 10:11:03 +0800 KubeletHasSufficientMemory kubelet has sufficient memory available
 DiskPressure False Sun, 20 Oct 2019 10:28:08 +0800 Sun, 20 Oct 2019 10:11:03 +0800 KubeletHasNoDiskPressure kubelet has no disk pressure
 PIDPressure False Sun, 20 Oct 2019 10:28:08 +0800 Sun, 20 Oct 2019 10:11:03 +0800 KubeletHasSufficientPID kubelet has sufficient PID available
 Ready True Sun, 20 Oct 2019 10:28:08 +0800 Sun, 20 Oct 2019 10:11:34 +0800 KubeletReady kubelet is posting ready status
Addresses:
 InternalIP: 10.0.2.15
 Hostname: k8s-node1
Capacity:
 cpu: 1
 ephemeral-storage: 41921540Ki
 hugepages-2Mi: 0
 memory: 1882296Ki
 pods: 110
Allocatable:
 cpu: 1
 ephemeral-storage: 38634891201
 hugepages-2Mi: 0
 memory: 1779896Ki
 pods: 110
System Info:
 Machine ID: 5d806dde9b0444bdb5c9800a228267d8
 System UUID: 5D806DDE-9B04-44BD-B5C9-800A228267D8
 Boot ID: e6b7222c-686c-49cb-8e56-9bb263852d36
 Kernel Version: 3.10.0-957.12.2.el7.x86_64
 OS Image: CentOS Linux 7 (Core)
 Operating System: linux
 Architecture: amd64
 Container Runtime Version: docker://18.9.8
 Kubelet Version: v1.16.2
 Kube-Proxy Version: v1.16.2
PodCIDR: 172.100.1.0/24
PodCIDRs: 172.100.1.0/24
Non-terminated Pods: (4 in total)
 Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
 --------- ---- ------------ ---------- --------------- ------------- ---
 default nginx-deployment-9f46bb5-7vlb7 0 (0%) 0 (0%) 0 (0%) 0 (0%) 3m53s
 default nginx-deployment-9f46bb5-c7czx 0 (0%) 0 (0%) 0 (0%) 0 (0%) 6m35s
 kube-system kube-flannel-ds-amd64-6fwg5 100m (10%) 100m (10%) 50Mi (2%) 50Mi (2%) 17m
 kube-system kube-proxy-cdcs4 0 (0%) 0 (0%) 0 (0%) 0 (0%) 17m
Allocated resources:
 (Total limits may be over 100 percent, i.e., overcommitted.)
 Resource Requests Limits
 -------- -------- ------
 cpu 100m (10%) 100m (10%)
 memory 50Mi (2%) 50Mi (2%)
 ephemeral-storage 0 (0%) 0 (0%)
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Normal Starting 17m kubelet, k8s-node1 Starting kubelet.
 Normal NodeHasSufficientMemory 17m kubelet, k8s-node1 Node k8s-node1 status is now: NodeHasSufficientMemory
 Normal NodeHasNoDiskPressure 17m kubelet, k8s-node1 Node k8s-node1 status is now: NodeHasNoDiskPressure
 Normal NodeHasSufficientPID 17m kubelet, k8s-node1 Node k8s-node1 status is now: NodeHasSufficientPID
 Normal NodeAllocatableEnforced 17m kubelet, k8s-node1 Updated Node Allocatable limit across pods
 Normal Starting 17m kube-proxy, k8s-node1 Starting kube-proxy.
 Normal NodeReady 16m kubelet, k8s-node1 Node k8s-node1 status is now: NodeReady


Name: k8s-node2
Roles: <none>
Labels: beta.kubernetes.io/arch=amd64
 beta.kubernetes.io/os=linux
 kubernetes.io/arch=amd64
 kubernetes.io/hostname=k8s-node2
 kubernetes.io/os=linux
Annotations: flannel.alpha.coreos.com/backend-data: {"VtepMAC":"76:49:24:1f:e5:d6"}
 flannel.alpha.coreos.com/backend-type: vxlan
 flannel.alpha.coreos.com/kube-subnet-manager: true
 flannel.alpha.coreos.com/public-ip: 10.0.2.15
 kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
 node.alpha.kubernetes.io/ttl: 0
 volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Sun, 20 Oct 2019 10:11:09 +0800
Taints: <none>
Unschedulable: false
Conditions:
 Type Status LastHeartbeatTime LastTransitionTime Reason Message
 ---- ------ ----------------- ------------------ ------ -------
 MemoryPressure False Sun, 20 Oct 2019 10:27:42 +0800 Sun, 20 Oct 2019 10:11:09 +0800 KubeletHasSufficientMemory kubelet has sufficient memory available
 DiskPressure False Sun, 20 Oct 2019 10:27:42 +0800 Sun, 20 Oct 2019 10:11:09 +0800 KubeletHasNoDiskPressure kubelet has no disk pressure
 PIDPressure False Sun, 20 Oct 2019 10:27:42 +0800 Sun, 20 Oct 2019 10:11:09 +0800 KubeletHasSufficientPID kubelet has sufficient PID available
 Ready True Sun, 20 Oct 2019 10:27:42 +0800 Sun, 20 Oct 2019 10:11:40 +0800 KubeletReady kubelet is posting ready status
Addresses:
 InternalIP: 10.0.2.15
 Hostname: k8s-node2
Capacity:
 cpu: 1
 ephemeral-storage: 41921540Ki
 hugepages-2Mi: 0
 memory: 1882296Ki
 pods: 110
Allocatable:
 cpu: 1
 ephemeral-storage: 38634891201
 hugepages-2Mi: 0
 memory: 1779896Ki
 pods: 110
System Info:
 Machine ID: 7c59ba12638e4fdfa20cfeda0aa2fda1
 System UUID: 7C59BA12-638E-4FDF-A20C-FEDA0AA2FDA1
 Boot ID: e402f75c-42e7-454d-aaca-f96853091c15
 Kernel Version: 3.10.0-957.12.2.el7.x86_64
 OS Image: CentOS Linux 7 (Core)
 Operating System: linux
 Architecture: amd64
 Container Runtime Version: docker://18.9.8
 Kubelet Version: v1.16.2
 Kube-Proxy Version: v1.16.2
PodCIDR: 172.100.2.0/24
PodCIDRs: 172.100.2.0/24
Non-terminated Pods: (3 in total)
 Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
 --------- ---- ------------ ---------- --------------- ------------- ---
 default nginx-deployment-9f46bb5-mv4b7 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m52s
 kube-system kube-flannel-ds-amd64-z4plh 100m (10%) 100m (10%) 50Mi (2%) 50Mi (2%) 17m
 kube-system kube-proxy-nlmbb 0 (0%) 0 (0%) 0 (0%) 0 (0%) 17m
Allocated resources:
 (Total limits may be over 100 percent, i.e., overcommitted.)
 Resource Requests Limits
 -------- -------- ------
 cpu 100m (10%) 100m (10%)
 memory 50Mi (2%) 50Mi (2%)
 ephemeral-storage 0 (0%) 0 (0%)
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Normal Starting 17m kubelet, k8s-node2 Starting kubelet.
 Normal NodeHasSufficientMemory 17m kubelet, k8s-node2 Node k8s-node2 status is now: NodeHasSufficientMemory
 Normal NodeHasNoDiskPressure 17m kubelet, k8s-node2 Node k8s-node2 status is now: NodeHasNoDiskPressure
 Normal NodeHasSufficientPID 17m kubelet, k8s-node2 Node k8s-node2 status is now: NodeHasSufficientPID
 Normal NodeAllocatableEnforced 17m kubelet, k8s-node2 Updated Node Allocatable limit across pods
 Normal Starting 17m kube-proxy, k8s-node2 Starting kube-proxy.
 Normal NodeReady 16m kubelet, k8s-node2 Node k8s-node2 status is now: NodeReady

服务伸缩

`scale`命令方式

$ kubectl scale --replicas=6 deployment/nginx-deployment
deployment.apps/nginx-deployment scaled

查看副本数

$ kubectl get replicaset
NAME DESIRED CURRENT READY AGE
nginx-deployment-54f57cf6bf 0 0 0 21m
nginx-deployment-56f8998dbc 6 6 3 5m7s
nginx-deployment-9f46bb5 0 0 0 16m

使用GitHub和DockerHub自动构建镜像

Wed, 31 Jul 2019 22:36:41 +0800

环境

DockerHub账号、GitHub账号

步骤

构建脚本

此处以构建mycat的docker镜像为例，在本地准备好可以运行的构建脚本Dockerfile，尽可能不使用准备好的配置文件，保证镜像的安全、纯净，并且利用环境变量配置精简构建脚本

FROM openjdk:8-jre
MAINTAINER fjy8018 fjy8018@gmail.com

# Mycat 版本
ENV MYCAT_VERSION 1.6-RELEASE
ENV MYCAT_PKG Mycat-server-1.6-RELEASE-20161028204710-linux.tar.gz

WORKDIR /usr/local

# 缓存层
RUN wget http://dl.mycat.io/$MYCAT_VERSION/$MYCAT_PKG
RUN apt-get update && \
 apt-get install -y procps && \
 apt-get clean && \
 tar -zxf /usr/local/$MYCAT_PKG && \
 rm -rf /usr/local/$MYCAT_PKG

VOLUME /usr/local/mycat/conf

EXPOSE 8066 9066

ENTRYPOINT /usr/local/mycat/bin/mycat console

创建GitHub仓库

在GitHub上创建对应的仓库，并推送Dockerfile

链接DockerHub与GitHub

登陆DockerHub选择repository新建仓库

首次链接可以参照指引操作，主要是做OAuth 授权

创建构建分支和标签

连接GitHub后可以选择需要构建的仓库，同时展开构建选项，可以指定不同分支构建出的标签，默认master分支构建为latest版本

触发构建

第一次会自动触发构建，随后代码推送都会触发构建

构建结果

在管理页面可以查看构建时长和构建结果

结果

用户视角

用户视角可以看到构建的GitHub仓库

自动构建产生的分支

构建历史记录

SpringBoot 2.x quartz多数据定时任务源配置

Mon, 29 Jul 2019 15:04:00 +0800

基础环境

SpringBoot 2.0 开始官方提供了对quartz定时任务的自动配置支持依赖spring-boot-starter-quartz组件，无需自行集成

<dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-quartz</artifactId>
</dependency>

环境版本：

SpringBoot 2.1.6.RELEASE
Java 8

定时任务配置

添加依赖

在pom.xml中添加quartz、Jpa、mysql驱动、lombok依赖

<dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
 <groupId>mysql</groupId>
 <artifactId>mysql-connector-java</artifactId>
 <scope>runtime</scope>
</dependency>
<dependency>
 <groupId>org.projectlombok</groupId>
 <artifactId>lombok</artifactId>
 <optional>true</optional>
</dependency>
<dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-quartz</artifactId>
</dependency>

添加配置文件

添加数据库配置

spring:
 datasource:
 primary:
 driver-class-name: com.mysql.cj.jdbc.Driver
 username: root
 password: hmdt
 url: jdbc:mysql://db/bussines_db?useSSL=false&useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai
 quartz:
 url: jdbc:mysql://db/bussines_quartz?useSSL=false&useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai
 driver-class-name: com.mysql.cj.jdbc.Driver
 username: root
 password: hmdt

 quartz:
 #相关属性配置
 properties:
 org:
 quartz:
 scheduler:
 instanceName: clusteredScheduler
 instanceId: AUTO
 jobStore:
 # 数据源名称
 dataSource: quartzDataSource
 class: org.quartz.impl.jdbcjobstore.JobStoreTX
 driverDelegateClass: org.quartz.impl.jdbcjobstore.StdJDBCDelegate
 tablePrefix: QRTZ_
 isClustered: true
 clusterCheckinInterval: 1000
 useProperties: false
 threadPool:
 class: org.quartz.simpl.SimpleThreadPool
 threadCount: 10
 threadPriority: 5
 threadsInheritContextClassLoaderOfInitializingThread: true
 job-store-type: jdbc
 #初始化表结构
 jdbc:
 initialize-schema: never
 jpa:
 hibernate:
 ddl-auto: update
 show-sql: true
 database-platform: org.hibernate.dialect.MySQL55Dialect
 properties:
 hibernate:
 format_sql: true
 use_sql_comments: true

数据库配置也有样例配在spring.quartz.properties中，但亲测配置无效，并且由于还需要指定连接池等信息，此处配置在spring的连接池中更容易管理

下载官网建表语句

尽管使用了JPA，但由于没有实体关联，spring并不会帮我们自动建表，还需要去官网下载sql脚本

quartz官网

官网压缩包路径quartz-2.3.0-SNAPSHOT\src\org\quartz\impl\jdbcjobstore中支持大量数据库建表语句，选择对应的使用即可

配置多数据源

多配置源之前在文章中详细说过配置样例和原因，这里不再赘述

编写数据库配置，统一使用HikariCP连接池

package com.gcb.invest.config;

import com.zaxxer.hikari.HikariDataSource;
import org.springframework.boot.autoconfigure.jdbc.DataSourceProperties;
import org.springframework.boot.autoconfigure.quartz.QuartzDataSource;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;

/**
 * 定时任务数据源配置
 *
 * @author F嘉阳
 * @date 2019-07-28 21:52
 */
@Configuration
public class DataSourceConfig {

 @Bean
 @Primary
 @ConfigurationProperties("spring.datasource.primary")
 public DataSourceProperties primaryDataSourceProperties() {
 return new DataSourceProperties();
 }

 @Bean
 @Primary
 @ConfigurationProperties("spring.datasource.primary.configuration")
 public HikariDataSource firstDataSource() {
 return primaryDataSourceProperties().initializeDataSourceBuilder()
 .type(HikariDataSource.class).build();
 }

 @Bean
 @ConfigurationProperties("spring.datasource.quartz")
 public DataSourceProperties quartzDataSourceProperties() {
 return new DataSourceProperties();
 }

 @Bean
 @QuartzDataSource
 @ConfigurationProperties("spring.datasource.quartz.configuration")
 public HikariDataSource quartzDataSource() {
 return quartzDataSourceProperties().initializeDataSourceBuilder()
 .type(HikariDataSource.class).build();
 }
}

定时任务配置

配置样例1——Bean注入

编写定时任务，由于经过Spring的封装，使用变得更加方便

此处通过内部类配置定时任务，并使用@DisallowConcurrentExecution注解指定其在分布式环境下不可并发执行

import lombok.extern.slf4j.Slf4j;
import org.quartz.*;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.quartz.QuartzJobBean;

/**
 * 定时任务配置
 *
 * @author ：F嘉阳
 * @date ：2019/7/29 9:16
 */
@Slf4j
@Configuration
public class QuartzConfig {

 /**
 * 测试定时任务构建
 *
 * @return
 */
 @Bean
 public JobDetail testTaskJobDetail() {
 return JobBuilder.newJob(TestTask.class)
 .withIdentity(TestTask.class.getName())
 .storeDurably(true)
 .build();
 }

 /**
 * 测试定时任务配置
 *
 * @return
 */
 @Bean
 public Trigger testTaskTrigger() {
 CronScheduleBuilder scheduleBuilder = CronScheduleBuilder.cronSchedule("0/2 * * * * ?");
 return TriggerBuilder.newTrigger()
 .forJob(testTaskJobDetail())
 .withIdentity(TestTask.class.getName())
 .withSchedule(scheduleBuilder)
 .build();
 }

 @DisallowConcurrentExecution
 private class TestTask extends QuartzJobBean {
 @Override
 protected void executeInternal(JobExecutionContext jobExecutionContext) throws JobExecutionException {
 log.debug("执行测试定时任务");
 }
 }
}

配置样例2——调度器注入

另一种方式可使用注入调度器手动构建任务，不过需要调用该方法之后调度器才会被创建

import org.quartz.*;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import java.util.Date;
import java.util.UUID;

/**
 * @author ：F嘉阳
 * @date ：2019/7/29 8:59
 */
@Service
@Transactional(rollbackFor = Exception.class)
public class QuartzTimerService {

 @Autowired
 private Scheduler scheduler;

 public void buildGoodStockTimer() throws Exception {
 //任务名称
 String name = UUID.randomUUID().toString();
 //任务所属分组
 String group = CustomQuartzJob.class.getName();

 CronScheduleBuilder scheduleBuilder = CronScheduleBuilder.cronSchedule("0/1 * * * * ?");
 //创建任务
 JobDetail jobDetail = JobBuilder.newJob(CustomQuartzJob.class).withIdentity(name,group).build();
 //创建任务触发器
 Trigger trigger = TriggerBuilder.newTrigger().withIdentity(name,group).withSchedule(scheduleBuilder).build();
 //将触发器与任务绑定到调度器内
 scheduler.scheduleJob(jobDetail, trigger);
 }
}

定时任务类

import lombok.extern.slf4j.Slf4j;
import org.quartz.DisallowConcurrentExecution;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;
import org.springframework.scheduling.quartz.QuartzJobBean;

/**
 * @author ：F嘉阳
 * @date ：2019/7/29 8:55
 */
@Slf4j
@DisallowConcurrentExecution
public class CustomQuartzJob extends QuartzJobBean {
 @Override
 protected void executeInternal(JobExecutionContext jobExecutionContext) throws JobExecutionException {
 log.info("执行定时任务");
 }
}

执行结果

执行成功

在分布式环境下自动进行作业节点迁移

Docker 19.03 版本对原Gitlab-Runner的影响

Sun, 28 Jul 2019 13:09:00 +0800

版本变化始末

在 2019.07.24 Docker社区发布了19.03的新版本，在此前版本号为18.0x

新版本带来的坑

原本没太在意docker版本的变化，因为多数情况都是兼容旧版的所有特性和功能的

结果在近期的CICD构建中，在ci脚本没有变化的情况下突然所有runner的构建都失败了

随后立即采取以下措施尝试修复

工程版本回退到上一次可用状态
更换runner服务器和容器
更改ci脚本
重启docker服务

均无效

执行构建步骤出现了以往没有出现过的警告信息

Running with gitlab-runner 11.8.0 (4745a6f3)
 on docker-in-docker-runner 4iAG2sYM
Using Docker executor with image docker:stable ...
Starting service docker:dind ...
Pulling docker image docker:dind ...
Using docker image sha256:fd0c64832f7e46b63a180e6000dbba7ad7a63542c5764841cba73429ba74a39e for docker:dind ...
Waiting for services to be up and running...

*** WARNING: Service runner-4iAG2sYM-project-89-concurrent-0-docker-0 probably didn't start properly.

Health check error:
service "runner-4iAG2sYM-project-89-concurrent-0-docker-0-wait-for-service" timeout

Health check container logs:

...

2019-07-24T03:11:35.365911823Z time="2019-07-24T03:11:35.365537741Z" level=info msg="API listen on [::]:2376"
2019-07-24T03:11:35.366107674Z time="2019-07-24T03:11:35.365626706Z" level=info msg="API listen on /var/run/docker.sock"

*********

Pulling docker image docker:stable ...
Using docker image sha256:c4154a2b47a18fe9437956ab981bd5924b19e7ae3eb3ed60c42cf8dfa394d550 for docker:stable ...

最终报错信息

Downloading artifacts for maven-build (280)...
Downloading artifacts from coordinator... ok id=280 responseStatus=200 OK token=L9iuC5Ss
$ docker build -t $CONTAINER_IMAGE:$IMAGE_VERSION .
Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?
ERROR: Job failed: exit code 1

寻找解决方法

该错误在早期版本就出现过，但与本次情况均不符合，以往通过挂载容器路径即可，而本次情况是runner服务和构建脚本均不变的情况下突然出现构建失败的情况

最终在社区ISSUS讨论中发现了相关讨论，并且问题发生时间相近

Jobs failing with “Cannot connect to the Docker daemon”

官方开发者解释当天几个小时前发布的新版本 19.03默认开启了TLS确实会影响已有的Docker in Docker 模式的构建环境

同时官方开发人员建议自行部署的runner建议开启TLS功能

该开发者在另一个讨论中提供了19.03环境下的runner启动和配置样板，同时还有样例ci脚本

Update DinD example for 19.03

解决方法

最好可以卸载旧版本的Docker，尽管运行runner和dind环境都基于docker容器中
加入TLS相关runner注册配置

此处采用更新Docker版本并使用最新版的runner镜像进行注册

更新docker版本

官方文档

卸载旧docker

$ sudo yum remove docker \
 docker-client \
 docker-client-latest \
 docker-common \
 docker-latest \
 docker-latest-logrotate \
 docker-logrotate \
 docker-engine

配置生产环境的依赖仓库

$ sudo yum-config-manager \
 --add-repo \
 https://download.docker.com/linux/centos/docker-ce.repo

官方建议的安装新版Docker命令

$ sudo yum install docker-ce docker-ce-cli containerd.io

但经过这次事件后个人不建议使用该命令，而是应该手动指定所有Docker环境版本号，防止以后版本更新导致不兼容问题

查看当前可以安装的所有版本

$ sudo yum list docker-ce --showduplicates | sort -r
 * updates: repos.lax.quadranet.com
Loaded plugins: fastestmirror
 * extras: mirror.hostduplex.com
 * epel: d2lzkl7pfhq30w.cloudfront.net
 * elrepo-kernel: repos.lax-noc.com
docker-ce.x86_64 3:19.03.1-3.el7 docker-ce-stable
docker-ce.x86_64 3:19.03.0-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.8-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.7-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.6-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.5-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.4-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.3-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.2-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.1-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.0-3.el7 docker-ce-stable
docker-ce.x86_64 18.06.3.ce-3.el7 docker-ce-stable
docker-ce.x86_64 18.06.2.ce-3.el7 docker-ce-stable
docker-ce.x86_64 18.06.1.ce-3.el7 docker-ce-stable
docker-ce.x86_64 18.06.0.ce-3.el7 docker-ce-stable
docker-ce.x86_64 18.03.1.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 18.03.0.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.12.1.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.12.0.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.09.1.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.09.0.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.06.2.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.06.1.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.06.0.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.03.3.ce-1.el7 docker-ce-stable
docker-ce.x86_64 17.03.2.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.03.1.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.03.0.ce-1.el7.centos docker-ce-stable
Determining fastest mirrors
 * base: mirror.hostduplex.com
Available Packages

指定最新版本

$ sudo yum install docker-ce-19.03.1 docker-ce-cli-19.03.1 containerd.io

启动docker

$ sudo systemctl start docker

设置开机自启

$ sudo systemctl enable docker

安装Gitlab Runner

官方对新版启用TLS的注册样例

sudo gitlab-runner register -n \
 --url https://gitlab.com/ \
 --registration-token REGISTRATION_TOKEN \
 --executor docker \
 --description "My Docker Runner" \
 --docker-image "docker:stable" \
 --docker-privileged \
 --docker-volumes "/certs/client"

此处采用容器化注册runner命令为

$ docker run --rm -t -i \
-v /etc/gitlab-runner/config:/etc/gitlab-runner \
gitlab/gitlab-runner:alpine-v12.1.0 register \
--url "https://gitlab.com/" \
--registration-token REGISTRATION_TOKEN \
--executor docker \
--non-interactive \
--run-untagged \
--tag-list "Runner,Docker" \
--run-untagged="true" \
--locked="false" \
--description "My Docker Runner" \
--docker-image "docker:stable" \
--docker-privileged \
--docker-volumes "/certs/client"

主要的变化至于增加了证书挂载--docker-volumes "/certs/client"配置，同时指定了runner的版本号

启动runner

$ docker run --name gitlab-runner --restart always -d \
-v /etc/gitlab-runner/config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:alpine-v12.1.0

修改构建脚本

官方对新版启用TLS的构建脚本样例，指定了构建时使用的docker版本与外部一直，防止不兼容的问题

同时加入了TLS环境变量配置

 image: docker:19.03

 variables:
 # When using dind service we need to instruct docker, to talk with
 # the daemon started inside of the service. The daemon is
 # available with a network connection instead of the default
 # /var/run/docker.sock socket. docker:19.03-dind does this
 # automatically by setting the DOCKER_HOST in
 # https://github.com/docker-library/docker/blob/d45051476babc297257df490d22cbd806f1b11e4/19.03/docker-entrypoint.sh#L23-L29
 #
 # The 'docker' hostname is the alias of the service container as described at
 # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services.
 #
 # Note that if you're using the Kubernetes executor, the variable should be set to
 # tcp://localhost:2376/ because of how the Kubernetes executor connects services
 # to the job container
 # DOCKER_HOST: tcp://localhost:2376/
 #
 # When using dind, it's wise to use the overlayfs driver for
 # improved performance.
 DOCKER_DRIVER: overlay2
 # Specify to Docker where to create the certificates, Docker will
 # create them automatically on boot, and will create
 # `/certs/client` that will be shared between the service and
 # build container.
 DOCKER_TLS_CERTDIR: "/certs"

 services:
 - docker:19.03-dind

 before_script:
 - echo $DOCKER_HOST
 - docker info

 build:
 stage: build
 script:
 - docker build -t my-docker-image .

在此前自己的构建脚本是按照官网提供的样板，没有指定版本，因此会导致每次构建都会使用当前最新的版本作为基础，最终导致外部Docker和内部Docker容器版本不一致

官网示例DinD构建脚本

image: docker:stable
services:
 - docker:dind
...

按照社区配置修改后重新构建则成功

总结

对于Docker等云服务平台仍然需要特别注意版本变化，为了防止不兼容的情况，在生产环境尽量不要使用latest版本标签，而是应该手动指定所有参与构建的版本环境

阿里云CDN + nginx多级代理获取客户端IP

Wed, 03 Apr 2019 10:01:44 +0800

无Nginx代理场景

业务层通过获取请求头参数即可拿到客户端IP

request.getRemoteAddr();

一级Nginx代理

使用代理后直接读取请求头参数会读取到代理服务器的IP地址，而非真实客户端IP

解决方法是添加Nginx请求头参数提前保存客户端IP

nginx.conf配置加入内容

location / {
 ...
 # IP地址转发
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header X-Real-Port $remote_port;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 }

业务层读取nginx配置的请求头参数即可

public static String getIpAddr(HttpServletRequest request) {
 String ip = request.getHeader("X-Real-IP");
 if (ip == null || ip.length() == 0 || " unknown ".equalsIgnoreCase(ip)) {
 ip = request.getHeader("Proxy-Client-IP");
 log.info("【Proxy-Client-IP】 {}", ip);
 }
 if (ip == null || ip.length() == 0 || " unknown ".equalsIgnoreCase(ip)) {
 ip = request.getHeader("WL-Proxy-Client-IP");
 log.info("【WL-Proxy-Client-IP】{}", ip);
 }
 if (ip == null || ip.length() == 0 || " unknown ".equalsIgnoreCase(ip)) {
 ip = request.getHeader("X-Forwarded-For");
 log.info("【X-Forwarded-For】{}", ip);
 }
 if (ip == null || ip.length() == 0 || " unknown ".equalsIgnoreCase(ip)) {
 ip = request.getRemoteAddr();
 log.info("【unknown】{}", ip);
 }
 return ip;
 }

多级Nginx代理

若存在多级Nginx代理，则需要在第一级代理时获取客户端IP，在后续代理逐层传递

第一级代理配置同上，第N级代理配置nginx.conf如下

location /{
 # IP地址转发
 proxy_set_header X-Real-IP $X-Real-IP;
 proxy_set_header X-Real-Port $X-Real-Port;
 proxy_set_header X-Forwarded-For $X-Forwarded-For;
}

业务层保持不变即可读取到传递的IP地址

阿里云CDN转发

在已经存在nginx代理的场景下，加入CDN后源客户端IP在CDN处被转发，故第一级nginx代理使用$remote_addr参数读取到的是CDN服务的地址，而根据一般约定，CDN转发会将IP地址存放在X-Forwarded-For参数下

修改顶级Nginx配置以支持优先获取CDN代理地址，也可以通过修改业务层读取优先级实现

location / {
 ...
 # IP地址转发
 # proxy_set_header X-Real-IP $remote_addr;
 # proxy_set_header X-Real-Port $remote_port;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 }

但经过对阿里云CDN测试，发现X-Forwarded-For下不仅包含真实客户端IP也包含CDN服务IP，故业务层需要做一定的处理进行区分

如图，第一个为真实客户端IP，第二个为CDN代理IP

public static String getIpAddr(HttpServletRequest request) {
 String ip = request.getHeader("X-Real-IP");
 if (ip == null || ip.length() == 0 || " unknown ".equalsIgnoreCase(ip)) {
 ip = request.getHeader("Proxy-Client-IP");
 log.info("【Proxy-Client-IP】 {}", ip);
 }
 if (ip == null || ip.length() == 0 || " unknown ".equalsIgnoreCase(ip)) {
 ip = request.getHeader("WL-Proxy-Client-IP");
 log.info("【WL-Proxy-Client-IP】{}", ip);
 }
 if (ip == null || ip.length() == 0 || " unknown ".equalsIgnoreCase(ip)) {
 ip = request.getHeader("X-Forwarded-For");
 if (ip.contains(",")) {
 // 通过阿里云CDN转发后可能读取到2个IP地址
 String[] cdnMutilIp = ip.split(",");
 ip = cdnMutilIp[0];
 }
 log.info("【X-Forwarded-For】{}", ip);
 }
 if (ip == null || ip.length() == 0 || " unknown ".equalsIgnoreCase(ip)) {
 ip = request.getRemoteAddr();
 log.info("【unknown】{}", ip);
 }
 return ip;
 }
}

Tyk网关Docker安装

Sun, 31 Mar 2019 15:57:00 +0800

文档指南

官方文档

查看官方Docker安装指南，发现只提供了镜像地址，并未给出详细操作步骤

With Docker
Tyk has three containers that are available to set up a Docker installation:

The Tyk Gateway container

The Tyk Dashboard container

The Tyk Pump container

All three are required for a full deployment. We recommend that each container is installed on a separate machine for optimum performance.

实践

Tyk Gateway container

参考镜像指南，此处的目标是搭建带Dashboard的Tyk网关

Gateway - Pro installation with Dashboard
The gateway in a Pro installation is dependent on the dashboard service. We will assume that the dashboard service is installed, up and running. If not, we would recommend that you follow the dashboard installation guide here:

https://github.com/TykTechnologies/tyk-dashboard-docker

The gateway relies upon the dashboard service to load it’s api definitions & proxy configurations. As such, there is no need to mount any app directory.
docker run -d \
 --name tyk_gateway \
 --network tyk \
 -p 8080:8080 \
 -v $(pwd)/tyk.with_dashboard.conf:/opt/tyk-gateway/tyk.conf \
 tykio/tyk-gateway:latest

文档中说明搭建网关需要先搭建Dashboard

Tyk Dashboard container

安装指南，指南的第六步指向的连接已经失效，由此推断之前存在一键安装脚本

Quickstart

Ensure you have set up the Redis, MongoDB and Tyk Gateway containers

Set up the docker instance IP as the Dashboard hostname (in your /etc/hosts file or as a DNS):

127.0.0.1 dashboard.tyk.docker

Run the Dashboard

docker run -d --name tyk_dashboard -p 3000:3000 --link tyk_redis:redis --link tyk_mongo:mongo --link tyk_gateway:tyk_gateway tykio/tyk-dashboard

You should now be able to access your Dashboard at http://dashboard.tyk.docker:3000/ (note for OSX users, replace 127.0.0.1 with whatever IP address your docker VM runs)

Enter your Dashboard License. Go to http://dashboard.tyk.docker:3000/. You will see a screen asking for a license, enter it in the section marked “Already have a license?” and click Use this license.

Grab the bootstrap script from our tyk-dashboard github repo and run:

./bootstrap.sh dashboard.tyk.docker

To use an external configuration files, use the -v option to mount it over /opt/tyk-dashboard/tyk_analytics.conf

经过查找发现了旧版指南链接仍然可以访问，但不建议使用，旧版指南

按照文档的说法需要申请许可证，官方提供开发许可证，提供邮箱即可获取12个月的授权，申请地址

申请后根据文档提供的Docker启动语句准备环境和Docker-compose文件便于一键启动

docker-compose.yml文件

version: "3.1"

services:

 tyk_gateway:
 image: tykio/tyk-gateway:latest
 restart: always
 networks:
 - tyk-service-bridge
 volumes:
 - /www/wwwroot/tyk-free/apps:/opt/tyk-gateway/apps
 depends_on:
 - tyk_dashboard

 tyk_dashboard:
 image: tykio/tyk-dashboard
 restart: always
 networks:
 - tyk-service-bridge
 ports:
 - 3000:3000
 depends_on:
 - mongo
 - redis

 mongo:
 image: mongo
 restart: always
 networks:
 - tyk-service-bridge

 redis:
 image: redis:4.0-alpine
 restart: always
 networks:
 - tyk-service-bridge

networks:
 tyk-service-bridge:
 driver: bridge

启动

$ docker-compose up

放行防火墙

$ firewall-cmd --zone=public --add-port=3000/tcp --permanent 
$ firewall-cmd --reload

访问浏览器IP:3000输入秘钥

许可证已经发送到注册邮件中，激活后可登陆

但此时没有默认的用户名和密码，并且暂时未找到配置的方式（差评），需要通过官方脚本注册用户名和密码

但官方脚本依赖Python环境，若主机没有python环境则无法执行

启动脚本：

#!/bin/bash
# Usage ./bootstrap.sh DASHBOARD_HOSTNAME

LOCALIP=$1
RANDOM_USER=$(env LC_CTYPE=C tr -dc "a-z0-9" < /dev/urandom | head -c 10)
PASS="test123"

echo "Creating Organisation"
ORGDATA=$(curl --silent --header "admin-auth: 12345" --header "Content-Type:application/json" --data '{"owner_name": "Default Org.","owner_slug": "default", "cname_enabled": true, "cname": ""}' http://$LOCALIP:3000/admin/organisations 2>&1)
#echo $ORGDATA
ORGID=$(echo $ORGDATA | python -c 'import json,sys;obj=json.load(sys.stdin);print(obj["Meta"])')
echo "ORGID: $ORGID"

echo "Adding new user"
USER_DATA=$(curl --silent --header "admin-auth: 12345" --header "Content-Type:application/json" --data '{"first_name": "John","last_name": "Smith","email_address": "'$RANDOM_USER'@example.com","password":"'$PASS'", "active": true,"org_id": "'$ORGID'"}' http://$LOCALIP:3000/admin/users 2>&1)
#echo $USER_DATA
USER_CODE=$(echo $USER_DATA | python -c 'import json,sys;obj=json.load(sys.stdin);print(obj["Message"])')
echo "USER AUTH: $USER_CODE"

USER_LIST=$(curl --silent --header "authorization: $USER_CODE" http://$LOCALIP:3000/api/users 2>&1)
#echo $USER_LIST

USER_ID=$(echo $USER_LIST | python -c 'import json,sys;obj=json.load(sys.stdin);print(obj["users"][0]["id"])')
echo "NEW ID: $USER_ID"

echo "Setting password"
OK=$(curl --silent --header "authorization: $USER_CODE" --header "Content-Type:application/json" http://$LOCALIP:3000/api/users/$USER_ID/actions/reset --data '{"new_password":"'$PASS'"}')

echo ""

echo "DONE"
echo "===="
echo "Login at http://$LOCALIP:3000/"
echo "User: $RANDOM_USER@example.com"
echo "Pass: $PASS"
echo ""

为了方便使用，此处创建了包含启动脚本的Docker镜像

仓库地址：fjy8018/tyk-bootstrap

使用方法：

$ docker run --rm -it --network tyk-free_tyk-service-bridge -e TARGET_IP=tyk_dashboard fjy8018/tyk-bootstrap:0.1

随即获得登陆名和密码

登陆成功

SpringBoot2.x Data JPA 多数据源爬坑

Sat, 30 Mar 2019 10:44:00 +0800

JPA 官方文档

官方文档

5.1.2. Annotation-based Configuration

The Spring Data JPA repositories support can be activated not only through an XML namespace but also by using an annotation through JavaConfig, as shown in the following example:

Example 51. Spring Data JPA repositories using JavaConfig

@Configuration
@EnableJpaRepositories
@EnableTransactionManagement
class ApplicationConfig {

 @Bean
 public DataSource dataSource() {

 EmbeddedDatabaseBuilder builder = new EmbeddedDatabaseBuilder();
 return builder.setType(EmbeddedDatabaseType.HSQL).build();
 }

 @Bean
 public LocalContainerEntityManagerFactoryBean entityManagerFactory() {

 HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
 vendorAdapter.setGenerateDdl(true);

 LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
 factory.setJpaVendorAdapter(vendorAdapter);
 factory.setPackagesToScan("com.acme.domain");
 factory.setDataSource(dataSource());
 return factory;
 }

 @Bean
 public PlatformTransactionManager transactionManager(EntityManagerFactory entityManagerFactory) {

 JpaTransactionManager txManager = new JpaTransactionManager();
 txManager.setEntityManagerFactory(entityManagerFactory);
 return txManager;
 }
}

文档中说明，基于注解的配置包含三个关键Bean：

PlatformTransactionManager——事务管理
LocalContainerEntityManagerFactoryBean——实体类管理
DataSource——数据源

同时还有一个关键注解@EnableJpaRepositories

SpringBoot官方文档

官方文档

多数据源使用@Primary指明优先级

You can even go further by leveraging what DataSourceProperties does for you — that is, by providing a default embedded database with a sensible username and password if no URL is provided. You can easily initialize a DataSourceBuilder from the state of any DataSourceProperties object, so you could also inject the DataSource that Spring Boot creates automatically. However, that would split your configuration into two namespaces: url, username, password, type, and driver on spring.datasource and the rest on your custom namespace (app.datasource). To avoid that, you can redefine a custom DataSourceProperties on your custom namespace, as shown in the following example:
@Bean
@Primary
@ConfigurationProperties("app.datasource")
public DataSourceProperties dataSourceProperties() {
	return new DataSourceProperties();
}

@Bean
@ConfigurationProperties("app.datasource.configuration")
public HikariDataSource dataSource(DataSourceProperties properties) {
	return properties.initializeDataSourceBuilder().type(HikariDataSource.class)
			.build();
}
84.2 Configure Two DataSources
If you need to configure multiple data sources, you can apply the same tricks that are described in the previous section. You must, however, mark one of the DataSource instances as @Primary, because various auto-configurations down the road expect to be able to get one by type.

If you create your own DataSource, the auto-configuration backs off. In the following example, we provide the exact same feature set as the auto-configuration provides on the primary data source:
@Bean
@Primary
@ConfigurationProperties("app.datasource.first")
public DataSourceProperties firstDataSourceProperties() {
	return new DataSourceProperties();
}

@Bean
@Primary
@ConfigurationProperties("app.datasource.first.configuration")
public HikariDataSource firstDataSource() {
	return firstDataSourceProperties().initializeDataSourceBuilder()
			.type(HikariDataSource.class).build();
}

@Bean
@ConfigurationProperties("app.datasource.second")
public BasicDataSource secondDataSource() {
	return DataSourceBuilder.create().type(BasicDataSource.class).build();
}

或者使用两个EntityManagers，并结合@EnableJpaRepositories进行配置

84.10 Use Two EntityManagers
Even if the default EntityManagerFactory works fine, you need to define a new one. Otherwise, the presence of the second bean of that type switches off the default. To make it easy to do, you can use the convenient EntityManagerBuilder provided by Spring Boot. Alternatively, you can just theLocalContainerEntityManagerFactoryBean directly from Spring ORM, as shown in the following example:
// add two data sources configured as above

@Bean
public LocalContainerEntityManagerFactoryBean customerEntityManagerFactory(
		EntityManagerFactoryBuilder builder) {
	return builder
			.dataSource(customerDataSource())
			.packages(Customer.class)
			.persistenceUnit("customers")
			.build();
}

@Bean
public LocalContainerEntityManagerFactoryBean orderEntityManagerFactory(
		EntityManagerFactoryBuilder builder) {
	return builder
			.dataSource(orderDataSource())
			.packages(Order.class)
			.persistenceUnit("orders")
			.build();
}
The configuration above almost works on its own. To complete the picture, you need to configure TransactionManagers for the two EntityManagers as well. If you mark one of them as @Primary, it could be picked up by the default JpaTransactionManager in Spring Boot. The other would have to be explicitly injected into a new instance. Alternatively, you might be able to use a JTA transaction manager that spans both.

If you use Spring Data, you need to configure @EnableJpaRepositories accordingly, as shown in the following example:
@Configuration
@EnableJpaRepositories(basePackageClasses = Customer.class,
		entityManagerFactoryRef = "customerEntityManagerFactory")
public class CustomerConfiguration {
	...
}

@Configuration
@EnableJpaRepositories(basePackageClasses = Order.class,
		entityManagerFactoryRef = "orderEntityManagerFactory")
public class OrderConfiguration {
	...
}

源码分析

自动装配类中声明了配置Bean和条件装配顺序

package org.springframework.boot.autoconfigure.data.jpa;

/**
 * {@link EnableAutoConfiguration Auto-configuration} for Spring Data's JPA Repositories.
 * <p>
 * Activates when there is a bean of type {@link javax.sql.DataSource} configured in the
 * context, the Spring Data JPA
 * {@link org.springframework.data.jpa.repository.JpaRepository} type is on the classpath,
 * and there is no other, existing
 * {@link org.springframework.data.jpa.repository.JpaRepository} configured.
 * <p>
 * Once in effect, the auto-configuration is the equivalent of enabling JPA repositories
 * using the {@link org.springframework.data.jpa.repository.config.EnableJpaRepositories}
 * annotation.
 * <p>
 * This configuration class will activate <em>after</em> the Hibernate auto-configuration.
 *
 * @author Phillip Webb
 * @author Josh Long
 * @see EnableJpaRepositories
 */
@Configuration
@ConditionalOnBean(DataSource.class)
@ConditionalOnClass(JpaRepository.class)
@ConditionalOnMissingBean({ JpaRepositoryFactoryBean.class,
		JpaRepositoryConfigExtension.class })
@ConditionalOnProperty(prefix = "spring.data.jpa.repositories", name = "enabled", havingValue = "true", matchIfMissing = true)
@Import(JpaRepositoriesAutoConfigureRegistrar.class)
@AutoConfigureAfter({ HibernateJpaAutoConfiguration.class,
		TaskExecutionAutoConfiguration.class })
public class JpaRepositoriesAutoConfiguration {

	@Bean
	@Conditional(BootstrapExecutorCondition.class)
	public EntityManagerFactoryBuilderCustomizer entityManagerFactoryBootstrapExecutorCustomizer(
			Map<String, AsyncTaskExecutor> taskExecutors) {
		return (builder) -> {
			AsyncTaskExecutor bootstrapExecutor = determineBootstrapExecutor(
					taskExecutors);
			if (bootstrapExecutor != null) {
				builder.setBootstrapExecutor(bootstrapExecutor);
			}
		};
	}

	private AsyncTaskExecutor determineBootstrapExecutor(
			Map<String, AsyncTaskExecutor> taskExecutors) {
		if (taskExecutors.size() == 1) {
			return taskExecutors.values().iterator().next();
		}
		return taskExecutors
				.get(TaskExecutionAutoConfiguration.APPLICATION_TASK_EXECUTOR_BEAN_NAME);
	}

	private static final class BootstrapExecutorCondition extends AnyNestedCondition {

		BootstrapExecutorCondition() {
			super(ConfigurationPhase.REGISTER_BEAN);
		}

		@ConditionalOnProperty(prefix = "spring.data.jpa.repositories", name = "bootstrap-mode", havingValue = "deferred", matchIfMissing = false)
		static class DeferredBootstrapMode {

		}

		@ConditionalOnProperty(prefix = "spring.data.jpa.repositories", name = "bootstrap-mode", havingValue = "lazy", matchIfMissing = false)
		static class LazyBootstrapMode {

		}

	}
}

引入的配置类中声明了EnableJpaRepositories注解

package org.springframework.boot.autoconfigure.data.jpa;

class JpaRepositoriesAutoConfigureRegistrar
		extends AbstractRepositoryConfigurationSourceSupport {

	private BootstrapMode bootstrapMode = null;

	@Override
	protected Class<? extends Annotation> getAnnotation() {
		return EnableJpaRepositories.class;
	}

	@Override
	protected Class<?> getConfiguration() {
		return EnableJpaRepositoriesConfiguration.class;
	}

	@Override
	protected RepositoryConfigurationExtension getRepositoryConfigurationExtension() {
		return new JpaRepositoryConfigExtension();
	}

	@Override
	protected BootstrapMode getBootstrapMode() {
		return (this.bootstrapMode == null) ? super.getBootstrapMode()
				: this.bootstrapMode;
	}

	@Override
	public void setEnvironment(Environment environment) {
		super.setEnvironment(environment);
		configureBootstrapMode(environment);
	}

	private void configureBootstrapMode(Environment environment) {
		String property = environment
				.getProperty("spring.data.jpa.repositories.bootstrap-mode");
		if (StringUtils.hasText(property)) {
			this.bootstrapMode = BootstrapMode
					.valueOf(property.toUpperCase(Locale.ENGLISH));
		}
	}
 
	@EnableJpaRepositories
	private static class EnableJpaRepositoriesConfiguration {}
}

EnableJpaRepositories注解定义

package org.springframework.data.jpa.repository.config;

/**
 * Annotation to enable JPA repositories. Will scan the package of the annotated configuration class for Spring Data
 * repositories by default.
 *
 * @author Oliver Gierke
 * @author Thomas Darimont
 */
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
@Import(JpaRepositoriesRegistrar.class)
public @interface EnableJpaRepositories {

	String[] value() default {};

	String[] basePackages() default {};

	Class<?>[] basePackageClasses() default {};

	Filter[] includeFilters() default {};

	Filter[] excludeFilters() default {};

	String repositoryImplementationPostfix() default "Impl";

	String namedQueriesLocation() default "";

	Key queryLookupStrategy() default Key.CREATE_IF_NOT_FOUND;

	Class<?> repositoryFactoryBeanClass() default JpaRepositoryFactoryBean.class;

	Class<?> repositoryBaseClass() default DefaultRepositoryBaseClass.class;

	/**
	 * 实体管理工厂
	 * @return
	 */
	String entityManagerFactoryRef() default "entityManagerFactory";

	/**
	 * 事务管理
	 * @return
	 */
	String transactionManagerRef() default "transactionManager";

	boolean considerNestedRepositories() default false;

	boolean enableDefaultTransactions() default true;

	/**
	 * Configures when the repositories are initialized in the bootstrap lifecycle. {@link BootstrapMode#DEFAULT}
	 * (default) means eager initialization except all repository interfaces annotated with {@link Lazy},
	 * {@link BootstrapMode#LAZY} means lazy by default including injection of lazy-initialization proxies into client
	 * beans so that those can be instantiated but will only trigger the initialization upon first repository usage (i.e a
	 * method invocation on it). This means repositories can still be uninitialized when the application context has
	 * completed its bootstrap. {@link BootstrapMode#DEFERRED} is fundamentally the same as {@link BootstrapMode#LAZY},
	 * but triggers repository initialization when the application context finishes its bootstrap.
	 * 
	 * @return
	 * @since 2.1
	 */
	BootstrapMode bootstrapMode() default BootstrapMode.DEFAULT;
}

查找关键Bean的注册时机

package org.springframework.boot.autoconfigure.orm.jpa;

@Configuration
@ConditionalOnClass({ LocalContainerEntityManagerFactoryBean.class, EntityManager.class })
@Conditional(HibernateEntityManagerCondition.class)
@EnableConfigurationProperties(JpaProperties.class)
@AutoConfigureAfter({ DataSourceAutoConfiguration.class })
@Import(HibernateJpaConfiguration.class)
public class HibernateJpaAutoConfiguration {}

进入配置Bean

package org.springframework.boot.autoconfigure.orm.jpa;

@Configuration
@EnableConfigurationProperties(HibernateProperties.class)
@ConditionalOnSingleCandidate(DataSource.class)
class HibernateJpaConfiguration extends JpaBaseConfiguration {}

在其父类JpaBaseConfiguration中声明了装配时机和条件

package org.springframework.boot.autoconfigure.orm.jpa;

/**
 * 该类在2.0版本进行了重构
 * @since 2.0.0
 */
@Configuration
@EnableConfigurationProperties(JpaProperties.class)
@Import(DataSourceInitializedPublisher.Registrar.class)
public abstract class JpaBaseConfiguration implements BeanFactoryAware {
 
 @Bean
	@ConditionalOnMissingBean
	public PlatformTransactionManager transactionManager() {
		JpaTransactionManager transactionManager = new JpaTransactionManager();
		if (this.transactionManagerCustomizers != null) {
 // 默认使用方法名称
			this.transactionManagerCustomizers.customize(transactionManager);
		}
		return transactionManager;
	}

	@Bean
	@ConditionalOnMissingBean
	public JpaVendorAdapter jpaVendorAdapter() {
		AbstractJpaVendorAdapter adapter = createJpaVendorAdapter();
		adapter.setShowSql(this.properties.isShowSql());
		adapter.setDatabase(this.properties.determineDatabase(this.dataSource));
		adapter.setDatabasePlatform(this.properties.getDatabasePlatform());
		adapter.setGenerateDdl(this.properties.isGenerateDdl());
		return adapter;
	}

	@Bean
	@ConditionalOnMissingBean
	public EntityManagerFactoryBuilder entityManagerFactoryBuilder(
			JpaVendorAdapter jpaVendorAdapter,
			ObjectProvider<PersistenceUnitManager> persistenceUnitManager,
			ObjectProvider<EntityManagerFactoryBuilderCustomizer> customizers) {
		EntityManagerFactoryBuilder builder = new EntityManagerFactoryBuilder(
				jpaVendorAdapter, this.properties.getProperties(),
				persistenceUnitManager.getIfAvailable());
		customizers.orderedStream()
				.forEach((customizer) -> customizer.customize(builder));
		return builder;
	}

	@Bean
	@Primary
	@ConditionalOnMissingBean({ LocalContainerEntityManagerFactoryBean.class,
			EntityManagerFactory.class })
	public LocalContainerEntityManagerFactoryBean entityManagerFactory(
			EntityManagerFactoryBuilder factoryBuilder) {
		Map<String, Object> vendorProperties = getVendorProperties();
		customizeVendorProperties(vendorProperties);
		return factoryBuilder.dataSource(this.dataSource).packages(getPackagesToScan())
				.properties(vendorProperties).mappingResources(getMappingResources())
				.jta(isJta()).build();
	}
}

测试

爬完文档和源码，对需要配置Bean有了基本了解，新建工程进行多数据源测试

以当前最新版本SpringBoot进行测试

maven

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
 <modelVersion>4.0.0</modelVersion>
 <parent>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-parent</artifactId>
 <version>2.1.3.RELEASE</version>
 <relativePath/> <!-- lookup parent from repository -->
 </parent>
 <groupId>top.fjy8018</groupId>
 <artifactId>jpadatasource</artifactId>
 <version>0.0.1-SNAPSHOT</version>
 <name>jpadatasource</name>
 <description>Demo project for Spring Boot</description>

 <properties>
 <java.version>1.8</java.version>
 </properties>

 <dependencies>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-data-jpa</artifactId>
 </dependency>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-web</artifactId>
 </dependency>

 <dependency>
 <groupId>mysql</groupId>
 <artifactId>mysql-connector-java</artifactId>
 <scope>runtime</scope>
 </dependency>

 <dependency>
 <groupId>org.projectlombok</groupId>
 <artifactId>lombok</artifactId>
 </dependency>

 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-test</artifactId>
 <scope>test</scope>
 </dependency>
 </dependencies>

 <build>
 <plugins>
 <plugin>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-maven-plugin</artifactId>
 </plugin>
 </plugins>
 </build>

</project>

实体类

通过包路径区分主库和备份库

主库实体类

package top.fjy8018.jpadatasource.entity.primary;

import lombok.Data;

import javax.persistence.Entity;
import javax.persistence.Id;

/**
 * @author F嘉阳
 * @date 2019-03-30 9:11
 */
@Data
@Entity
public class Product {

 @Id
 private Integer id;

 private String name;

 private Integer price;
}

备份库实体类

package top.fjy8018.jpadatasource.entity.backup;

import lombok.Data;

import javax.persistence.Entity;
import javax.persistence.Id;

/**
 * @author F嘉阳
 * @date 2019-03-30 9:36
 */
@Data
@Entity(name = "tb_order")
public class Order {

 @Id
 private Integer id;

 private String orderName;

 private Integer price;
}

坑一：表名关键字冲突

若此处不指定表名为tb_order则会与MySQL关键字冲突导致建表失败

报错：

org.hibernate.tool.schema.spi.CommandAcceptanceException: Error executing DDL "
 create table Order (
 id integer not null,
 orderName varchar(255),
 price integer,
 primary key (id)
 ) engine=InnoDB" via JDBC Statement
 ...

DAO

package top.fjy8018.jpadatasource.repository.backup;

import org.springframework.data.jpa.repository.JpaRepository;
import top.fjy8018.jpadatasource.entity.backup.Order;

/**
 * @author F嘉阳
 * @date 2019-03-30 9:12
 */
public interface OrderRepository extends JpaRepository<Order, Integer> {
}

package top.fjy8018.jpadatasource.repository.primary;

import org.springframework.data.jpa.repository.JpaRepository;
import top.fjy8018.jpadatasource.entity.primary.Product;

/**
 * @author F嘉阳
 * @date 2019-03-30 9:12
 */
public interface ProductRepository extends JpaRepository<Product, Integer> {
}

配置Bean

官方配置样例：

Both data sources are also bound for advanced customizations. For instance, you could configure them as follows:

app.datasource.first.url=jdbc:mysql://localhost/first
app.datasource.first.username=dbuser
app.datasource.first.password=dbpass
app.datasource.first.configuration.maximum-pool-size=30

app.datasource.second.url=jdbc:mysql://localhost/second
app.datasource.second.username=dbuser
app.datasource.second.password=dbpass
app.datasource.second.max-total=30

You can apply the same concept to the secondary DataSource as well, as shown in the following example:

@Bean
@Primary
@ConfigurationProperties("app.datasource.first")
public DataSourceProperties firstDataSourceProperties() {
	return new DataSourceProperties();
}

@Bean
@Primary
@ConfigurationProperties("app.datasource.first.configuration")
public HikariDataSource firstDataSource() {
	return firstDataSourceProperties().initializeDataSourceBuilder()
			.type(HikariDataSource.class).build();
}

@Bean
@ConfigurationProperties("app.datasource.second")
public DataSourceProperties secondDataSourceProperties() {
	return new DataSourceProperties();
}

@Bean
@ConfigurationProperties("app.datasource.second.configuration")
public BasicDataSource secondDataSource() {
	return secondDataSourceProperties().initializeDataSourceBuilder()
			.type(BasicDataSource.class).build();
}

The preceding example configures two data sources on custom namespaces with the same logic as Spring Boot would use in auto-configuration. Note that each configuration sub namespace provides advanced settings based on the chosen implementation.

参照编写配置文件，此处注意SpringBoot 2.x的驱动变化

坑二：mysql驱动类名变化

spring:
 datasource:
 first:
 driver-class-name: com.mysql.cj.jdbc.Driver
 username: financial_adminer
 password: financial_adminer_pass
 url: jdbc:mysql://localhost/jpa_test?useSSL=false&useUnicode=true&characterEncoding=UTF-8&serverTimezone=UTC
 configuration:
 maximum-pool-size: 30
 second:
 driver-class-name: com.mysql.cj.jdbc.Driver
 username: financial_adminer
 password: financial_adminer_pass
 url: jdbc:mysql://localhost/jpa_test_bak?useSSL=false&useUnicode=true&characterEncoding=UTF-8&serverTimezone=UTC
 configuration:
 maximum-pool-size: 20

 jpa:
 hibernate:
 ddl-auto: update
 show-sql: true
 database-platform: org.hibernate.dialect.MySQL55Dialect
 properties:
 hibernate:
 format_sql: true
 use_sql_comments: true

配置类

package top.fjy8018.jpadatasource.config;

import com.zaxxer.hikari.HikariDataSource;
import org.springframework.boot.autoconfigure.jdbc.DataSourceProperties;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;

/**
 * @author F嘉阳
 * @date 2019-03-30 9:24
 */
@Configuration
public class DataAccessConfig {
 @Bean
 @Primary
 @ConfigurationProperties("spring.datasource.first")
 public DataSourceProperties firstDataSourceProperties() {
 return new DataSourceProperties();
 }

 @Bean
 @Primary
 @ConfigurationProperties("spring.datasource.first.configuration")
 public HikariDataSource firstDataSource() {
 return firstDataSourceProperties().initializeDataSourceBuilder()
 .type(HikariDataSource.class).build();
 }

 @Bean
 @ConfigurationProperties("spring.datasource.second")
 public DataSourceProperties secondDataSourceProperties() {
 return new DataSourceProperties();
 }

 @Bean
 @ConfigurationProperties("spring.datasource.second.configuration")
 public HikariDataSource secondDataSource() {
 return secondDataSourceProperties().initializeDataSourceBuilder()
 .type(HikariDataSource.class).build();
 }
}

配置完成后发现只在主库自动建表，备份库无变化，属于预期情况，因为目前仅配置了三个必须Bean的其中一个

继续配置实体管理和事务管理Bean

官方文档样例

84.10 Use Two EntityManagers
Even if the default EntityManagerFactory works fine, you need to define a new one. Otherwise, the presence of the second bean of that type switches off the default. To make it easy to do, you can use the convenient EntityManagerBuilder provided by Spring Boot. Alternatively, you can just theLocalContainerEntityManagerFactoryBean directly from Spring ORM, as shown in the following example:
// add two data sources configured as above

@Bean
public LocalContainerEntityManagerFactoryBean customerEntityManagerFactory(
		EntityManagerFactoryBuilder builder) {
	return builder
			.dataSource(customerDataSource())
			.packages(Customer.class)
			.persistenceUnit("customers")
			.build();
}

@Bean
public LocalContainerEntityManagerFactoryBean orderEntityManagerFactory(
		EntityManagerFactoryBuilder builder) {
	return builder
			.dataSource(orderDataSource())
			.packages(Order.class)
			.persistenceUnit("orders")
			.build();
}
The configuration above almost works on its own. To complete the picture, you need to configure TransactionManagers for the two EntityManagers as well. If you mark one of them as @Primary, it could be picked up by the default JpaTransactionManager in Spring Boot. The other would have to be explicitly injected into a new instance. Alternatively, you might be able to use a JTA transaction manager that spans both.

If you use Spring Data, you need to configure @EnableJpaRepositories accordingly, as shown in the following example:
@Configuration
@EnableJpaRepositories(basePackageClasses = Customer.class,
		entityManagerFactoryRef = "customerEntityManagerFactory")
public class CustomerConfiguration {
	...
}

@Configuration
@EnableJpaRepositories(basePackageClasses = Order.class,
		entityManagerFactoryRef = "orderEntityManagerFactory")
public class OrderConfiguration {
	...
}

工程配置

package top.fjy8018.jpadatasource.config;

import com.zaxxer.hikari.HikariDataSource;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.jdbc.DataSourceProperties;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.orm.jpa.JpaTransactionManager;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;
import org.springframework.transaction.PlatformTransactionManager;
import top.fjy8018.jpadatasource.entity.backup.Order;
import top.fjy8018.jpadatasource.entity.primary.Product;
import top.fjy8018.jpadatasource.repository.backup.OrderRepository;
import top.fjy8018.jpadatasource.repository.primary.ProductRepository;

import javax.sql.DataSource;

/**
 * @author F嘉阳
 * @date 2019-03-30 9:24
 */
@Configuration
public class DataAccessConfig {
 @Bean
 @Primary
 @ConfigurationProperties("spring.datasource.first")
 public DataSourceProperties firstDataSourceProperties() {
 return new DataSourceProperties();
 }

 @Bean
 @Primary
 @ConfigurationProperties("spring.datasource.first.configuration")
 public HikariDataSource firstDataSource() {
 return firstDataSourceProperties().initializeDataSourceBuilder()
 .type(HikariDataSource.class).build();
 }

 @Bean
 @ConfigurationProperties("spring.datasource.second")
 public DataSourceProperties secondDataSourceProperties() {
 return new DataSourceProperties();
 }

 @Bean
 @ConfigurationProperties("spring.datasource.second.configuration")
 public HikariDataSource secondDataSource() {
 return secondDataSourceProperties().initializeDataSourceBuilder()
 .type(HikariDataSource.class).build();
 }

 @Bean
 public LocalContainerEntityManagerFactoryBean firstEntityManagerFactory(
 EntityManagerFactoryBuilder builder, @Qualifier("firstDataSource") DataSource dataSource) {
 return builder
 .dataSource(dataSource)
 .packages(Product.class)
 .persistenceUnit("first")
 .build();
 }

 @Bean
 public LocalContainerEntityManagerFactoryBean secondEntityManagerFactory(
 EntityManagerFactoryBuilder builder, @Qualifier("secondDataSource") DataSource dataSource) {
 return builder
 .dataSource(dataSource)
 .packages(Order.class)
 .persistenceUnit("second")
 .build();
 }

 @Bean
 public PlatformTransactionManager primaryTransactionManager(
 @Qualifier("firstEntityManagerFactory") LocalContainerEntityManagerFactoryBean entityManagerFactory) {
 return new JpaTransactionManager(entityManagerFactory.getObject());
 }

 @Bean
 public PlatformTransactionManager backupTransactionManager(
 @Qualifier("secondEntityManagerFactory") LocalContainerEntityManagerFactoryBean entityManagerFactory) {
 return new JpaTransactionManager(entityManagerFactory.getObject());
 }

 @EnableJpaRepositories(basePackageClasses = ProductRepository.class,
 entityManagerFactoryRef = "firstEntityManagerFactory")
 @Primary
 public class PrimaryConfiguration {
 }

 @EnableJpaRepositories(basePackageClasses = OrderRepository.class,
 entityManagerFactoryRef = "secondEntityManagerFactory")
 public class secondConfiguration {
 }
}

坑三：官方文档样例有误

此时配置直接启动会报错

Method requestMappingHandlerMapping in org.springframework.boot.autoconfigure.web.servlet.WebMvcAutoConfiguration$EnableWebMvcConfiguration required a single bean, but 2 were found:
	- firstEntityManagerFactory: defined by method 'firstEntityManagerFactory' in class path resource [top/fjy8018/jpadatasource/config/DataAccessConfig.class]
	- secondEntityManagerFactory: defined by method 'secondEntityManagerFactory' in class path resource [top/fjy8018/jpadatasource/config/DataAccessConfig.class]


Action:

Consider marking one of the beans as @Primary, updating the consumer to accept multiple beans, or using @Qualifier to identify the bean that should be consumed

意思是自动装配时发现两个类型一致的Bean，主要指定优先级，而官方文档对此处可能出现的问题并未提示

指定firstEntityManagerFactory为优先Bean，同时文档样例创建了事务管理器但未配置到@EnableJpaRepositories注解中，此处也一并配置

最终配置类如下

package top.fjy8018.jpadatasource.config;

import com.zaxxer.hikari.HikariDataSource;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.jdbc.DataSourceProperties;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.orm.jpa.JpaTransactionManager;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;
import org.springframework.transaction.PlatformTransactionManager;
import top.fjy8018.jpadatasource.entity.backup.Order;
import top.fjy8018.jpadatasource.entity.primary.Product;
import top.fjy8018.jpadatasource.repository.backup.OrderRepository;
import top.fjy8018.jpadatasource.repository.primary.ProductRepository;

import javax.sql.DataSource;

/**
 * @author F嘉阳
 * @date 2019-03-30 9:24
 */
@Configuration
public class DataAccessConfig {
 @Bean
 @Primary
 @ConfigurationProperties("spring.datasource.first")
 public DataSourceProperties firstDataSourceProperties() {
 return new DataSourceProperties();
 }

 @Bean
 @Primary
 @ConfigurationProperties("spring.datasource.first.configuration")
 public HikariDataSource firstDataSource() {
 return firstDataSourceProperties().initializeDataSourceBuilder()
 .type(HikariDataSource.class).build();
 }

 @Bean
 @ConfigurationProperties("spring.datasource.second")
 public DataSourceProperties secondDataSourceProperties() {
 return new DataSourceProperties();
 }

 @Bean
 @ConfigurationProperties("spring.datasource.second.configuration")
 public HikariDataSource secondDataSource() {
 return secondDataSourceProperties().initializeDataSourceBuilder()
 .type(HikariDataSource.class).build();
 }

 @Bean
 public LocalContainerEntityManagerFactoryBean firstEntityManagerFactory(
 EntityManagerFactoryBuilder builder, @Qualifier("firstDataSource") DataSource dataSource) {
 return builder
 .dataSource(dataSource)
 .packages(Product.class)
 .persistenceUnit("first")
 .build();
 }

 @Bean
 public LocalContainerEntityManagerFactoryBean secondEntityManagerFactory(
 EntityManagerFactoryBuilder builder, @Qualifier("secondDataSource") DataSource dataSource) {
 return builder
 .dataSource(dataSource)
 .packages(Order.class)
 .persistenceUnit("second")
 .build();
 }

 @Bean
 public PlatformTransactionManager primaryTransactionManager(
 @Qualifier("firstEntityManagerFactory") LocalContainerEntityManagerFactoryBean entityManagerFactory) {
 return new JpaTransactionManager(entityManagerFactory.getObject());
 }

 @Bean
 public PlatformTransactionManager backupTransactionManager(
 @Qualifier("secondEntityManagerFactory") LocalContainerEntityManagerFactoryBean entityManagerFactory) {
 return new JpaTransactionManager(entityManagerFactory.getObject());
 }

 @EnableJpaRepositories(basePackageClasses = ProductRepository.class,
 entityManagerFactoryRef = "firstEntityManagerFactory", transactionManagerRef = "primaryTransactionManager")
 @Primary
 public class PrimaryConfiguration {
 }

 @EnableJpaRepositories(basePackageClasses = OrderRepository.class,
 entityManagerFactoryRef = "secondEntityManagerFactory", transactionManagerRef = "backupTransactionManager")
 public class secondConfiguration {
 }
}

坑四：JPA不会自动建表

配置完成后启动

发现日志中建表SQL没有输出，同时查看数据库，确认表未创建

但官方文档对多数据源的配置说明已经结束，令人疑惑

自动建表语句没生成猜想可能由于与ddl相关的属性未配置导致

进入配置文件查看ddl配置，发现默认为false

将其配置为true后总配置文件如下

spring:
 datasource:
 first:
 driver-class-name: com.mysql.cj.jdbc.Driver
 username: financial_adminer
 password: financial_adminer_pass
 url: jdbc:mysql://120.79.226.26/jpa_test?useSSL=false&useUnicode=true&characterEncoding=UTF-8&serverTimezone=UTC
 configuration:
 maximum-pool-size: 30
 second:
 driver-class-name: com.mysql.cj.jdbc.Driver
 username: financial_adminer
 password: financial_adminer_pass
 url: jdbc:mysql://120.79.226.26/jpa_test_bak?useSSL=false&useUnicode=true&characterEncoding=UTF-8&serverTimezone=UTC
 configuration:
 maximum-pool-size: 20

 jpa:
 hibernate:
 ddl-auto: update
 show-sql: true
 database-platform: org.hibernate.dialect.MySQL55Dialect
 properties:
 hibernate:
 format_sql: true
 use_sql_comments: true
 generate-ddl: true

启动工程发现自动建表生效

查看数据库情况，与预期一致，主库只包含主库的表，备份库只包含备份库的表，多数据配置完成

坑五：hibernate默认配置被覆盖

若在单数据源配置下应当无需配置该项，同时发现本应该默认将驼峰属性名映射为_拼接名称策略也失效，说明自定义配置源时覆盖了默认配置或者导致默认配置失效

查询源码分析原因，发现hibernate相关配置均在HibernateProperties配置类中

package org.springframework.boot.autoconfigure.orm.jpa;

/**
 * Configuration properties for Hibernate.
 *
 * @author Stephane Nicoll
 * @since 2.1.0
 * @see JpaProperties
 */
@ConfigurationProperties("spring.jpa.hibernate")
public class HibernateProperties {}

而该类在HibernateJpaConfiguration中进行注入，但配置类中进行了条件装配注解@ConditionalOnSingleCandidate标注，导致在多数据源情况下不满足该配置，类没有被装配到Spring Bean中

package org.springframework.boot.autoconfigure.orm.jpa;

/**
 * {@link JpaBaseConfiguration} implementation for Hibernate.
 * @since 2.0.0
 */
@Configuration
@EnableConfigurationProperties(HibernateProperties.class)
@ConditionalOnSingleCandidate(DataSource.class)
class HibernateJpaConfiguration extends JpaBaseConfiguration {

	private final HibernateProperties hibernateProperties;

	HibernateJpaConfiguration(DataSource dataSource, JpaProperties jpaProperties,
			ConfigurableListableBeanFactory beanFactory,
			ObjectProvider<JtaTransactionManager> jtaTransactionManager,
			ObjectProvider<TransactionManagerCustomizers> transactionManagerCustomizers,
			HibernateProperties hibernateProperties,
			ObjectProvider<Collection<DataSourcePoolMetadataProvider>> metadataProviders,
			ObjectProvider<SchemaManagementProvider> providers,
			ObjectProvider<PhysicalNamingStrategy> physicalNamingStrategy,
			ObjectProvider<ImplicitNamingStrategy> implicitNamingStrategy,
			ObjectProvider<HibernatePropertiesCustomizer> hibernatePropertiesCustomizers) {
		super(dataSource, jpaProperties, jtaTransactionManager,
				transactionManagerCustomizers);
		this.hibernateProperties = hibernateProperties;
		this.defaultDdlAutoProvider = new HibernateDefaultDdlAutoProvider(providers);
		this.poolMetadataProvider = new CompositeDataSourcePoolMetadataProvider(
				metadataProviders.getIfAvailable());
		this.hibernatePropertiesCustomizers = determineHibernatePropertiesCustomizers(
				physicalNamingStrategy.getIfAvailable(),
				implicitNamingStrategy.getIfAvailable(), beanFactory,
				hibernatePropertiesCustomizers.orderedStream()
						.collect(Collectors.toList()));
	}
 ...
}

解决方法就只能自己工程内实现该类的配置，注意，Spring 2.x版本自动装配变化较大，当前配置基于最新版2.1.3有效，并不代表对后续版本一定兼容

工程配置

package top.fjy8018.jpadatasource.config;

import com.zaxxer.hikari.HikariDataSource;
import org.hibernate.boot.model.naming.ImplicitNamingStrategy;
import org.hibernate.boot.model.naming.PhysicalNamingStrategy;
import org.hibernate.cfg.AvailableSettings;
import org.springframework.beans.factory.ObjectProvider;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.config.ConfigurableListableBeanFactory;
import org.springframework.boot.autoconfigure.jdbc.DataSourceProperties;
import org.springframework.boot.autoconfigure.orm.jpa.HibernateProperties;
import org.springframework.boot.autoconfigure.orm.jpa.HibernatePropertiesCustomizer;
import org.springframework.boot.autoconfigure.orm.jpa.HibernateSettings;
import org.springframework.boot.autoconfigure.orm.jpa.JpaProperties;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.jdbc.EmbeddedDatabaseConnection;
import org.springframework.boot.jdbc.SchemaManagement;
import org.springframework.boot.jdbc.SchemaManagementProvider;
import org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.orm.hibernate5.SpringBeanContainer;
import org.springframework.orm.jpa.JpaTransactionManager;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.util.ClassUtils;
import top.fjy8018.jpadatasource.entity.backup.Order;
import top.fjy8018.jpadatasource.entity.primary.Product;
import top.fjy8018.jpadatasource.repository.backup.OrderRepository;
import top.fjy8018.jpadatasource.repository.primary.ProductRepository;

import javax.sql.DataSource;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Supplier;
import java.util.stream.Collectors;
import java.util.stream.StreamSupport;

/**
 * @author F嘉阳
 * @date 2019-03-30 9:24
 */
@Configuration
public class DataAccessConfig {

 private final ObjectProvider<SchemaManagementProvider> providers;

 private final HibernateProperties hibernateProperties;

 private final JpaProperties properties;

 private final ObjectProvider<PhysicalNamingStrategy> physicalNamingStrategy;

 private final ObjectProvider<ImplicitNamingStrategy> implicitNamingStrategy;

 private final ConfigurableListableBeanFactory beanFactory;

 private final ObjectProvider<HibernatePropertiesCustomizer> hibernatePropertiesCustomizers;

 @Autowired
 public DataAccessConfig(ObjectProvider<SchemaManagementProvider> providers, HibernateProperties hibernateProperties, JpaProperties properties, ObjectProvider<PhysicalNamingStrategy> physicalNamingStrategy, ObjectProvider<ImplicitNamingStrategy> implicitNamingStrategy, ConfigurableListableBeanFactory beanFactory, ObjectProvider<HibernatePropertiesCustomizer> hibernatePropertiesCustomizers) {
 this.providers = providers;
 this.hibernateProperties = hibernateProperties;
 this.properties = properties;
 this.physicalNamingStrategy = physicalNamingStrategy;
 this.implicitNamingStrategy = implicitNamingStrategy;
 this.beanFactory = beanFactory;
 this.hibernatePropertiesCustomizers = hibernatePropertiesCustomizers;
 }

 @Bean
 @Primary
 @ConfigurationProperties("spring.datasource.first")
 public DataSourceProperties firstDataSourceProperties() {
 return new DataSourceProperties();
 }

 @Bean
 @Primary
 @ConfigurationProperties("spring.datasource.first.configuration")
 public HikariDataSource firstDataSource() {
 return firstDataSourceProperties().initializeDataSourceBuilder()
 .type(HikariDataSource.class).build();
 }

 @Bean
 @ConfigurationProperties("spring.datasource.second")
 public DataSourceProperties secondDataSourceProperties() {
 return new DataSourceProperties();
 }

 @Bean
 @ConfigurationProperties("spring.datasource.second.configuration")
 public HikariDataSource secondDataSource() {
 return secondDataSourceProperties().initializeDataSourceBuilder()
 .type(HikariDataSource.class).build();
 }

 @Bean
 @Primary
 public LocalContainerEntityManagerFactoryBean firstEntityManagerFactory(
 EntityManagerFactoryBuilder builder, @Qualifier("firstDataSource") DataSource dataSource) {
 return builder
 .dataSource(dataSource)
 // 加入规约配置
 .properties(getVendorProperties(dataSource))
 .packages(Product.class)
 .persistenceUnit("first")
 .build();
 }

 @Bean
 public LocalContainerEntityManagerFactoryBean secondEntityManagerFactory(
 EntityManagerFactoryBuilder builder, @Qualifier("secondDataSource") DataSource dataSource) {
 return builder
 .dataSource(dataSource)
 .properties(getVendorProperties(dataSource))
 .packages(Order.class)
 .persistenceUnit("second")
 .build();
 }

 @Bean
 public PlatformTransactionManager primaryTransactionManager(
 @Qualifier("firstEntityManagerFactory") LocalContainerEntityManagerFactoryBean entityManagerFactory) {
 return new JpaTransactionManager(entityManagerFactory.getObject());
 }

 @Bean
 public PlatformTransactionManager backupTransactionManager(
 @Qualifier("secondEntityManagerFactory") LocalContainerEntityManagerFactoryBean entityManagerFactory) {
 return new JpaTransactionManager(entityManagerFactory.getObject());
 }

 @EnableJpaRepositories(basePackageClasses = ProductRepository.class,
 entityManagerFactoryRef = "firstEntityManagerFactory", transactionManagerRef = "primaryTransactionManager")
 @Primary
 public class PrimaryConfiguration {
 }

 @EnableJpaRepositories(basePackageClasses = OrderRepository.class,
 entityManagerFactoryRef = "secondEntityManagerFactory", transactionManagerRef = "backupTransactionManager")
 public class secondConfiguration {
 }

 /**
 * 获取配置文件信息
 *
 * @param dataSource
 * @return
 */
 private Map<String, Object> getVendorProperties(DataSource dataSource) {
 List<HibernatePropertiesCustomizer> hibernatePropertiesCustomizers = determineHibernatePropertiesCustomizers(
 physicalNamingStrategy.getIfAvailable(),
 implicitNamingStrategy.getIfAvailable(), beanFactory,
 this.hibernatePropertiesCustomizers.orderedStream()
 .collect(Collectors.toList()));
 Supplier<String> defaultDdlMode = () -> new HibernateDefaultDdlAutoProvider(providers)
 .getDefaultDdlAuto(dataSource);
 return new LinkedHashMap<>(this.hibernateProperties.determineHibernateProperties(
 properties.getProperties(),
 new HibernateSettings().ddlAuto(defaultDdlMode)
 .hibernatePropertiesCustomizers(
 hibernatePropertiesCustomizers)));
 }

 /**
 * 命名策略自动判断
 *
 * @param physicalNamingStrategy
 * @param implicitNamingStrategy
 * @param beanFactory
 * @param hibernatePropertiesCustomizers
 * @return
 */
 private List<HibernatePropertiesCustomizer> determineHibernatePropertiesCustomizers(
 PhysicalNamingStrategy physicalNamingStrategy,
 ImplicitNamingStrategy implicitNamingStrategy,
 ConfigurableListableBeanFactory beanFactory,
 List<HibernatePropertiesCustomizer> hibernatePropertiesCustomizers) {
 List<HibernatePropertiesCustomizer> customizers = new ArrayList<>();
 if (ClassUtils.isPresent(
 "org.hibernate.resource.beans.container.spi.BeanContainer",
 getClass().getClassLoader())) {
 customizers
 .add((properties) -> properties.put(AvailableSettings.BEAN_CONTAINER,
 new SpringBeanContainer(beanFactory)));
 }
 if (physicalNamingStrategy != null || implicitNamingStrategy != null) {
 customizers.add(new NamingStrategiesHibernatePropertiesCustomizer(
 physicalNamingStrategy, implicitNamingStrategy));
 }
 customizers.addAll(hibernatePropertiesCustomizers);
 return customizers;
 }

 /**
 * 自动进行建表操作
 */
 class HibernateDefaultDdlAutoProvider implements SchemaManagementProvider {

 private final Iterable<SchemaManagementProvider> providers;

 HibernateDefaultDdlAutoProvider(Iterable<SchemaManagementProvider> providers) {
 this.providers = providers;
 }

 public String getDefaultDdlAuto(DataSource dataSource) {
 if (!EmbeddedDatabaseConnection.isEmbedded(dataSource)) {
 return "none";
 }
 SchemaManagement schemaManagement = getSchemaManagement(dataSource);
 if (SchemaManagement.MANAGED.equals(schemaManagement)) {
 return "none";
 }
 return "create-drop";

 }

 @Override
 public SchemaManagement getSchemaManagement(DataSource dataSource) {
 return StreamSupport.stream(this.providers.spliterator(), false)
 .map((provider) -> provider.getSchemaManagement(dataSource))
 .filter(SchemaManagement.MANAGED::equals).findFirst()
 .orElse(SchemaManagement.UNMANAGED);
 }

 }

 private static class NamingStrategiesHibernatePropertiesCustomizer
 implements HibernatePropertiesCustomizer {

 private final PhysicalNamingStrategy physicalNamingStrategy;

 private final ImplicitNamingStrategy implicitNamingStrategy;

 NamingStrategiesHibernatePropertiesCustomizer(
 PhysicalNamingStrategy physicalNamingStrategy,
 ImplicitNamingStrategy implicitNamingStrategy) {
 this.physicalNamingStrategy = physicalNamingStrategy;
 this.implicitNamingStrategy = implicitNamingStrategy;
 }

 /**
 * 数据库命名映射策略
 *
 * @param hibernateProperties the JPA vendor properties to customize
 */
 @Override
 public void customize(Map<String, Object> hibernateProperties) {
 if (this.physicalNamingStrategy != null) {
 hibernateProperties.put("hibernate.physical_naming_strategy",
 this.physicalNamingStrategy);
 }
 if (this.implicitNamingStrategy != null) {
 hibernateProperties.put("hibernate.implicit_naming_strategy",
 this.implicitNamingStrategy);
 }
 }
 }
}

配置Bean完成后则规约配置会生效，便无需在配置文件显示配置DDL语句执行开关

启动日志也可看出结果符合预期

总结

由此发现，在SpringBoot版本迁移过程中不一定互相兼容，同时官方文档尽管全面，但随着版本变化文档也有不准确情况。而如果完全依靠爬源码复制源码配置也是无法实现最简单化的配置，需要结合文档、源码、日志逐一分析

工程源码已开放，注意配置自己的数据库连接地址，工程内的连接地址已经失效

GitHub源码

LeetCode刷题记录

Sun, 06 Jan 2019 19:43:00 +0800

第1天 Jewels and Stones （771）

题号771 Jewels and Stones
题目描述：

给定字符串J 代表石头中宝石的类型，和字符串 S代表你拥有的石头。 S 中每个字符代表了一种你拥有的石头的类型，你想知道你拥有的石头中有多少是宝石。

J 中的字母不重复，J 和 S中的所有字符都是字母。字母区分大小写，因此"a"和"A"是不同类型的石头。
解题思路：字符串遍历比对
心得：查看了排名较前的解法，大部分也是暴力比对，少数取巧解法获得更好的时间复杂度

排名最前的解法

第2天 Unique Email Addresse （929）

题号929 Unique Email Addresses
题目描述：
每封电子邮件都由一个本地名称和一个域名组成，以 @ 符号分隔。

例如，在 alice@leetcode.com中， alice 是本地名称，而 leetcode.com 是域名。

除了小写字母，这些电子邮件还可能包含 '.' 或 '+'。

如果在电子邮件地址的本地名称部分中的某些字符之间添加句点（'.'），则发往那里的邮件将会转发到本地名称中没有点的同一地址。例如，"alice.z@leetcode.com” 和 “alicez@leetcode.com” 会转发到同一电子邮件地址。（请注意，此规则不适用于域名。）

如果在本地名称中添加加号（'+'），则会忽略第一个加号后面的所有内容。这允许过滤某些电子邮件，例如 m.y+name@email.com 将转发到 my@email.com。（同样，此规则不适用于域名。）

可以同时使用这两个规则。

给定电子邮件列表 emails，我们会向列表中的每个地址发送一封电子邮件。实际收到邮件的不同地址有多少？

示例：
```
输入：["test.email+alex@leetcode.com","test.e.mail+bob.cathy@leetcode.com","testemail+david@lee.tcode.com"]
输出：2
解释：实际收到邮件的是 "testemail@leetcode.com" 和 "testemail@lee.tcode.com"。
```
提示：
- 1 <= emails[i].length <= 100
- 1 <= emails.length <= 100
- 每封 emails[i] 都包含有且仅有一个 '@' 字符。
解题思路：字符串替换+去除重复元素
心得体会：网站测试用例不全导致排名较前的解法有问题，并不能完全符合题目要求，自己的代码时间复杂度还是较高

国外Java精简正则表达式解法

第3天 Sort Array By Parity （905）

题号905 Sort Array By Parity
题目描述：给定一个非负整数数组 A，返回一个由 A 的所有偶数元素组成的数组，后面跟 A 的所有奇数元素。
解题思路：双指针
心得体会：经典解法，时间复杂度为O(n)，遍历数组，将偶数从前往后存放，奇数则从后往前存放

第4天 Encode and Decode TinyURL（535）

题号535 Encode and Decode TinyURL
题目描述：

TinyURL是一种URL简化服务，比如：当你输入一个URL https://leetcode.com/problems/design-tinyurl 时，它将返回一个简化的URL http://tinyurl.com/4e9iAk.
解题思路：短网址算法简化版
心得体会：哈希运算后再转16进制即可明显缩短网址长度，题目没有要求生成的长度，故没有实现真正的短网址算法，也有不运算通过集合直接存储返回下标的，目前暂时没有在讨论区看到更好的Java解法

还有更加取巧的解法，直接暴力返回。。然而还显示739 / 739 个通过测试用例。。排名较前的多为该解法

第5天 Subsets（78）

题号78 Subsets
题目描述：给定一组不含重复元素的整数数组 nums，返回该数组所有可能的子集（幂集）。说明：解集不能包含重复的子集。
解题思路：深度优先或者广度优先算法
心得体会：思考了一段时间，之后在讨论区看到BFS、DFS解法，最后看到使用排序和递归结合的DFS简洁解法，拓宽了思路

排序和递归结合的DFS解法

第6天 Minimum Add to Make Parentheses Valid（921）

题号921 Minimum Add to Make Parentheses Valid
题目描述：
给定一个由 '(' 和 ')' 括号组成的字符串 S，我们需要添加最少的括号（ '(' 或是 ')'，可以在任何位置），以使得到的括号字符串有效。

从形式上讲，只有满足下面几点之一，括号字符串才是有效的：
- 它是一个空字符串，或者
- 它可以被写成 AB （A 与 B 连接）, 其中 A 和 B 都是有效字符串，或者
- 它可以被写作 (A)，其中 A 是有效字符串。
给定一个括号字符串，返回为使结果字符串有效而必须添加的最少括号数。
解题思路：栈，遇到左括号则右括号入栈，遇到右括号则弹出栈顶元素并判断其是否为左括号，栈为空或者判断为false都将计数器自增，最终返回栈大小与计数器之和即可
心得体会：按照题目标签选题为栈，故使用了栈来实现，但耗时其实比直接使用左右括号计数器要长

左右括号计数器解法

第7天 Reverse String（344）

题号 344 Reverse String
题目描述：编写一个函数，其作用是将输入的字符串反转过来。
解题思路：双指针
心得体会：简单题，两个指针分别从前往后和从后往前遍历并交换，但从讨论区也看到了更多的思路，例如使用ACII位运算解法等

ACII位运算解法

第8天 Remove Linked List Elements（203）

题号 203 Remove Linked List Elements
题目描述：删除链表中等于给定值 val 的所有节点。
解题思路：
- 单链表迭代删除元素
- 带虚拟头结点单链表迭代删除元素
- 递归实现
心得体会：简单题，使用指针逐个判断下一个元素是否为待删除元素，迭代直至链表尾部。题目虽然难度不大，但可以增强对链表和递归用法的理解。

非虚拟头结点实现

虚拟头结点实现

递归实现

第9天 Valid Parentheses（20）

题号 20 Valid Parentheses
题目描述：
给定一个只包括 '('，')'，'{'，'}'，'['，']' 的字符串，判断字符串是否有效。

有效字符串需满足：
1. 左括号必须用相同类型的右括号闭合。
2. 左括号必须以正确的顺序闭合。
注意空字符串可被认为是有效字符串。
解题思路：栈
心得体会：简单题，栈的基础操作，但注意出栈后仍需判断括号是否匹配以及遍历完毕还需判断栈是否为空

第10天 Binary Tree Inorder Traversal（94）

题号 94 Binary Tree Inorder Traversal
题目描述：
给定一个二叉树，返回它的中序遍历。

示例:
```
输入: [1,null,2,3]
 1
 \
 2
 /
 3

输出: [1,3,2]
```
解题思路：考察二叉树中序遍历递归实现和非递归实现
心得体会：简单题，二叉树的基础操作

递归实现

非递归实现

第11天 Search in a Binary Search Tree（700）

题号 700 Search in a Binary Search Tree
题目描述：
给定二叉搜索树（BST）的根节点和一个值。你需要在BST中找到节点值等于给定值的节点。返回以该节点为根的子树。如果节点不存在，则返回 NULL。

例如，
```
给定二叉搜索树:

 4
 / \
 2 7
 / \
 1 3

和值: 2
```
你应该返回如下子树:
```
 2 
 / \ 
 1 3
```
在上述示例中，如果要找的值是 5，但因为没有节点值为 5，我们应该返回 NULL。
解题思路：二叉树节点值比对、查找、递归实现
心得体会：简单题，二叉树的查找操作

第12天 Binary Tree Level Order Traversal II（107）

题号 107 Binary Tree Level Order Traversal II
题目描述：
给定一个二叉树，返回其节点值自底向上的层次遍历。（即按从叶子节点所在层到根节点所在的层，逐层从左向右遍历）

例如：给定二叉树 [3,9,20,null,null,15,7],
```
 3
 / \
 9 20
 / \
 15 7
```
返回其自底向上的层次遍历为：
```
[
 [15,7],
 [9,20],
 [3]
]
```
解题思路：深度优先遍历、广度优先遍历
心得体会：由于题目要求自底向上层次遍历，反向输出使用栈实现，考虑使用深度优先并结合队列实现，故使用非递归方式，但每层都要组合为一个集合，考虑使用双队列，解题过程中清楚这并不是最优解，后在评论区看到更优解法

评论区赞同数较高的解法

DFS solution:

public class Solution {
 public List<List<Integer>> levelOrderBottom(TreeNode root) {
 Queue<TreeNode> queue = new LinkedList<TreeNode>();
 List<List<Integer>> wrapList = new LinkedList<List<Integer>>();
 
 if(root == null) return wrapList;
 
 queue.offer(root);
 while(!queue.isEmpty()){
 // 记录队列中根节点数量
 int levelNum = queue.size();
 List<Integer> subList = new LinkedList<Integer>();
 for(int i=0; i<levelNum; i++) {
 if(queue.peek().left != null) queue.offer(queue.peek().left);
 if(queue.peek().right != null) queue.offer(queue.peek().right);
 subList.add(queue.poll().val);
 }
 wrapList.add(0, subList);
 }
 return wrapList;
 }
}

BFS solution:

public class Solution {
 public List<List<Integer>> levelOrderBottom(TreeNode root) {
 List<List<Integer>> wrapList = new LinkedList<List<Integer>>();
 levelMaker(wrapList, root, 0);
 return wrapList;
 }
 
 public void levelMaker(List<List<Integer>> list, TreeNode root, int level) {
 if(root == null) return;
 if(level >= list.size()) {
 list.add(0, new LinkedList<Integer>());
 }
 levelMaker(list, root.left, level+1);
 levelMaker(list, root.right, level+1);
 list.get(list.size()-level-1).add(root.val);
 }
}

第13天 Binary Tree Preorder Traversal（144）

题号 144 Binary Tree Preorder Traversal
题目描述：
给定一个二叉树，返回它的前序遍历。

示例:
```
输入: [1,null,2,3] 
 1
 \
 2
 /
 3 

输出: [1,2,3]
```
解题思路：二叉树先序遍历递归实现、非递归实现
心得体会：简单题，二叉树的基础操作

递归实现

非递归实现

第14天 Binary Tree Postorder Traversal（145）

题号 145 Binary Tree Postorder Traversal
题目描述：
给定一个二叉树，返回它的后序遍历。

示例:
```
输入: [1,null,2,3] 
 1
 \
 2
 /
 3 

输出: [3,2,1]
```
解题思路：二叉树递归操作、双栈使用
心得体会：递归实现较为简单，但非递归实现需要对栈有深刻的理解，通过双栈使用让每个节点在第三次遍历时输出

递归实现

非递归实现

第15天 Merge Two Sorted Lists（21）

题号 21 Merge Two Sorted Lists
题目描述：
将两个有序链表合并为一个新的有序链表并返回。新链表是通过拼接给定的两个链表的所有节点组成的。

示例：
```
输入：1->2->4, 1->3->4
输出：1->1->2->3->4->4
```
解题思路：链表插入操作
心得体会：简单题，注意边界值以及可能出现两个链表长度不等的情况

第16天 Remove Nth Node From End of List（19）

题号 19 Remove Nth Node From End of List
题目描述：
给定一个链表，删除链表的倒数第 n 个节点，并且返回链表的头结点。

示例：
```
给定一个链表: 1->2->3->4->5, 和 n = 2.

当删除了倒数第二个节点后，链表变为 1->2->3->5.
```
说明：

给定的 n 保证是有效的。

进阶：

使用一趟扫描实现
解题思路：第一趟扫描得到链表大小、用链表大小减去n得到节点顺位下标，按链表删除元素操作即可
心得体会：两趟扫描比较容易想到，解题后在讨论区看到评价较高的一趟扫描排序实现

两趟扫描实现

一趟扫描实现

A one pass solution can be done using pointers. Move one pointer fast –> n+1 places forward, to maintain a gap of n between the two pointers and then move both at the same speed. Finally, when the fast pointer reaches the end, the slow pointer will be n+1 places behind - just the right spot for it to be able to skip the next node.

Since the question gives that n is valid, not too many checks have to be put in place. Otherwise, this would be necessary.
public ListNode removeNthFromEnd(ListNode head, int n) {
 
 ListNode start = new ListNode(0);
 ListNode slow = start, fast = start;
 slow.next = head;
 
 //Move fast in front so that the gap between slow and fast becomes n
 for(int i=1; i<=n+1; i++) {
 fast = fast.next;
 }
 //Move fast to the end, maintaining the gap
 while(fast != null) {
 slow = slow.next;
 fast = fast.next;
 }
 //Skip the desired node
 slow.next = slow.next.next;
 return start.next;
}

第17天 Unique Morse Code Words（804）

题号 804 Unique Morse Code Words
题目描述：
国际摩尔斯密码定义一种标准编码方式，将每个字母对应于一个由一系列点和短线组成的字符串，比如: "a" 对应 ".-", "b" 对应 "-...", "c" 对应 "-.-.", 等等。

为了方便，所有26个英文字母对应摩尔斯密码表如下：
```
[".-","-...","-.-.","-..",".","..-.","--.","....","..",".---","-.-",".-..","--","-.","---",".--.","--.-",".-.","...","-","..-","...-",".--","-..-","-.--","--.."]
```
给定一个单词列表，每个单词可以写成每个字母对应摩尔斯密码的组合。例如，“cab” 可以写成 “-.-..–…"，(即 “-.-.” + “-…” + “.-“字符串的结合)。我们将这样一个连接过程称作单词翻译。

返回我们可以获得所有词不同单词翻译的数量。
```
例如:
输入: words = ["gin", "zen", "gig", "msg"]
输出: 2
解释: 
各单词翻译如下:
"gin" -> "--...-."
"zen" -> "--...-."
"gig" -> "--...--."
"msg" -> "--...--."

共有 2 种不同翻译, "--...-." 和 "--...--.".
```
注意:
- 单词列表words 的长度不会超过 100。
- 每个单词 words[i]的长度范围为 [1, 12]。
- 每个单词 words[i]只包含小写字母。
解题思路：通过给定的摩斯密码表翻译成字符串再利用Set去重
心得体会：翻译过程注意数组下标从0开始，故原本a对应96则需要-97，同时该问题不存在大写字母，可减少一部分逻辑

第18天 Intersection of Two Arrays（349）

题号 349 Intersection of Two Arrays
题目描述：
给定两个数组，编写一个函数来计算它们的交集。

示例 1:
```
输入: nums1 = [1,2,2,1], nums2 = [2,2]
输出: [2]
```
示例 2:
```
输入: nums1 = [4,9,5], nums2 = [9,4,9,8,4]
输出: [9,4]
```
说明:
- 输出结果中的每个元素一定是唯一的。
- 我们可以不考虑输出结果的顺序。
解题思路：使用集合对数组去重，并遍历另一个数组，遇到相同元素则添加进结果集，同时删除第一个数组集合中的元素，防止下次再次遇到相同的元素
心得体会：最容易想到的方式是对两个数组都进行去重后比较，但通过使用第一个去重集合在第二个数组遍历过程中遇到相同元素时去除该元素的方式，减少对第二个数组的去重操作，提高性能

第19天 Intersection of Two Arrays II（350）

题号 350 Intersection of Two Arrays II
题目描述：
给定两个数组，编写一个函数来计算它们的交集。

示例 1:
```
输入: nums1 = [1,2,2,1], nums2 = [2,2]
输出: [2,2]
```
示例 2:
```
输入: nums1 = [4,9,5], nums2 = [9,4,9,8,4]
输出: [4,9]
```
说明：
- 输出结果中每个元素出现的次数，应与元素在两个数组中出现的次数一致。
- 我们可以不考虑输出结果的顺序。
进阶:
- 如果给定的数组已经排好序呢？你将如何优化你的算法？
- 如果 nums1 的大小比 nums2 小很多，哪种方法更优？
- 如果 nums2 的元素存储在磁盘上，磁盘内存是有限的，并且你不能一次加载所有的元素到内存中，你该怎么办？
解题思路：首先对第一个数组进行词频统计，在对第二个数组遍历过程中遇到相同元素对词频减一并加入结果集，当词频为0则移除元素，由此实现每个元素出现的次数，应与元素在两个数组中出现的次数一致
心得体会：Map作为词频统计的经典用法，但注意当词频为0时需要移除元素，否则词频计数便没有意义

第20天 N-Repeated Element in Size 2N Array（961）

题号 961 N-Repeated Element in Size 2N Array
题目描述：
在大小为 2N 的数组 A 中有 N+1 个不同的元素，其中有一个元素重复了 N 次。

返回重复了 N 次的那个元素。

示例 1：
```
输入：[1,2,3,3]
输出：3
```
示例 2：
```
输入：[2,1,2,5,3,2]
输出：2
```
示例 3：
```
输入：[5,1,5,2,5,3,5,4]
输出：5
```
提示：
1. 4 <= A.length <= 10000
2. 0 <= A[i] < 10000
3. A.length 为偶数
解题思路：Map词频统计用法，一旦某个元素重复了N次则不再继续统计
心得体会：词频统计的经典用法

第21天 Single Number（136）

题号 136 Single Number
题目描述：
给定一个非空整数数组，除了某个元素只出现一次以外，其余每个元素均出现两次。找出那个只出现了一次的元素。

说明：

你的算法应该具有线性时间复杂度。你可以不使用额外空间来实现吗？

示例 1:
```
输入: [2,2,1]
输出: 1
```
示例 2:
```
输入: [4,1,2,1,2]
输出: 4
```
解题思路：Map词频统计用法，统计完成后寻找词频为1的即可
心得体会：Map词频统计法比较容易想到，解题后在官方解答区看到两个更灵活的解法，分别通过数学计算和位运算实现，开阔了解题思路

Map词频统计法

数学运算法

原理

$$ 2∗(a+b+c)−(a+a+b+b+c)=c $$

class Solution {
 public int singleNumber(int[] nums) {
 return (2 * sum(set(nums)) - sum(nums));
 }

 private int[] set(int[] nums) {
 HashSet<Integer> set = new HashSet<>();
 for (int num : nums) {
 set.add(num);
 }
 int[] res = new int[set.size()];
 int i = 0;
 for (Integer integer : set) {
 res[i++] = integer;
 }
 return res;
 }

 private int sum(int[] nums) {
 int sum = 0;
 for (int num : nums) {
 sum += num;
 }
 return sum;
 }
}

位运算法（已知性能最佳解法）

class Solution {
 public int singleNumber(int[] nums) {
 int result = 0;
 for (int i : nums) {
 // 一个数和0异或还是自己，一个数和自己异或是0
 result ^= i;
 }
 return result;
 }
}

第22天 Top K Frequent Elements（347）

题号 347 Top K Frequent Elements
题目描述：
给定一个非空的整数数组，返回其中出现频率前 k 高的元素。

示例 1:
```
输入: nums = [1,1,1,2,2,3], k = 2
输出: [1,2]
```
示例 2:
```
输入: nums = [1], k = 1
输出: [1]
```
说明：
- 你可以假设给定的 k 总是合理的，且 1 ≤ k ≤ 数组中不相同的元素的个数。
- 你的算法的时间复杂度必须优于 O(n log n) , n 是数组的大小。
解题思路：使用优先队列，维护出现频率前 k 高的元素
心得体会：Java内置的优先队列为小根堆，同时支持定义比较器，结合优先队列特性即可解答

第23天 Top K Frequent Words（692）

题号 692 Top K Frequent Words

题目描述：

给一非空的单词列表，返回前 k 个出现次数最多的单词。

返回的答案应该按单词出现频率由高到低排序。如果不同的单词有相同出现频率，按字母顺序排序。

示例 1：
输入: ["i", "love", "leetcode", "i", "love", "coding"], k = 2
输出: ["i", "love"]
解析: "i" 和 "love" 为出现次数最多的两个单词，均为2次。
 注意，按字母顺序 "i" 在 "love" 之前。
示例 2：
输入: ["the", "day", "is", "sunny", "the", "the", "the", "sunny", "is", "is"], k = 4
输出: ["the", "is", "sunny", "day"]
解析: "the", "is", "sunny" 和 "day" 是出现次数最多的四个单词，
 出现次数依次为 4, 3, 2 和 1 次。
注意：

假定 k 总为有效值， 1 ≤ k ≤ 集合元素数。

输入的单词均由小写字母组成。

扩展练习：

尝试以 O(n log k) 时间复杂度和 O(n) 空间复杂度解决。

解题思路：使用优先队列，维护前 k 个出现次数最多的单词，同时维护字典顺序
心得体会：与347题类似，但需要处理单词有相同出现频率，按字母顺序排序的情况，按照Java默认字符串排序方式和频率大小定义比较器即可

第24天 Range Sum Query - Immutable（303）

题号 303 Range Sum Query - Immutable
题目描述：
给定一个整数数组 nums，求出数组从索引 i 到 j (i ≤ j) 范围内元素的总和，包含 i, j 两点。

示例：
```
给定 nums = [-2, 0, 3, -5, 2, -1]，求和函数为 sumRange()

sumRange(0, 2) -> 1
sumRange(2, 5) -> -1
sumRange(0, 5) -> -3
```
说明:
1. 你可以假设数组不可变。
2. 会多次调用 sumRange 方法。
解题思路：典型的线段树应用场景，但该题可以使用数组维护区间总和，对于该题来说时间复杂度更低
心得体会：线段树解法从读题即可想到，而后由于说明为Immutable，故通过维护数组实现时间复杂度更低

数组实现

线段树实现

class NumArray {
 
 private SegmentTree<Integer> tree;

 public NumArray(int[] nums) {
 Integer[] list = new Integer[nums.length];
 for (int i = 0; i < nums.length; i++) {
 list[i] = nums[i];
 }
 tree = new SegmentTree<>(list,(a, b) -> a+b);
 }
 
 public int sumRange(int i, int j) {
 return tree.query(i,j);
 }
 
 class SegmentTree<E> {

 private E[] data;
 private E[] tree;
 private Merger<E> merger;

 public SegmentTree(E[] arr, Merger<E> merger) {
 this.merger = merger;
 this.data = (E[]) new Object[arr.length];
 // 拷贝副本
 for (int i = 0; i < arr.length; i++) {
 data[i] = arr[i];
 }
 // 创建线段树存储空间，4倍空间可以存储最坏情况下的线段树
 this.tree = (E[]) new Object[4 * arr.length];
 // 创建线段树
 buildSegmentTree(0, 0, data.length - 1);
 }

 /**
 * 在treeIndex的位置创建表示区间[l...r]的线段树
 *
 * @param treeIndex
 * @param left
 * @param right
 */
 private void buildSegmentTree(int treeIndex, int left, int right) {
 if (data.length==0){
 return;
 }
 // 递归终止条件为叶子节点
 if (left == right) {
 tree[treeIndex] = data[left];
 return;
 }
 // 左子树索引
 int leftIndex = leftChild(treeIndex);
 // 右子树索引
 int rightIndex = rightChild(treeIndex);
 // 等价于 (left + right)/2
 int mid = left + (right - left) / 2;
 buildSegmentTree(leftIndex, left, mid);
 buildSegmentTree(rightIndex, mid + 1, right);
 // 线段值由业务情况决定
 tree[treeIndex] = merger.merge(tree[leftIndex], tree[rightIndex]);
 }

 public E query(int queryL, int queryR) {
 if (queryL < 0 || queryL >= data.length ||
 queryR < 0 || queryR >= data.length || queryL > queryR) {
 throw new IllegalArgumentException("Index is illegal.");
 }
 return query(0, 0, data.length - 1, queryL, queryR);
 }

 private E query(int treeIndex, int left, int right, int queryL, int queryR) {
 // 查询区间与线段树区间相等则直接返回
 if (left == queryL && right == queryR) {
 return tree[treeIndex];
 }
 int leftIndex = leftChild(treeIndex);
 int rightIndex = rightChild(treeIndex);
 int mid = left + (right - left) / 2;
 // 若全在左子树
 if (queryR <= mid) {
 return query(leftIndex, left, mid, queryL, queryR);
 } else if (queryL >= mid + 1) {
 // 全在右子树
 return query(rightIndex, mid + 1, right, queryL, queryR);
 }
 // 分散在左右子树
 E leftResult = query(leftIndex, left, mid, queryL, mid);
 E rightResult = query(rightIndex, mid + 1, right, mid + 1, queryR);
 return merger.merge(leftResult, rightResult);
 }

 public int getSize() {
 return data.length;
 }

 public E get(int index) {
 isLegal(index);
 return data[index];
 }

 private void isLegal(int index) {
 if (index < 0 || index > data.length - 1) {
 throw new IllegalArgumentException("index out of bound");
 }
 }

 /**
 * 返回平衡二叉树的数组表示中，一个索引所表示的元素的左孩子节点的索引
 *
 * @param index
 * @return
 */
 private int leftChild(int index) {
 return index * 2 + 1;
 }

 /**
 * 返回平衡二叉树的数组表示中，一个索引所表示的元素的右孩子节点的索引
 *
 * @param index
 * @return
 */
 private int rightChild(int index) {
 return index * 2 + 2;
 }}
 
 @FunctionalInterface
 public interface Merger<E> {
 /**
 * 合并a、b两个对象
 *
 * @param a
 * @param b
 * @return
 */
 E merge(E a, E b);
 }
}

性能差异

解决方式	耗时	内存
数组	52 ms	41.2 MB
线段树	59 ms	42.2 MB

第25天 Range Sum Query - Mutable（307）

题号 307 Range Sum Query - Mutable
题目描述：
给定一个整数数组 nums，求出数组从索引 i 到 j (i ≤ j) 范围内元素的总和，包含 i, j 两点。

update(i, val) 函数可以通过将下标为 i 的数值更新为 val，从而对数列进行修改。

示例:
```
Given nums = [1, 3, 5]

sumRange(0, 2) -> 9
update(1, 2)
sumRange(0, 2) -> 8
```
说明:
1. 数组仅可以在 update 函数下进行修改。
2. 你可以假设 update 函数与 sumRange 函数的调用次数是均匀分布的。
解题思路：使用线段树或者数组动态维护区间总和
心得体会：与303题类似，也是线段树的典型应用场景，但此时数组为Mutable，虽然也可以通过数组维护区间总和，但更新时需要同时维护多个数值，此时的效率不如线段树

线段树实现

class NumArray {

 private SegmentTree<Integer> tree;

 public NumArray(int[] nums) {
 Integer[] arr = new Integer[nums.length];
 for (int i = 0; i < nums.length; i++) {
 arr[i] = nums[i];
 }
 tree = new SegmentTree<>(arr,(a, b) -> a+b);
 }

 public void update(int i, int val) {
 tree.set(i,val);
 }

 public int sumRange(int i, int j) {
 return tree.query(i,j);
 }
 
 class SegmentTree<E> {

 private E[] data;
 private E[] tree;
 private Merger<E> merger;

 public SegmentTree(E[] arr, Merger<E> merger) {
 this.merger = merger;
 this.data = (E[]) new Object[arr.length];
 // 拷贝副本
 for (int i = 0; i < arr.length; i++) {
 data[i] = arr[i];
 }
 // 创建线段树存储空间，4倍空间可以存储最坏情况下的线段树
 this.tree = (E[]) new Object[4 * arr.length];
 // 创建线段树
 buildSegmentTree(0, 0, data.length - 1);
 }

 /**
 * 在treeIndex的位置创建表示区间[l...r]的线段树
 *
 * @param treeIndex
 * @param left
 * @param right
 */
 private void buildSegmentTree(int treeIndex, int left, int right) {
 if (data.length==0){
 return;
 }
 // 递归终止条件为叶子节点
 if (left == right) {
 tree[treeIndex] = data[left];
 return;
 }
 // 左子树索引
 int leftIndex = leftChild(treeIndex);
 // 右子树索引
 int rightIndex = rightChild(treeIndex);
 // 等价于 (left + right)/2
 int mid = left + (right - left) / 2;
 buildSegmentTree(leftIndex, left, mid);
 buildSegmentTree(rightIndex, mid + 1, right);
 // 线段值由业务情况决定
 tree[treeIndex] = merger.merge(tree[leftIndex], tree[rightIndex]);
 }

 public E query(int queryL, int queryR) {
 if (queryL < 0 || queryL >= data.length ||
 queryR < 0 || queryR >= data.length || queryL > queryR) {
 throw new IllegalArgumentException("Index is illegal.");
 }
 return query(0, 0, data.length - 1, queryL, queryR);
 }

 /**
 * 在以treeIndex为根的线段树中[l...r]的范围里，搜索区间[queryL...queryR]的值
 *
 * @param treeIndex
 * @param left
 * @param right
 * @param queryL
 * @param queryR
 * @return
 */
 private E query(int treeIndex, int left, int right, int queryL, int queryR) {
 // 查询区间与线段树区间相等则直接返回
 if (left == queryL && right == queryR) {
 return tree[treeIndex];
 }
 int leftIndex = leftChild(treeIndex);
 int rightIndex = rightChild(treeIndex);
 int mid = left + (right - left) / 2;
 // 若全在左子树
 if (queryR <= mid) {
 return query(leftIndex, left, mid, queryL, queryR);
 } else if (queryL >= mid + 1) {
 // 全在右子树
 return query(rightIndex, mid + 1, right, queryL, queryR);
 }
 // 分散在左右子树
 E leftResult = query(leftIndex, left, mid, queryL, mid);
 E rightResult = query(rightIndex, mid + 1, right, mid + 1, queryR);
 return merger.merge(leftResult, rightResult);
 }

 /**
 * 将index位置的值，更新为e
 *
 * @param index
 * @param e
 */
 public void set(int index,E e){
 if (data.length==0){
 return;
 }
 isLegal(index);
 data[index] = e;
 set(0,0,data.length-1,index,e);
 }

 /**
 * 在以treeIndex为根的线段树中更新index的值为e
 *
 * @param treeIndex
 * @param left
 * @param right
 * @param index
 * @param e
 */
 private void set(int treeIndex,int left,int right,int index,E e){
 if (left == right){
 // 更新叶子结点
 tree[treeIndex] = e;
 return;
 }

 int mid = left+(right-left)/2;
 int leftTreeIndex = leftChild(treeIndex);
 int rightTreeIndex = rightChild(treeIndex);

 if (index<=mid){
 set(leftTreeIndex,left,mid,index,e);
 }else {
 set(rightTreeIndex,mid+1,right,index,e);
 }
 // 更新线段树节点值
 tree[treeIndex] = merger.merge(tree[leftTreeIndex],tree[rightTreeIndex]);
 }

 public int getSize() {
 return data.length;
 }

 public E get(int index) {
 isLegal(index);
 return data[index];
 }

 private void isLegal(int index) {
 if (index < 0 || index > data.length - 1) {
 throw new IllegalArgumentException("index out of bound");
 }
 }

 /**
 * 返回平衡二叉树的数组表示中，一个索引所表示的元素的左孩子节点的索引
 *
 * @param index
 * @return
 */
 private int leftChild(int index) {
 return index * 2 + 1;
 }

 /**
 * 返回平衡二叉树的数组表示中，一个索引所表示的元素的右孩子节点的索引
 *
 * @param index
 * @return
 */
 private int rightChild(int index) {
 return index * 2 + 2;
 }}
 
 @FunctionalInterface
public interface Merger<E> {
 /**
 * 合并a、b两个对象
 *
 * @param a
 * @param b
 * @return
 */
 E merge(E a, E b);
}
 
 
}

/**
 * Your NumArray object will be instantiated and called as such:
 * NumArray obj = new NumArray(nums);
 * obj.update(i,val);
 * int param_2 = obj.sumRange(i,j);
 */

数组实现

class NumArray {

 int[] nums;
 int[] origin;

 public NumArray(int[] nums) {
 origin = new int[nums.length];
 if (nums.length>0){
 origin[0] = nums[0];
 }
 // 第i个位置存储索引[0..i]的和，最后一个位置则为整个数组的和
 for (int i = 1; i < nums.length; i++) {
 origin[i] = nums[i];
 nums[i] += nums[i - 1];
 }

 this.nums = nums;
 }

 public void update(int i, int val) {
 // 更新位置之后的值均要变化
 int oldVal = origin[i];
 int diff = val - oldVal;
 for (int j = i ; j < nums.length; j++) {
 nums[j] += diff;
 }
 origin[i] = val;
 }

 public int sumRange(int i, int j) {
 if (i == 0) {
 return nums[j];
 }
 // 计算区间和
 return nums[j] - nums[i - 1];
 }
}

性能差异

解决方式	耗时	内存
线段树	60 ms	47.3 MB
数组	199 ms	49.3 MB

第26天 Implement Trie (Prefix Tree)（208）

题号 208 Implement Trie (Prefix Tree)
题目描述：
实现一个 Trie (前缀树)，包含 insert, search, 和 startsWith 这三个操作。

示例:
```
Trie trie = new Trie();

trie.insert("apple");
trie.search("apple"); // 返回 true
trie.search("app"); // 返回 false
trie.startsWith("app"); // 返回 true
trie.insert("app"); 
trie.search("app"); // 返回 true
```
说明:
- 你可以假设所有的输入都是由小写字母 a-z 构成的。
- 保证所有输入均为非空字符串。
解题思路：字典树（前缀树）实现
心得体会：字典树（前缀树）实现

字典树实现

class Trie {

 /**
 * 多叉树结构
 */
 private class Node {
 private boolean isWord;
 private HashMap<Character, Node> next;

 public Node(boolean isWord) {
 this.isWord = isWord;
 this.next = new HashMap<>();
 }

 public Node() {
 this(false);
 }
 }

 private Node root;
 private int size;

 /** Initialize your data structure here. */
 public Trie() {
 // 带虚拟头结点
 this.root = new Node();
 this.size = 0;
 }

 /** Inserts a word into the trie. */
 public void insert(String word) {
 Node cur = root;
 for (int i = 0; i < word.length(); i++) {
 char c = word.charAt(i);
 // 不存在当前字母节点
 if (cur.next.get(c) == null) {
 cur.next.put(c, new Node());
 }
 // 已经存在当前字母节点，继续向下寻找
 cur = cur.next.get(c);
 }
 // 字母添加完毕，查看当前单词末尾是否已经被标记，若被标记则说明重复添加
 if (!cur.isWord) {
 // 标记为单词末尾
 cur.isWord = true;
 size++;
 }
 }

 /** Returns if the word is in the trie. */
 public boolean search(String word) {
 Node cur = root;
 for (int i = 0; i < word.length(); i++) {
 char c = word.charAt(i);
 if (cur.next.get(c) == null) {
 return false;
 }
 cur = cur.next.get(c);
 }
 return cur.isWord;
 }

 /** Returns if there is any word in the trie that starts with the given prefix. */
 public boolean startsWith(String prefix) {
 Node cur = root;
 for (int i = 0; i < prefix.length(); i++) {
 char c = prefix.charAt(i);
 if (cur.next.get(c)==null){
 return false;
 }
 cur = cur.next.get(c);
 }
 return true;
 }
}

第27天 Add and Search Word - Data structure design（211）

题号 211 Add and Search Word - Data structure design
题目描述：
设计一个支持以下两种操作的数据结构：
```
void addWord(word)
bool search(word)
```
search(word) 可以搜索文字或正则表达式字符串，字符串只包含字母 . 或 a-z 。 . 可以表示任何一个字母。

示例:
```
addWord("bad")
addWord("dad")
addWord("mad")
search("pad") -> false
search("bad") -> true
search(".ad") -> true
search("b..") -> true
```
说明:

你可以假设所有单词都是由小写字母 a-z 组成的。
解题思路：字典树实现
心得体会：典型字典树（前缀树）应用场景

字典树实现

class WordDictionary {

 /**
 * 多叉树结构
 */
 private class Node {
 private boolean isWord;
 private HashMap<Character, Node> next;

 public Node(boolean isWord) {
 this.isWord = isWord;
 this.next = new HashMap<>();
 }

 public Node() {
 this(false);
 }
 }

 private Node root;

 /** Initialize your data structure here. */
 public WordDictionary() {
 // 带虚拟头结点
 this.root = new Node();
 }

 /** Adds a word into the data structure. */
 public void addWord(String word) {
 Node cur = root;
 for (int i = 0; i < word.length(); i++) {
 char c = word.charAt(i);
 // 不存在当前字母节点
 if (cur.next.get(c) == null) {
 cur.next.put(c, new Node());
 }
 // 已经存在当前字母节点，继续向下寻找
 cur = cur.next.get(c);
 }
 // 字母添加完毕，查看当前单词末尾是否已经被标记，若被标记则说明重复添加
 if (!cur.isWord) {
 // 标记为单词末尾
 cur.isWord = true;
 }
 }

 /** Returns if the word is in the data structure. A word could contain the dot character '.' to represent any one letter. */
 public boolean search(String word) {
 return match(word,root,0);
 }
 
 private boolean match(String word,Node node,int index){
 // 终止条件
 if (index == word.length()){
 return node.isWord;
 }
 char c = word.charAt(index);
 // 判断通配符
 if (c != '.'){
 if (node.next.get(c)==null){
 return false;
 }
 return match(word,node.next.get(c),index+1);
 }
 // 遍历当前节点所有的子节点
 for (Character w : node.next.keySet()) {
 // 如果有一个节点返回true则为true
 if (match(word,node.next.get(w),index+1)){
 return true;
 }
 }
 return false;
 }
 }

/**
 * Your WordDictionary object will be instantiated and called as such:
 * WordDictionary obj = new WordDictionary();
 * obj.addWord(word);
 * boolean param_2 = obj.search(word);
 */

第28天 Map Sum Pairs（677）

题号 677 Map Sum Pairs
题目描述：
实现一个 MapSum 类里的两个方法，insert 和 sum。

对于方法 insert，你将得到一对（字符串，整数）的键值对。字符串表示键，整数表示值。如果键已经存在，那么原来的键值对将被替代成新的键值对。

对于方法 sum，你将得到一个表示前缀的字符串，你需要返回所有以该前缀开头的键的值的总和。

示例 1:
```
输入: insert("apple", 3), 输出: Null
输入: sum("ap"), 输出: 3
输入: insert("app", 2), 输出: Null
输入: sum("ap"), 输出: 5
```
解题思路：字典树
心得体会：使用字典树记录字母节点，到节点尾部维护权重值，更新时同时更新权重值即可

字典树实现

第29天 Validate Binary Search Tree（98）

题号 98 Validate Binary Search Tree
题目描述：
给定一个二叉树，判断其是否是一个有效的二叉搜索树。

假设一个二叉搜索树具有如下特征：
- 节点的左子树只包含小于当前节点的数。
- 节点的右子树只包含大于当前节点的数。
- 所有左子树和右子树自身必须也是二叉搜索树。
示例 1:
```
输入:
 2
 / \
 1 3
输出: true
```
示例 2:
```
输入:
 5
 / \
 1 4
 / \
 3 6
输出: false
解释: 输入为: [5,1,4,null,null,3,6]。
 根节点的值为 5 ，但是其右子节点值为 4 。
```
解题思路：二叉树中序遍历
心得体会：利用二分搜索树中序遍历有序性的特点，对中序遍历集合进行检查，若发现无序则返回。该解法可进一步优化为在执行中序遍历过程中保留上一次递归结果进行判断，若发现无序则无需遍历所有节点。

第30天 Delete Node in a BST（450）

题号 450 Delete Node in a BST
题目描述：
给定一个二叉搜索树的根节点 root 和一个值 key，删除二叉搜索树中的 key 对应的节点，并保证二叉搜索树的性质不变。返回二叉搜索树（有可能被更新）的根节点的引用。

一般来说，删除节点可分为两个步骤：
1. 首先找到需要删除的节点；
2. 如果找到了，删除它。
说明： 要求算法时间复杂度为 O(h)，h 为树的高度。

示例:
```
root = [5,3,6,2,4,null,7]
key = 3

 5
 / \
 3 6
 / \ \
2 4 7

给定需要删除的节点值是 3，所以我们首先找到 3 这个节点，然后删除它。

一个正确的答案是 [5,4,6,2,null,null,7], 如下图所示。

 5
 / \
 4 6
 / \
2 7

另一个正确答案是 [5,2,6,null,4,null,7]。

 5
 / \
 2 6
 \ \
 4 7
```
解题思路：二分搜索树删除节点
心得体会：对于二分搜索树删除节点，此处选择该节点右子树中最小的节点替代被删除节点以保证二分搜索树的性质，相反，也可以通过使用该节点的左子树中最大的节点替代。

第31天 Binary Tree Level Order Traversal（102）

题号 102 Binary Tree Level Order Traversal
题目描述：
给定一个二叉树，返回其按层次遍历的节点值。（即逐层地，从左到右访问所有节点）。

例如: 给定二叉树: [3,9,20,null,null,15,7],
```
 3
 / \
 9 20
 / \
 15 7
```
返回其层次遍历结果：
```
[
 [3],
 [9,20],
 [15,7]
]
```
解题思路：二分搜索树层次遍历
心得体会：与[107题](#第12天 Binary Tree Level Order Traversal II（107)类似

第32天 Binary Search Tree Iterator（173）

题号 173 Binary Search Tree Iterator
题目描述：
实现一个二叉搜索树迭代器。你将使用二叉搜索树的根节点初始化迭代器。

调用 next() 将返回二叉搜索树中的下一个最小的数。

示例：
```
BSTIterator iterator = new BSTIterator(root);
iterator.next(); // 返回 3
iterator.next(); // 返回 7
iterator.hasNext(); // 返回 true
iterator.next(); // 返回 9
iterator.hasNext(); // 返回 true
iterator.next(); // 返回 15
iterator.hasNext(); // 返回 true
iterator.next(); // 返回 20
iterator.hasNext(); // 返回 false
```
提示：
- next() 和 hasNext() 操作的时间复杂度是 O(1)，并使用 O(h) 内存，其中 h 是树的高度。
- 你可以假设 next() 调用总是有效的，也就是说，当调用 next() 时，BST 中至少存在一个下一个最小的数。
解题思路：使用栈或者队列实现
心得体会：比较简单的方式是使用队列存储中序遍历集合，依次出队则保证为树中最小的元素，也评论区看到栈的解法，时间复杂度更低，但实现相对复杂

队列解法

栈解法

/**
 * Definition for binary tree
 * public class TreeNode {
 * int val;
 * TreeNode left;
 * TreeNode right;
 * TreeNode(int x) { val = x; }
 * }
 */

public class BSTIterator {
 
 private Stack<TreeNode> stack;
 public BSTIterator(TreeNode root) {
 stack = new Stack<>();
 TreeNode cur = root;
 while(cur != null){
 stack.push(cur);
 if(cur.left != null)
 cur = cur.left;
 else
 break;
 }
 }

 /** @return whether we have a next smallest number */
 public boolean hasNext() {
 return !stack.isEmpty();
 }

 /** @return the next smallest number */
 public int next() {
 TreeNode node = stack.pop();
 TreeNode cur = node;
 // traversal right branch
 if(cur.right != null){
 cur = cur.right;
 while(cur != null){
 stack.push(cur);
 if(cur.left != null)
 cur = cur.left;
 else
 break;
 }
 }
 return node.val;
 }
}

/**
 * Your BSTIterator will be called like this:
 * BSTIterator i = new BSTIterator(root);
 * while (i.hasNext()) v[f()] = i.next();
 */

mybatis中遇到的坑

Thu, 27 Dec 2018 12:37:00 +0800

mysql驱动问题

问题描述

问题发生时间（2018-12-27）

环境版本：

SpringBoot 2.1.1
Mybatis 1.3.2

原始配置

spring:
 datasource:
 driver-class-name: com.mysql.jdbc.Driver
 username: hikaricp
 password: hikaricp
 url: jdbc:mysql://localhost:3306/db_test?characterEncoding=utf-8&useSSL=false

报错信息

Registered driver with driverClassName=com.mysql.jdbc.Driver was not found,

报错提示

Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.

解决办法

从提示中可看出，驱动名称有变，更正驱动名称即可

更正后配置

spring:
 datasource:
 driver-class-name: com.mysql.cj.jdbc.Driver
 username: hikaricp
 password: hikaricp
 url: jdbc:mysql://localhost:3306/db_test?characterEncoding=utf-8&useSSL=false

同时若使用IDEA自带提示也可看出问题，有两种驱动都存在的问题，希望后续修复

另一种解决方案，从SpringBoot2.0之后连接池使用hikari，可根据url自动识别驱动，即如下配置也可成功运行

spring:
 datasource:
 username: hikaricp
 password: hikaricp
 url: jdbc:mysql://localhost:3306/db_test?characterEncoding=utf-8&useSSL=false

注解使用简化

前景提要

进入SpringBoot时代后“注解驱动开发”就已经成为趋势，mybatis官方也提供注解替代xml配置的方案

问题描述

问题发现时间（2018-12-27）

环境版本：

SpringBoot 2.1.1
Mybatis 1.3.2

在早期版本的mybatis-starter还需要进行实体映射xml文件配置，这次在使用1.3.2版本时再次查了下官方源码，发现已经无需使用映射文件，其底层会自动通过反射识别对象属性类型，并且去除了不少早期版本使用的注解（当前仍可以使用，但不使用功能正常）

无需手动创建SqlSession或DataSource

官方文档说明：

As you may already know, to use MyBatis with Spring you need at least an SqlSessionFactory and at least one mapper interface.

MyBatis-Spring-Boot-Starter will:

Autodetect an existing DataSource.

Will create and register an instance of a SqlSessionFactory passing that DataSource as an input using the SqlSessionFactoryBean.

Will create and register an instance of a SqlSessionTemplate got out of the SqlSessionFactory.

Autoscan your mappers, link them to the SqlSessionTemplate and register them to Spring context so they can be injected into your beans.

支持版本为Spring 4.3+

You just need to create a normal Spring boot application and let the mapper be injected like follows(available on Spring 4.3+):

@MapperScan -> @Mapper

早期版本需要在启动类上加上@MapperScan(basePackages = "xxx")注解扫描mapper类，新版本可直接在mapper包上使用@Mapper注解即可

The MyBatis-Spring-Boot-Starter will search, by default, for mappers marked with the @Mapper annotation.

除非需要自定义注解或者不使用@Mapper才需要加上@MapperScan

You may want to specify a custom annotation or a marker interface for scanning. If so, you must use the @MapperScan annotation.

@Results与Mapper.xml

早期版本需要手动指定类型映射，并在resources目录下进行xml配置，新版本自动通过反射识别属性类型，无需进行xml配置

HBase伪分布式安装

Sat, 15 Dec 2018 19:18:00 +0800

Hadoop伪分布式安装

基础环境

Hadoop：版本2.7.3，下载地址
HBase：版本1.2.4，下载地址
JDK8

jdk 安装命令

$ yum -y install java-devel java

查看当前系统jre和jdk路径，输入update-alternatives --config java和update-alternatives --config javac

$ update-alternatives --config java

共有 1 个提供“java”的程序。

 选项 命令
-----------------------------------------------
*+ 1 java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java)

$ update-alternatives --config javac

共有 1 个提供“javac”的程序。

 选项 命令
-----------------------------------------------
*+ 1 java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/bin/javac)

查看环境

$ ll
总用量 311132
-rw-------. 1 root root 1241 8月 11 10:53 anaconda-ks.cfg
-rw-r--r--. 1 root root 214092195 8月 26 2016 hadoop-2.7.3.tar.gz
-rw-r--r--. 1 root root 104497899 10月 26 2016 hbase-1.2.4-bin.tar.gz
$ java -version
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
$ javac -version
javac 1.8.0_191

免密登录

新建hadoop用户，忽略密码强度提示

$ adduser hadoop
$ passwd hadoop

为方便远程连接和学习，同时授予root权限，输入whereis sudoers查看权限控制文件位置

$ whereis sudoers
sudoers: /etc/sudoers /etc/sudoers.d /usr/libexec/sudoers.so /usr/share/man/man5/sudoers.5.gz

vi /etc/sudoers修改内容为

root ALL=(ALL) ALL
hadoop ALL=(ALL) ALL

切换至hadoop用户，使用 ssh-keygen -t rsa生成私钥信息，位置在/home/hadoop/，并且忽略所有提示输入的信息

查看生成的文件，该SSH文件夹为隐藏文件夹，故用ls -a查看，可见自动生成了.ssh目录和id_rsa,id_rsa.pub两个文件

$ ls /home/hadoop/.ssh/
id_rsa id_rsa.pub

复制公钥到.ssh目录

$ cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys

免密测试

$ ssh localhost
The authenticity of host 'localhost (::1)' can't be established.
ECDSA key fingerprint is SHA256:3UEr3lx2jznmN3FL3SflViR05IZe6AWweb3TeSsfX0M.
ECDSA key fingerprint is MD5:0f:31:c1:cc:54:69:7f:d8:8d:7c:8c:22:95:2e:03:4e.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
Last login: Sat Dec 15 11:14:27 2018
$ exit
登出
Connection to localhost closed.

安装Hadoop

解压文件

$ tar -zxvf hadoop-2.7.3.tar.gz

修改配置文件

$ vi /hadoop-2.7.3/etc/hadoop/hadoop-env.sh

配置Java路径

# The java implementation to use.
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64

修改HDFS配置

$ vi /hadoop-2.7.3/etc/hadoop/hdfs-site.xml

配置内容

<configuration>
	<property>
	 <name>dfs.replication</name>
	 <!-- 配置副本数量-->
	 <value>1</value>
	</property>
	<property>
	 <name>dfs.namenode.name.dir</name>
	 <value>file:/home/hadoop/hadoop_data/dfs/name</value>
	</property>
	<property>
	 <name>dfs.namenode.data.dir</name>
	 <value>file:/home/hadoop/hadoop_data/dfs/data</value>
	</property>
</configuration>

手动创建相关目录

$ mkdir -p /home/hadoop/hadoop_data/dfs/name
$ mkdir -p /home/hadoop/hadoop_data/dfs/data

配置core-site.xml，内容为

<configuration>
 <property>
 <name>fs.defaultFS</name>
 <!-- 配置主机IP-->
 <value>hdfs://0.0.0.0:8020</value>
 </property>

 <!-- 配置临时文件位置-->
 <property>
 <name>hadoop.tmp.dir</name>
 <value>file:/home/hadoop/hadoop_data</value>
 </property>
</configuration>

放行端口并重载

$ firewall-cmd --zone=public --add-port=8020/tcp --add-port=50070/tcp --permanent
$ firewall-cmd --reload

启动Hadoop

初始化

进入/hadoop-2.7.3/bin目录，使用如下语句初始化Hadoop，该操作仅执行一次即可，以后使用无需重复执行，否则执行后会删除当前hadoop上的所有文件

$ hadoop namenode -format

滚动均为info级别的日志并且status 0表示已初始化成功

...
18/12/15 17:07:32 INFO namenode.FSImage: Allocated new BlockPoolId: BP-99731851-127.0.0.1-1544864852133
18/12/15 17:07:32 INFO common.Storage: Storage directory /home/hadoop/hadoop_data/dfs/name has been successfully formatted.
18/12/15 17:07:32 INFO namenode.FSImageFormatProtobuf: Saving image file /home/hadoop/hadoop_data/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression
18/12/15 17:07:32 INFO namenode.FSImageFormatProtobuf: Image file /home/hadoop/hadoop_data/dfs/name/current/fsimage.ckpt_0000000000000000000 of size 353 bytes saved in 0 seconds.
18/12/15 17:07:32 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
18/12/15 17:07:32 INFO util.ExitUtil: Exiting with status 0
18/12/15 17:07:32 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at localhost/127.0.0.1
************************************************************/

启动HDFS

进入/hadoop-2.7.3/sbin目录输入./start-dfs.sh启动HDFS服务

$ ./start-dfs.sh
Starting namenodes on [0.0.0.0]
The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.
ECDSA key fingerprint is SHA256:3UEr3lx2jznmN3FL3SflViR05IZe6AWweb3TeSsfX0M.
ECDSA key fingerprint is MD5:0f:31:c1:cc:54:69:7f:d8:8d:7c:8c:22:95:2e:03:4e.
Are you sure you want to continue connecting (yes/no)? yes
0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts.
0.0.0.0: starting namenode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-namenode-localhost.localdomain.out
localhost: starting datanode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-datanode-localhost.localdomain.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-secondarynamenode-localhost.localdomain.out

进程验证，若出现如下四个进程表示HDFS服务启动成功

$ jps
28930 Jps
28821 SecondaryNameNode
28663 DataNode
28541 NameNode

配置hdfs命令

使用vi ~/.bash_profile进入配置文件，按如下配置

export HADOOP_HOME=/home/hadoop/hadoop-2.7.3
export PATH=$HADOOP_HOME/bin:$PATH

使配置文件生效source ~/.bash_profile 输入hdfs回车测试命令是否生效，若出现如下画面则表示命令已经生效

$ hdfs
Usage: hdfs [--config confdir] [--loglevel loglevel] COMMAND
 where COMMAND is one of:
 dfs run a filesystem command on the file systems supported in Hadoop.
 classpath prints the classpath
 namenode -format format the DFS filesystem
 secondarynamenode run the DFS secondary namenode
 namenode run the DFS namenode
 journalnode run the DFS journalnode
 zkfc run the ZK Failover Co
 ...

命令测试

$ hadoop fs -mkdir /test
$ hadoop fs -ls /
Found 1 items
drwxr-xr-x - hadoop supergroup 0 2018-12-15 17:14 /test

HBase伪分布式搭建

HBase安装

解压安装包，目录如下

$ ll
总用量 320
drwxr-xr-x. 4 hadoop hadoop 4096 1月 29 2016 bin
-rw-r--r--. 1 hadoop hadoop 122439 10月 26 2016 CHANGES.txt
drwxr-xr-x. 2 hadoop hadoop 178 1月 29 2016 conf
drwxr-xr-x. 12 hadoop hadoop 4096 10月 26 2016 docs
drwxr-xr-x. 7 hadoop hadoop 80 10月 26 2016 hbase-webapps
-rw-rw-r--. 1 hadoop hadoop 261 10月 26 2016 LEGAL
drwxrwxr-x. 3 hadoop hadoop 8192 12月 15 11:24 lib
-rw-rw-r--. 1 hadoop hadoop 130696 10月 26 2016 LICENSE.txt
-rw-rw-r--. 1 hadoop hadoop 42025 10月 26 2016 NOTICE.txt
-rw-r--r--. 1 hadoop hadoop 1477 12月 27 2015 README.txt

拷贝Hadoop配置文件到/conf目录

$ cp /home/hadoop/hadoop-2.7.3/etc/hadoop/hdfs-site.xml .
$ cp /home/hadoop/hadoop-2.7.3/etc/hadoop/core-site.xml .

修改hbase-env.sh文件，增加JDK路径

# The java implementation to use. Java 1.7+ required.
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64

由于使用的是jdk8，主要注释内存配置

# Configure PermSize. Only needed in JDK7. You can safely remove it for JDK8+
#export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS -XX:PermSize=128m -XX:MaxPermSize=128m"
#export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -XX:PermSize=128m -XX:MaxPermSize=128m"

另外打开文件最后zookeeper配置，设置为使用内置的zookeeper

# Tell HBase whether it should manage it's own instance of Zookeeper or not.
export HBASE_MANAGES_ZK=true

修改hbase-site.xml文件，内容如下

<configuration>
	<property>
		<name>hbase.rootdir</name>
 <!-- 配置文件根目录-->
		<value>hdfs://localhost:8020/hbase</value>
	</property>
	<property>
		<name>hbase.zookeeper.property.dataDir</name>
		<value>/home/hadoop/hadoop_data/zookeeper</value>
	</property>
	<property>
		<name>hbase.cluster.distributed</name>
 <!-- 配置以集群方式运行-->
		<value>true</value>
	</property>
</configuration>

参考文档

启动HBase

在/bin目录下执行./start-hbase.sh启动HBase

$ ./start-hbase.sh 
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
localhost: starting zookeeper, logging to /home/hadoop/hbase-1.2.4/bin/../logs/hbase-hadoop-zookeeper-localhost.localdomain.out
localhost: OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
starting master, logging to /home/hadoop/hbase-1.2.4/bin/../logs/hbase-hadoop-master-localhost.localdomain.out
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
starting regionserver, logging to /home/hadoop/hbase-1.2.4/bin/../logs/hbase-hadoop-1-regionserver-localhost.localdomain.out

由于虚拟机只分配了单核单线程，GC警告暂时忽略

进程验证，确认启动成功

$ jps
28821 SecondaryNameNode
28663 DataNode
29911 HRegionServer
29785 HMaster
29724 HQuorumPeer
28541 NameNode
30303 Jps

进入HBase命令行测试

$ ./hbase shell
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hbase-1.2.4/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 1.2.4, r67592f3d062743907f8c5ae00dbbe1ae4f69e5af, Tue Oct 25 18:10:20 CDT 2016

hbase(main):001:0> status
1 active master, 0 backup masters, 1 servers, 0 dead, 2.0000 average load

hbase(main):002:0>

查看其是否自动在HDFS上创建目录/hbase

$ hadoop fs -ls /
Found 2 items
drwxr-xr-x - hadoop supergroup 0 2018-12-15 18:43 /hbase
drwxr-xr-x - hadoop supergroup 0 2018-12-15 17:14 /test

Spring Data JPA 知识记录

Sun, 09 Dec 2018 17:04:00 +0800

jpa方法命名规则

关键字	方法命名	sql where字句
And	findByNameAndPwd	where name= ? and pwd =?
Or	findByNameOrSex	where name= ? or sex=?
Is,Equals	findById,findByIdEquals	where id= ?
Between	findByIdBetween	where id between ? and ?
LessThan	findByIdLessThan	where id < ?
LessThanEquals	findByIdLessThanEquals	where id <= ?
GreaterThan	findByIdGreaterThan	where id > ?
GreaterThanEquals	findByIdGreaterThanEquals	where id > = ?
After	findByIdAfter	where id > ?
Before	findByIdBefore	where id < ?
IsNull	findByNameIsNull	where name is null
isNotNull,NotNull	findByNameNotNull	where name is not null
Like	findByNameLike	where name like ?
NotLike	findByNameNotLike	where name not like ?
StartingWith	findByNameStartingWith	where name like ‘?%’
EndingWith	findByNameEndingWith	where name like ‘%?’
Containing	findByNameContaining	where name like ‘%?%’
OrderBy	findByIdOrderByXDesc	where id=? order by x desc
Not	findByNameNot	where name <> ?
In	findByIdIn(Collection c)	where id in (?)
NotIn	findByIdNotIn(Collection c)	where id not in (?)
TRUE	findByAaaTue	where aaa = true
FALSE	findByAaaFalse	where aaa = false
IgnoreCase	findByNameIgnoreCase	where UPPER(name)=UPPER(?)

常用注解

自动生成时间和修改者

@CreatedDate、@CreatedBy、@LastModifiedDate、@LastModifiedBy

在spring jpa中，支持在字段或者方法上进行注解@CreatedDate、@CreatedBy、@LastModifiedDate、@LastModifiedBy，从字面意思可以很清楚的了解，这几个注解的用处。

@CreatedDate 表示该字段为创建时间时间字段，在这个实体被insert的时候，会设置值

@CreatedBy 表示该字段为创建人，在这个实体被insert的时候，会设置值

@LastModifiedDate、@LastModifiedBy同理。

自动建表引擎设置

使用SpringDataJpa自动建表默认为MyISAM，而主流大多数场景都需要使用InnoDB，可通过配置文件配置即可

spring:
 jpa:
 database-platform: org.hibernate.dialect.MySQL55Dialect

注意：类名有所变化

早期通过网上搜索的类名都说为org.hibernate.dialect.MySQL5InnoDBDialect，当实际测试时发现引擎依然为MyISAM，通过进一步查询源码发现早期类名已经弃用

/*
 * Hibernate, Relational Persistence for Idiomatic Java
 *
 * License: GNU Lesser General Public License (LGPL), version 2.1 or later.
 * See the lgpl.txt file in the root directory or <http://www.gnu.org/licenses/lgpl-2.1.html>.
 */
package org.hibernate.dialect;

/** A Dialect for MySQL 5 using InnoDB engine
 *
 * @author Gavin King,
 * @author Scott Marlow
 * @deprecated Use "hibernate.dialect.storage_engine=innodb" environment variable or JVM system property instead.
 */
@Deprecated
public class MySQL5InnoDBDialect extends MySQL5Dialect {

	@Override
	protected MySQLStorageEngine getDefaultMySQLStorageEngine() {
		return InnoDBStorageEngine.INSTANCE;
	}
}

但注释里并没有指明新的替代类名，而是建议通过配置实现，但SpringDataJpa自动装配的类里没有支持该配置，故如果不单独引入hibernate依赖是无法使用该配置的

最后在stackoverflow上看到解决方案：where to put hibernate.dialect.storage_engine property

即使用新的类名org.hibernate.dialect.MySQL55Dialect替换即可，尽管两个类名初看没差别，实际上差了一个数字5，影响比较大

MySQL驱动变化

在使用了新版的驱动类后，不管是MyBatis还是Jpa，都会报时区不一致和报错信息乱码的错误需要将连接URL更新

新配置

spring:
 datasource:
 driver-class-name: com.mysql.cj.jdbc.Driver
 username: root
 password: 
 url: jdbc:mysql://localhost/pw_manager?useSSL=false&useUnicode=true&characterEncoding=UTF-8&serverTimezone=UTC

原配置

spring:
 datasource:
 driver-class-name: com.mysql.jdbc.Driver
 username: cmsspringcloud
 password: cmsdbadmin
 url: jdbc:mysql://localhost:3306/cms_springcloud?characterEncoding=utf-8&useSSL=false

redis多种启动配置方式

Sun, 09 Dec 2018 11:19:49 +0800

总览

三种启动方式：

动态参数启动（指定端口号）
配置文件启动（生产环境推荐）
最简单启动

官方安装指南

安装前置环境gcc-c++

$ yum install -y gcc-c++

官方下载安装文档

$ wget http://download.redis.io/releases/redis-5.0.0.tar.gz
$ tar xzf redis-5.0.0.tar.gz
$ cd redis-5.0.0
$ make MALLOC=libc
$ make install

编译后在src目录下生成可执行文件

$ ls | grep redis-

redis-benchmark ——性能测试
redis-benchmark.c
redis-benchmark.o
redis-check-aof ——aof修复
redis-check-aof.c
redis-check-aof.o
redis-check-rdb ——rdb修复
redis-check-rdb.c
redis-check-rdb.o
redis-cli ——客户端
redis-cli.c
redis-cli.o
redis-sentinel ——集群
redis-server ——服务器
redis-trib.rb

启动

简单启动

$ redis-server 
2660:C 21 Nov 2018 20:01:10.958 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2660:C 21 Nov 2018 20:01:10.958 # Redis version=5.0.0, bits=64, commit=00000000, modified=0, pid=2660, just started
2660:C 21 Nov 2018 20:01:10.958 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
2660:M 21 Nov 2018 20:01:10.959 * Increased maximum number of open files to 10032 (it was originally set to 1024).
 _._ 
 _.-``__ ''-._ 
 _.-`` `. `_. ''-._ Redis 5.0.0 (00000000/0) 64 bit
 .-`` .-```. ```\/ _.,_ ''-._ 
 ( ' , .-` | `, ) Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379
 | `-._ `._ / _.-' | PID: 2660
 `-._ `-._ `-./ _.-' _.-' 
 |`-._`-._ `-.__.-' _.-'_.-'| 
 | `-._`-._ _.-'_.-' | http://redis.io 
 `-._ `-._`-.__.-'_.-' _.-' 
 |`-._`-._ `-.__.-' _.-'_.-'| 
 | `-._`-._ _.-'_.-' | 
 `-._ `-._`-.__.-'_.-' _.-' 
 `-._ `-.__.-' _.-' 
 `-._ _.-' 
 `-.__.-' 

2660:M 21 Nov 2018 20:01:10.961 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
2660:M 21 Nov 2018 20:01:10.961 # Server initialized
2660:M 21 Nov 2018 20:01:10.961 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
2660:M 21 Nov 2018 20:01:10.961 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
2660:M 21 Nov 2018 20:01:10.961 * Ready to accept connections

客户端连接测试

$ redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> ping
PONG
127.0.0.1:6379> set hello world
OK
127.0.0.1:6379> get hello
"world"

动态参数启动

$ redis-server --port 6380

连接默认端口测试

$ redis-cli -h 127.0.0.1 -p 6379
Could not connect to Redis at 127.0.0.1:6379: Connection refused
not connected> exit

连接新端口测试

$ redis-cli -h 127.0.0.1 -p 6380
127.0.0.1:6380> ping
PONG
127.0.0.1:6380> exit

通过进程查看

$ ps -ef | grep redis-server |grep -v grep
root 2666 1469 0 20:04 pts/0 00:00:00 redis-server *:6380

配置文件启动

拷贝默认配置

$ pwd
/root/redis-5.0.0
$ mkdir config
$ cp redis.conf config/
$ cd config/
$ ls
redis.conf

查看默认配置，以下命令用于去除注释和空格

$ cat redis.conf | grep -v "#" | grep -v "^$" > redis-default.conf
$ cat redis-default.conf 
bind 127.0.0.1
protected-mode yes
port 6379 #端口号
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize no #默认非守护进程启动
supervised no
pidfile /var/run/redis_6379.pid
loglevel notice
logfile "" #日志文件名称
databases 16
always-show-logo yes
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir ./ #工作目录
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
replica-priority 100
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
stream-node-max-bytes 4096
stream-node-max-entries 100
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
dynamic-hz yes
aof-rewrite-incremental-fsync yes
rdb-save-incremental-fsync yes

由于默认配置redis为单线程，故一台服务器一般起多个redis实例，配置文件以端口号区分

$ cp redis-default.conf redis-6379.conf
$ ll
总用量 72
-rw-r--r--. 1 root root 65 11月 21 20:46 redis-6379.conf
-rw-r--r--. 1 root root 62155 11月 21 20:36 redis.conf
-rw-r--r--. 1 root root 1425 11月 21 20:41 redis-default.conf

简化配置如下：

port 6380
daemonize yes
logfile "6380.log"
dir "/root/tmp/redis"

配置文件启动

 $ redis-server config/redis-6380.conf

Mesos集群安装

Wed, 03 Oct 2018 22:10:00 +0800

基础环境

4台服务器，一台外部客户机

安装指南

官方支持C++源码编译安装和Docker安装

C++源码编译安装地址

Docker镜像仓库

停止所有容器命令

docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)

安装

mesos主节点

node1

docker pull mesosphere/mesos-master:1.7.0

node2(host)

docker run --name mesos-zookeeper -p 2181:2181 --restart always -d zookeeper:3.5

node1执行脚本(需替换zk IP地址)

MESOS_QUORUM=1即1为单机运行，3个节点为2，计算方式为 n÷2+1 \

若不手动指定hostname映射的IP，则所有节点默认通过主机名进行通信

使用host通信脚本

docker run -d --net=host \
 -e MESOS_PORT=5050 \
 -e MESOS_ZK=zk://192.168.0.32:2181/mesos \
 -e MESOS_QUORUM=1 \
 -e MESOS_REGISTRY=in_memory \
 -e MESOS_LOG_DIR=/var/log/mesos \
 -e MESOS_WORK_DIR=/var/tmp/mesos \
 -v "$(pwd)/log/mesos:/var/log/mesos" \
 -v "$(pwd)/work/mesos:/var/tmp/mesos" \
 -p 5050:5050 \
 --name=mesos-master \
 mesosphere/mesos-master:1.7.0 \

使用IP通信脚本

docker run -d --net=host \
 --hostname=192.168.0.32 \
 -e MESOS_PORT=5050 \
 -e MESOS_ZK=zk://192.168.0.32:2181/mesos \
 -e MESOS_QUORUM=1 \
 -e MESOS_REGISTRY=in_memory \
 -e MESOS_LOG_DIR=/var/log/mesos \
 -e MESOS_WORK_DIR=/var/tmp/mesos \
 -v "$(pwd)/log/mesos:/var/log/mesos" \
 -v "$(pwd)/work/mesos:/var/tmp/mesos" \
 -p 5050:5050 \
 --name=mesos-master \
 mesosphere/mesos-master:1.7.0 \
 --no-hostname_lookup \
 --ip=192.168.0.33

查看日志

docker logs mesos-master

成功输出日志

I1003 08:41:29.868827 14 group.cpp:341] Group process (zookeeper-group(2)@192.168.0.33:5050) connected to ZooKeeper
I1003 08:41:29.868902 14 group.cpp:831] Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0)
I1003 08:41:29.868921 14 group.cpp:419] Trying to create path '/mesos' in ZooKeeper
I1003 08:41:29.902194 14 contender.cpp:268] New candidate (id='1') has entered the contest for leadership
I1003 08:41:29.904409 8 detector.cpp:152] Detected a new leader: (id='1')
I1003 08:41:29.904513 8 group.cpp:700] Trying to get '/mesos/json.info_0000000001' in ZooKeeper
I1003 08:41:29.912968 8 zookeeper.cpp:262] A new leading master (UPID=master@192.168.0.33:5050) is detected
I1003 08:41:29.913091 8 master.cpp:2113] Elected as the leading master!
I1003 08:41:29.913115 8 master.cpp:1668] Recovering from registrar
I1003 08:41:29.916913 11 registrar.cpp:383] Successfully fetched the registry (0B) in 3.734016ms
I1003 08:41:29.917054 11 registrar.cpp:487] Applied 1 operations in 26us; attempting to update the registry
I1003 08:41:29.918582 11 registrar.cpp:544] Successfully updated the registry in 1.449984ms
I1003 08:41:29.918630 11 registrar.cpp:416] Successfully recovered registrar
I1003 08:41:29.918772 8 master.cpp:1782] Recovered 0 agents from the registry (121B); allowing 10mins for agents to reregister

访问IP:5050，对于PWD，可以访问http://SSHIP-5050.direct.labs.play-with-docker.com/#/，将SSH ip替换成实际提供的IP即可

mesos从节点

node3 (slave-1)、node4(slave-2)

启动脚本

使用host通信脚本

mkdir /cgroup
docker run -d --net=host --privileged \
 -e MESOS_PORT=5051 \
 -e MESOS_MASTER=zk://192.168.0.32:2181/mesos \
 -e MESOS_SWITCH_USER=0 \
 -e MESOS_CONTAINERIZERS=docker,mesos \
 -e MESOS_LOG_DIR=/var/log/mesos \
 -e MESOS_WORK_DIR=/var/tmp/mesos \
 -v "$(pwd)/log/mesos:/var/log/mesos" \
 -v "$(pwd)/tmp/mesos:/var/tmp/mesos" \
 -v /var/run/docker.sock:/var/run/docker.sock \
 -v /cgroup:/cgroup \
 -v /sys:/sys \
 -v "$(which docker):/usr/local/bin/docker" \
 --name=mesos-slave \
 -p 5051:5051 \
 mesosphere/mesos-slave:1.7.0 \
 --no-systemd_enable_support

使用IP通信脚本，--ip需要改成实际IP

mkdir /cgroup
docker run -d --net=host --privileged \
 -e MESOS_PORT=5051 \
 -e MESOS_MASTER=zk://192.168.0.32:2181/mesos \
 -e MESOS_SWITCH_USER=0 \
 -e MESOS_CONTAINERIZERS=docker,mesos \
 -e MESOS_LOG_DIR=/var/log/mesos \
 -e MESOS_WORK_DIR=/var/tmp/mesos \
 -v "$(pwd)/log/mesos:/var/log/mesos" \
 -v "$(pwd)/tmp/mesos:/var/tmp/mesos" \
 -v /var/run/docker.sock:/var/run/docker.sock \
 -v /cgroup:/cgroup \
 -v /sys:/sys \
 -v "$(which docker):/usr/local/bin/docker" \
 --name=mesos-slave \
 -p 5051:5051 \
 mesosphere/mesos-slave:1.7.0 \
 --no-systemd_enable_support \
 --no-hostname_lookup \
 --ip=192.168.0.31

查看日志

docker logs mesos-slave

启动成功日志

I1003 09:05:08.615039 12 group.cpp:341] Group process (zookeeper-group(1)@192.168.0.31:5051) connected to ZooKeeper
I1003 09:05:08.615104 12 group.cpp:831] Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0)
I1003 09:05:08.615121 12 group.cpp:419] Trying to create path '/mesos' in ZooKeeper
I1003 09:05:08.618291 13 state.cpp:66] Recovering state from '/var/tmp/mesos/meta'
I1003 09:05:08.618405 15 slave.cpp:6930] Finished recovering checkpointed state from '/var/tmp/mesos/meta', beginning agent recovery
I1003 09:05:08.618475 8 task_status_update_manager.cpp:207] Recovering task status update manager
I1003 09:05:08.618628 8 docker.cpp:908] Recovering Docker containers
I1003 09:05:08.626359 13 detector.cpp:152] Detected a new leader: (id='3')
I1003 09:05:08.626487 15 group.cpp:700] Trying to get '/mesos/json.info_0000000003' in ZooKeeper
I1003 09:05:08.628664 15 zookeeper.cpp:262] A new leading master (UPID=master@192.168.0.33:5050) is detected
I1003 09:05:08.997036 15 docker.cpp:921] Got the list of Docker containers
I1003 09:05:08.997324 15 docker.cpp:1157] Finished processing orphaned Docker containers
I1003 09:05:08.997436 8 slave.cpp:7159] Recovering executors
I1003 09:05:08.997553 8 slave.cpp:7312] Finished recovery
I1003 09:05:08.998044 11 task_status_update_manager.cpp:181] Pausing sending task status updates
I1003 09:05:08.998045 8 slave.cpp:1275] New master detected at master@192.168.0.33:5050
I1003 09:05:08.998159 8 slave.cpp:1329] No credentials provided. Attempting to register without authentication
I1003 09:05:08.998181 8 slave.cpp:1340] Detecting new master
I1003 09:05:09.446756 11 slave.cpp:1500] Registered with master master@192.168.0.33:5050; given agent ID b597695d-6e37-418a-9879-4ffe46117e17-S0
I1003 09:05:09.449091 10 task_status_update_manager.cpp:188] Resuming sending task status updates

访问主节点web界面可看见资源邀约有效

节点情况

marathon安装

node5

启动脚本

docker run -d --net=host \
 -p 8080:8080 \
 mesosphere/marathon:latest \
 --master zk://192.168.0.32:2181/mesos \
 --zk zk://192.168.0.32:2181/marathon

通过IP:8080查看运行结果

marathon-lb安装

node6

启动脚本

docker run -d -e PORTS=9090 --net=host \
 -p 9090:9090 \
 mesosphere/marathon-lb sse \
 --group external \
 --marathon http://192.168.0.29:8080

访问IP:9090/haproxy?stats可查看启动状态

Jenkins+GitLab+Ansible实现静态网站自动部署

Sat, 01 Sep 2018 12:45:00 +0800

环境准备

参考前几篇文章配置后实现以下环境

部署好Jenkins服务器一台(192.168.163.133),并在Jenkins服务器上部署ansible，且与目标主机(192.168.163.132)做了免密登录
可以访问的GitLab服务器一台，并注册了Jenkins相关账号且在Jenkins系统设置中配置完毕
一台工作环境主机，用于代码编写，该主机可以通过域名访问目标主机(192.168.163.132)

编写ansible playbook

搭建基础工程目录结构

从仓库下载之前编写的ansible工程目录结构

若没有也可自行创建，最终文件目录结构如下

└─ansible-playbook-repo
 │ deploy.yml
 │
 ├─inventory
 │ dev
 │ prod
 │
 └─roles
 └─nginx
 ├─files
 │ │ health_check.sh
 │ │ index.html
 │ │
 │ └─website
 │
 ├─tasks
 │ main.yml
 │
 └─templates
 nginx.conf.j2

文件编写

工程入口deploy.yml编写

- hosts: "nginx"
 gather_facts: true
 remote_user: root
 roles:
 - nginx

编写开发环境配置文件inventory/dev，此处dev与prod配置一致，若实际需求可自行修改，用于最后jenkins参数化构建的选项 inventory/dev文件内容

[nginx]
test.fjy8018.top

[nginx:vars]
server_name=test.fjy8018.top
port=80
user=ansible
worker_processes=4
max_open_file=65505
root=/www

inventory/prod文件内容

[nginx]
test.fjy8018.top

[nginx:vars]
server_name=test.fjy8018.top
port=80
user=ansible
worker_processes=4
max_open_file=65505
root=/www

编写健康检查shell脚本health_check.sh roles/nginx/files/health_check.sh文件内容

#!/bin/bash

# 接受命令行参数
URL=$1

# 访问目标地址，判断是否能正常访问并输出
curl -Is http://$URL > /dev/null && echo "远程主机状态正常" || echo "远程主机已停止"

添加静态网站文件

/roles/nginx/files/website为静态网页模板目录，里面存放从网上下载的静态网站模板配置nginx指向该资源 /roles/nginx/templates/nginx.conf.j2文件内容

# For more information on configuration, see: 
user {{ user }}; 
worker_processes {{ worker_processes }}; 
 
error_log /var/log/nginx/error.log; 
 
pid /var/run/nginx.pid; 
 
events { 
 worker_connections {{ max_open_file }}; 
} 
 
 
http { 
 include /etc/nginx/mime.types; 
 default_type application/octet-stream; 
 
 log_format main '$remote_addr - $remote_user [$time_local] "$request" ' 
 '$status $body_bytes_sent "$http_referer" ' 
 '"$http_user_agent" "$http_x_forwarded_for"'; 
 
 access_log /var/log/nginx/access.log main; 
 
 sendfile on; 
 #tcp_nopush on; 
 
 #keepalive_timeout 0; 
 keepalive_timeout 65; 
 
 #gzip on; 
 
 # Load config files from the /etc/nginx/conf.d directory 
 # The default server is in conf.d/default.conf 
 #include /etc/nginx/conf.d/*.conf; 
 server { 
 listen {{ port }} default_server; 
 server_name {{ server_name }}; 
 
 #charset koi8-r; 
 
 #access_log logs/host.access.log main; 
 
 location / { 
 root {{ root }}/website; 
 index index.html index.htm; 
 } 
 
 error_page 404 /404.html; 
 location = /404.html { 
 root /usr/share/nginx/html; 
 } 
 
 # redirect server error pages to the static page /50x.html 
 # 
 error_page 500 502 503 504 /50x.html; 
 location = /50x.html { 
 root /usr/share/nginx/html; 
 } 
 
 } 
 
}

编写主任务文件

主任务文件main.yml内容如下

- name: 关闭系统防火墙
 service: name=firewalld state=stopped

- name: 关闭强制访问控制策略
 selinux: state=disabled

- name: 设置yum源
 yum: pkg=epel-release state=latest

- name: 写入nginx配置文件
 template: src=roles/nginx/templates/nginx.conf.j2 dest=/etc/nginx/nginx.conf

- name: 创建nginx根目录文件夹
 file: 'path={{ root }} state=directory owner={{ user }} group={{ user }} mode=0755'

- name: 文件拷贝
 copy: 'remote_src=no src=roles/nginx/files/website dest=/www/website owner={{ user }} mode=0755'

- name: 启动nginx
 service: name=nginx state=restarted

- name: 运行健康检查
 shell: "sh roles/nginx/files/health_check.sh {{ server_name }}"
 # 本地执行
 delegate_to: localhost
 # 参数传递
 register: health_status

- debug: msg="{{ health_status.stdout }}"

Git提交

提交变更到版本仓库GitLab

git add .
git commit -m "添加模板文件"
git push origin master

Jenkins创建并执行

创建自由风格构建任务，添加参数化构建、添加选项参数和文本参数

添加Git仓库地址和账号

添加Shell构建，Shell内容为

#!/bin/bash
# 关闭命令行扩展环境，避免环境参数日志干扰
set +x
# 加载虚拟环境
source /home/deploy/.py3-a2.5-env/bin/activate
# 加载ansible到当前用户目录
source /home/deploy/.py3-a2.5-env/ansible/hacking/env-setup -q

cd $WORKSPACE
ls -a
# 查看版本
ansible --version
ansible-playbook --version

# 引入资源清单，测试远程命令
ansible-playbook -i inventory/$deploy_env ./deploy.yml -e project=nginx -e branch=$branch -e env=$deploy_env

保存并构建

结果

构建成功

访问测试

在工作主机上访问目标地址+资源路径即可，访问通过

后续构建

后期只要在工作主机变更代码推送到git仓库，再到Jenkins的web界面点击参数化构建即可让Jenkins自动拉取最新分支并执行Ansible脚本

Jenkins 与Maven集成

Sun, 26 Aug 2018 17:15:00 +0800

环境准备

安装jdk

yum -y install java-devel java

检测安装结果

[root@localhost maven-freestyle-job]# javac -version
javac 1.8.0_181
[root@localhost maven-freestyle-job]# java -version
openjdk version "1.8.0_181"
OpenJDK Runtime Environment (build 1.8.0_181-b13)
OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)

安装maven环境

wget http://apache.osuosl.org/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz

解压，并指定解压路径

tar -zxvf apache-maven-3.5.4-bin.tar.gz -C /opt

查看安装结果

[root@localhost ~]# cd /opt/
[root@localhost opt]# cd apache-maven-3.5.4/
[root@localhost apache-maven-3.5.4]# cd bin/
[root@localhost bin]# ./mvn -v
Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-18T02:33:14+08:00)
Maven home: /opt/apache-maven-3.5.4
Java version: 1.8.0_181, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181-3.b13.el7_5.x86_64/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-862.el7.x86_64", arch: "amd64", family: "unix"
[root@localhost bin]# pwd
/opt/apache-maven-3.5.4/bin

配置环境

进入Jenkins系统管理中的全局工具配置，配置本地的maven地址

此处的jdk报错正常，若不配置则会自动调用系统环境里的jdk

指定maven名称和路径

准备maven工程

提前在gitlab中装备一个简单的Springboot web工程，通过springboot脚手架快速生成，pom文件如下

新建构建

新建构建并填写目标仓库地址

添加构建设置为maven构建

添加构建参数，即在命令行mvn之后的参数

选择maven版本

构建

构建成功

Jenkins job编写测试

Sun, 26 Aug 2018 12:00:00 +0800

系统预配置

关闭防止跨站请求伪造

测试期间系统设置中临时关闭防止跨站请求伪造，否则容易出现登录超时无法新建任务

添加远程仓库配置

GitLab的用户邮箱、用户名、密码，用于拉取远程仓库代码

凭据

添加凭据，凭据为用户的标识

选择全局凭据

添加凭据

添加为Git用户

Jenkins job

freestyle Job

freestyle Job简介

需要在页面添加模块配置项与参数完成配置
每个Job仅能实现一个开发功能
无法将配置代码化，不利于Job配置迁移与版本控制
没有版本日志记录
逻辑简单

新建freestyle Job

新建任务

添加选项参数

添加文本参数

指定远程仓库地址

添加Shell脚本

保存后选择通过参数构建

构建报错

检查后发现Shell脚本开头的#!/bin/bash写成了#! /bin/bash，中间包含空格后系统无法识别

修正错误后重新构建

报语法错误，12行缺少空格

检查发现if [ -s test.properties ]写成了if [ -s test.properties]，修正错误后再次构建修正后的Shell脚本为

#!/bin/bash
# export PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"

# 打印环境变量
echo "[INFO] 打印环境变量"
echo "[INFO] 当前发布环境为 $deploy_env" >> test.properties
echo "[INFO] 构建版本为 $version " >> test.properties
echo "[INFO] 打印环境变量完成..."

# 检查test.properties
echo "[INFO] 检查test.properties"
if [ -s test.properties ]
then
 cat test.properties
 echo "[INFO] 检查完成..."
else
 echo "[WARN] test.properties为空"
fi

echo "[INFO] 构建完成..."

构建成功，输出正常

查看构建历史

同时显示构建稳定性

pipeline job

pipeline job简介

所有代码包裹在pipeline{}内
stages{}用来包含pipeline所有子层
stage(管道名称){}每个子层可以执行一个任务，并不互相影响
steps{}用来添加具体需要调用的模块语句

新建pipeline任务

编写pipeline 脚本

#!groovy

pipeline {
 // agent 定义pipeline在何处运行，使用any、none或者Jenkins node主机名
	agent {node {label 'master'}}
	// 声明环境变量，environment 定义系统环境变量，可以应用于所有stages
	// 定义局部环境变量则于steps平级即可
	environment {
		PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"
	}

	parameters {
		// 声明可选参数
		choice(
				// 可选参数为两个，以空行分隔
				choices: 'dev\nprod',
				// 设置参数描述
				description: '选择发布环境',
				// 声明参数变量名
				name: 'deploy_env'
			)
		// 声明文本参数，和默认值
		string (name: 'version', defaultValue: '0.0.1',description: '构建版本号')
	}

	stages {
		// 执行一个任务
		stage("切换到测试版本仓库") {
			// 添加具体需要调用的模块语句
			steps {
				// 关闭Git ssl安全认证
				sh 'git config --system http.sslVerify false'
				// 指定工作空间
				dir ("${env.WORKSPACE}") {
					// 指定分支和Jenkins凭据ID
					git branch: 'master',
						credentialsId: '52f4c34f-544c-4830-828d-6691d603b8e4',
						url: 'http://home.fjy8018.top:8099/jenkins/test-repo.git'
				}
			}
		}
		stage("打印环境变量") {
			steps {
				dir ("${env.WORKSPACE}") {
					sh """
						echo "[INFO] 打印环境变量"
						echo "[INFO] 当前发布环境为 $deploy_env" >> test.properties
						echo "[INFO] 构建版本为 $version " >> test.properties
						echo "[INFO] 打印环境变量完成..."
					"""
				}
			}
		}
		stage("检查test.properties") {
			steps {
				dir ("${env.WORKSPACE}") {
					sh """
						echo "[INFO] 检查test.properties"
						if [ -s test.properties ]
						then
						 cat test.properties
						 echo "[INFO] 检查完成..."
						else
						 echo "[WARN] test.properties为空"
						fi
					"""
					echo "[INFO] 构建完成..."
				}
			}
		}
	}
}

其中credentialsId来自用户的凭据唯一标识，可在凭据页面查看

写入Shell脚本

执行报错，error: could not lock config file /etc/gitconfig: ????

由于使用root用户执行同样命令不会报错，故猜测没有权限，修改语法为git config --global http.sslVerify false即可重新构建

日志符合预期

freestyle shell Job

新建任务

写入shell脚本如下

#!/bin/bash

user=`whoami`

if [ $user == 'deploy' ]; then
	echo "成功，当前用户为 $user"
else
	echo "失败，当前用户不是 $user"
fi
# 查看主机ip
ip addr
# 查看系统版本
cat /etc/system-release
# 查看内存状态
free -m
# 查看磁盘状态
df -h
# 将python安装位置赋值给变量
py_cmd=`which python`
# 查看python版本号
$py_cmd --version

运行并查看日志，符合预期

参数任务

新建参数化构建任务

添加选项参数等

添加shell构建参数

构建

输出符合预期

Jenkins 安装配置

Sun, 26 Aug 2018 11:32:00 +0800

使用Docker安装

基础环境

docker环境查看docker版本

[root@localhost ~]# docker -v
Docker version 18.06.0-ce, build 0ffa825

新建用户目录，用户挂载容器存储

useradd jenkins-docker

运行docker命令

docker run -p 8081:8080 -p 50001:50000 -v /home/jenkins-docker:/var/jenkins_home hub.c.163.com/library/jenkins:latest

命令中没有使用后台启动为了查看输出的管理员密码

访问web界面，输入密码即可

使用原生系统安装

基础环境

CentOS7 1804最小安装

安装基础wget工具yum install -y wget

导入仓库源

wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins.io/redhat-stable/jenkins.repo

下载并导入仓库的key，用于验证仓库安全性

rpm --import http://pkg.jenkins.io/redhat-stable/jenkins.io.key

安装java8

yum -y install java

验证java版本

[root@localhost ~]# java -version
openjdk version "1.8.0_181"
OpenJDK Runtime Environment (build 1.8.0_181-b13)
OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)

关闭防火墙

[root@localhost ~]# systemctl stop firewalld

关闭防火墙开机启动

[root@localhost ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

关闭强制访问控制安全策略

vi /etc/sysconfig/selinux

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of three two values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted

SELINUX=enforcing

改成

SELINUX=disabled

reboot使禁用操作生效

查看操作是否生效

[root@localhost ~]# getenforce
Disabled

安装jenkins

直接安装

yum -y install jenkins

安装后会自动添加jenkins，此时可以自定义发布用户

[root@localhost ~]# useradd jenkins
useradd：用户“jenkins”已存在
[root@localhost ~]# useradd deploy

配置

编辑配置文件

[root@localhost ~]# vi /etc/sysconfig/jenkins

将用户改为新建的用户JENKINS_USER="deploy"，端口号默认为8080

修改log目录和权限

[root@localhost ~]# chown -R deploy:deploy /var/lib/jenkins
[root@localhost ~]# chown -R deploy:deploy /var/log/jenkins
[root@localhost ~]# chown -R deploy:deploy /var/cache/jenkins

启动jenkins

[root@localhost ~]# systemctl start jenkins

查看服务是否启动

[root@localhost ~]# ps -ef | grep jenkins
root 1445 1311 0 19:06 pts/0 00:00:00 grep --color=auto jenkins

或者使用lsof（需要额外安装）

[root@localhost jenkins]# lsof -i:8080
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 1683 deploy 161u IPv6 24353 0t0 TCP *:webcache (LISTEN)

访问web界面

地址为ip:8080

去提示的目录下查找密码

安装推荐配置插件

新建管理员用户

安装成功

Ansible Playbook编写测试

Sat, 25 Aug 2018 16:19:00 +0800

环境准备

两台CentOS 7虚拟机，主IP为192.168.163.128，并在主IP虚拟机上安装了ansible2.5版本，目标IP为192.168.163.132

Ansible编写

切换用户

[root@localhost ~]# su - ansible
上一次登录：六 8月 18 20:12:23 CST 2018pts/0 上

进入虚拟环境

[ansible@localhost ~]$ source .py3-a2.5-env/bin/activate
(.py3-a2.5-env) [ansible@localhost ~]$

加载ansible到当前用户目录

source .py3-a2.5-env/ansible/hacking/env-setup -q

验证版本

(.py3-a2.5-env) [ansible@localhost ~]$ ansible-playbook --version
ansible-playbook 2.5.8 (stable-2.5 b23b1c2ee0) last updated 2018/08/18 20:35:45 (GMT +800)
 config file = None
 configured module search path = ['/home/ansible/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
 ansible python module location = /home/ansible/.py3-a2.5-env/ansible/lib/ansible
 executable location = /home/ansible/.py3-a2.5-env/ansible/bin/ansible-playbook
 python version = 3.7.0 (default, Aug 18 2018, 20:09:08) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
(.py3-a2.5-env) [ansible@localhost ~]$ ll
总用量 0
(.py3-a2.5-env) [ansible@localhost ~]$ pwd
/home/ansible

配置目标主机的秘钥认证

(.py3-a2.5-env) [ansible@localhost ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/ansible/.ssh/id_rsa): 
Created directory '/home/ansible/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/ansible/.ssh/id_rsa.
Your public key has been saved in /home/ansible/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:pXMJqz9DPkjt6nKws35NYoMQeUdfu4CGATw76tr9lJY ansible@localhost.localdomain
The key's randomart image is:
+---[RSA 2048]----+
| ..o... . |
| = .o.o . . |
| =..o + o |
| + . * o |
| . o . .S + |
|. o =++o |
|. =EX |
| o .+o+.B |
|o ..+O+..+ |
+----[SHA256]-----+

拷贝公钥

(.py3-a2.5-env) [ansible@localhost ~]$ ssh-copy-id -i /home/ansible/.ssh/id_rsa.pub root@test.fjy8018.top
/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/ansible/.ssh/id_rsa.pub"
The authenticity of host 'test.fjy8018.top (192.168.163.132)' can't be established.
ECDSA key fingerprint is SHA256:3UEr3lx2jznmN3FL3SflViR05IZe6AWweb3TeSsfX0M.
ECDSA key fingerprint is MD5:0f:31:c1:cc:54:69:7f:d8:8d:7c:8c:22:95:2e:03:4e.
Are you sure you want to continue connecting (yes/no)? yes
/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@test.fjy8018.top's password: 

Number of key(s) added: 1

Now try logging into the machine, with: "ssh 'root@test.fjy8018.top'"
and check to make sure that only the key(s) you wanted were added.

测试免密登录

(.py3-a2.5-env) [ansible@localhost ~]$ ssh root@test.fjy8018.top
Last login: Sat Aug 25 11:27:35 2018 from 192.168.163.1
[root@localhost ~]# exit
登出
Connection to test.fjy8018.top closed.

编写环境变量

预期的文件结构为

├── deploy.yml
├── inventory
│ └── testenv
└── roles
 └── testbox
 └── tasks
 └── main.yml

deploy.yml为工程入口，testenv为环境变量文件，tasks为任务文件夹，其中的main.yml为要执行的任务新建playbook工程目录，并根据ansible规范创建对应的文件夹

(.py3-a2.5-env) [ansible@localhost ~]$ mkdir test_playbook
(.py3-a2.5-env) [ansible@localhost ~]$ cd test_playbook/
(.py3-a2.5-env) [ansible@localhost test_playbook]$ mkdir inventory
(.py3-a2.5-env) [ansible@localhost test_playbook]$ mkdir roles
(.py3-a2.5-env) [ansible@localhost test_playbook]$ cd inventory/
(.py3-a2.5-env) [ansible@localhost inventory]$ vi testenv

testenv文件内容如下

[testservers]
test.fjy8018.top

[testservers:vars]
server_name=test.fjy8018.top
user=root
output=/root/test.txt

远程执行命令并加载环境变量的值

(.py3-a2.5-env) [ansible@localhost test_playbook]$ ll
总用量 0
drwxrwxr-x 2 ansible ansible 21 8月 18 20:49 inventory
drwxrwxr-x 2 ansible ansible 6 8月 18 20:47 roles
(.py3-a2.5-env) [ansible@localhost test_playbook]$ cd roles/
(.py3-a2.5-env) [ansible@localhost roles]$ mkdir testbox
(.py3-a2.5-env) [ansible@localhost roles]$ cd testbox/
(.py3-a2.5-env) [ansible@localhost testbox]$ mkdir tasks
(.py3-a2.5-env) [ansible@localhost testbox]$ cd tasks/
(.py3-a2.5-env) [ansible@localhost tasks]$ vi main.yml

main.yml文件内容如下

- name: Print server name and user to remote testbox
 shell: "echo 'Currently {{ user }} is loggining {{ server_name }} ' > {{ output }} "

编写工程入口文件

(.py3-a2.5-env) [ansible@localhost test_playbook]$ pwd
/home/ansible/test_playbook
(.py3-a2.5-env) [ansible@localhost test_playbook]$ ll
总用量 0
drwxrwxr-x 2 ansible ansible 21 8月 18 20:49 inventory
drwxrwxr-x 3 ansible ansible 21 8月 18 20:51 roles
(.py3-a2.5-env) [ansible@localhost test_playbook]$ vi deploy.yml

deploy.yml文件内容如下

# 调用testenv里的servers标签定义的目标主机
- hosts: "testservers"
# 用于获取目标主机的信息
 gather_facts: true
 # 使用root用户操作
 remote_user: root
 # 指定进入roles目录下的testbox任务文件执行任务
 roles:
 - testbox

查看当前目录结构，符合预期

[ansible@localhost test_playbook]$ tree .
.
├── deploy.yml
├── inventory
│ └── testenv
└── roles
 └── testbox
 └── tasks
 └── main.yml

4 directories, 3 files

执行任务

(.py3-a2.5-env) [ansible@localhost ~]$ cd test_playbook/
(.py3-a2.5-env) [ansible@localhost test_playbook]$ ls
deploy.yml inventory roles
(.py3-a2.5-env) [ansible@localhost test_playbook]$ ansible-playbook -i inventory/testenv ./deploy.yml

部署完成

登录目标主机查看

远程文件创建

编辑任务文件

(.py3-a2.5-env) [ansible@localhost test_playbook]$ vi roles/testbox/tasks/main.yml

修改为在远程创建文件，指定路径并赋予权限为755，所有者为foo用户，属于foo用户组，故添加如下内容

- name: create a file
 file: 'path=/root/foo.txt state=touch mode=0755 owner=foo group=foo'

执行文件

执行成功，登录目标主机查看是否创建

远程文件拷贝

新建存放文件目录

(.py3-a2.5-env) [ansible@localhost test_playbook]$ mkdir roles/testbox/files
(.py3-a2.5-env) [ansible@localhost test_playbook]$ tree .
.
├── deploy.yml
├── inventory
│ └── testenv
└── roles
 └── testbox
 ├── files
 └── tasks
 └── main.yml

5 directories, 3 files

新建目标文件foo.sh

(.py3-a2.5-env) [ansible@localhost test_playbook]$ vi roles/testbox/files/foo.sh

文件内容如下

echo "This is a test script for ansible"

编辑任务文件

(.py3-a2.5-env) [ansible@localhost test_playbook]$ vi roles/testbox/tasks/main.yml

remote_src=no定义为文件传输，并设置644权限，指定强制执行，故添加任务如下

- name: copy a file
 copy: 'remote_src=no src=roles/testbox/files/foo.sh dest=/root/foo.sh mode=0644 force=yes'

执行成功

判断文件是否存在

在roles/testbox/tasks/main.yml添加任务如下

# 任务名称
- name: check if foo.sh exists
 stat: 'path=/root/foo.sh'
 register: script_stat
 # 若when中返回信息，则输出msg后的值
- debug: msg="foo.sh exists"
 # 调用模块任务
 when: script_stat.stat.exists

执行，成功输出信息

远程执行命令

在roles/testbox/tasks/main.yml添加任务如下

- name: run the script
 command: 'sh /root/foo.sh'

执行成功

template模块任务

环境变量配置文件中inventory/testenv添加如下命令

server_name=test.fjy8018.top
port=80
user=ansible
worker_processes=4
max_open_file=65505
root=/www

新建template文件夹

(.py3-a2.5-env) [ansible@localhost test_playbook]$ mkdir roles/testbox/templates

添加nginx j2文件

(.py3-a2.5-env) [ansible@localhost test_playbook]$ vi roles/testbox/templates/nginx.conf.j2

文件内容如下，其中调用了环境变量中的配置

# For more information on configuration, see: 
user {{ user }}; 
worker_processes {{ worker_processes }}; 
 
error_log /var/log/nginx/error.log; 
 
pid /var/run/nginx.pid; 
 
events { 
 worker_connections {{ max_open_file }}; 
} 
...

在roles/testbox/tasks/main.yml添加任务如下

# 复制nginx配置文件
- name: write the nginx config file
 template: src=roles/testbox/templates/nginx.conf.j2 dest=/etc/nginx/nginx.conf
# 远程安装最新版nginx
- name: install nginx at latest version
 yum: pkg=nginx state=latest
# 新增模块任务启动nginx服务
- name: start nginx service
 service: name=nginx state=started

所有任务列表

查看变量参数是否应用成功

ssh root@test.fjy8018.top ps -ef | grep nginx查看nginx是否启动成功

gitlab-ce安装配置

Sun, 19 Aug 2018 16:39:00 +0800

系统预配置

关闭防火墙

[root@localhost ~]# systemctl stop firewalld

关闭防火墙开机启动

[root@localhost ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

关闭强制访问控制安全策略

vi /etc/sysconfig/selinux

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded. SELINUX=enforcing
# SELINUXTYPE= can take one of three two values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection. SELINUXTYPE=targeted

SELINUX=enforcing改成SELINUX=disabled

reboot使禁用操作生效

查看操作是否生效

[root@localhost ~]# getenforce
Disabled

安装Gitlab依赖

yum install curl policycoreutils openssh-server openssh-clients postfixs

[root@localhost ~]# yum install curl policycoreutils openssh-server openssh-clients postfixs
已加载插件：fastestmirror
Determining fastest mirrors
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
base | 3.6 kB 00:00:00
extras | 3.4 kB 00:00:00
updates | 3.4 kB 00:00:00
(1/4): base/7/x86_64/group_gz | 166 kB 00:00:00
(2/4): extras/7/x86_64/primary_db | 174 kB 00:00:00
(3/4): base/7/x86_64/primary_db | 5.9 MB 00:00:01
(4/4): updates/7/x86_64/primary_db | 5.0 MB 00:00:02
软件包 curl-7.29.0-46.el7.x86_64 已安装并且是最新版本
软件包 policycoreutils-2.5-22.el7.x86_64 已安装并且是最新版本
软件包 openssh-server-7.4p1-16.el7.x86_64 已安装并且是最新版本
软件包 openssh-clients-7.4p1-16.el7.x86_64 已安装并且是最新版本
没有可用软件包 postfixs。
无须任何处理

下载gitlab仓库源

curl -sS https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh | sudo bash

启动邮件服务

[root@localhost ~]# systemctl start postfix

配置开机启动

[root@localhost ~]# systemctl enable postfix

安装gitlab

yum -y install gitlab-ce

配置SSL证书

mkdir -p /etc/gitlab/ssl

创建本地私有秘钥

[root@localhost ~]# openssl genrsa -out "/etc/gitlab/ssl/gitlab.fjy8018.top.key" 2048
Generating RSA private key, 2048 bit long modulus
..................................................+++
......................+++
e is 65537 (0x10001)

创建CSR证书

[root@localhost ~]# openssl req -new -key "/etc/gitlab/ssl/gitlab.fjy8018.top.key" -out "/etc/gitlab/ssl/gitlab.fjy8018.top.csr"
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:cn
State or Province Name (full name) []:gd
Locality Name (eg, city) [Default City]:gz
Organization Name (eg, company) [Default Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:gitlab.fjy8018.top
Email Address []:admin@example.com

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:12345678
An optional company name []:

查看证书是否成功创建

[root@localhost ~]# cd /etc/gitlab/ssl
[root@localhost ssl]# ll
总用量 8
-rw-r--r-- 1 root root 1070 8月 18 11:11 gitlab.fjy8018.top.csr
-rw-r--r-- 1 root root 1675 8月 18 11:08 gitlab.fjy8018.top.key

创建签署证书

利用秘钥和csr证书创建签署证书 X.509为证书格式 365为有效期

[root@localhost ssl]# openssl x509 -req -days 365 -in "/etc/gitlab/ssl/gitlab.fjy8018.top.csr" -signkey "/etc/gitlab/ssl/gitlab.fjy8018.top.key" -out "/etc/gitlab/ssl/gitlab.fjy8018.top.crt"
Signature ok
subject=/C=cn/ST=gd/L=gz/O= /OU= /CN=gitlab.fjy8018.top/emailAddress=admin@example.com
Getting Private key

查看证书目录

[root@localhost ssl]# ll
总用量 12
-rw-r--r-- 1 root root 1265 8月 18 11:14 gitlab.fjy8018.top.crt
-rw-r--r-- 1 root root 1070 8月 18 11:11 gitlab.fjy8018.top.csr
-rw-r--r-- 1 root root 1675 8月 18 11:08 gitlab.fjy8018.top.key

输出pem证书

openssl dhparam -out /etc/gitlab/ssl/dhparams.pem 2048

[root@localhost ssl]# ll
总用量 16
-rw-r--r-- 1 root root 424 8月 18 11:17 dhparams.pem
-rw-r--r-- 1 root root 1265 8月 18 11:14 gitlab.fjy8018.top.crt
-rw-r--r-- 1 root root 1070 8月 18 11:11 gitlab.fjy8018.top.csr
-rw-r--r-- 1 root root 1675 8月 18 11:08 gitlab.fjy8018.top.key

证书创建完成修改证书权限

[root@localhost ssl]# chmod 700 *

查看权限

[root@localhost ssl]# ll
总用量 16
-rw------- 1 root root 424 8月 18 11:17 dhparams.pem
-rw------- 1 root root 1265 8月 18 11:14 gitlab.fjy8018.top.crt
-rw------- 1 root root 1070 8月 18 11:11 gitlab.fjy8018.top.csr
-rw------- 1 root root 1675 8月 18 11:08 gitlab.fjy8018.top.key

配置gitlab证书文件

vi /etc/gitlab/gitlab.rb

修改external_url为https的域名

external_url 'https://gitlab.fjy8018.top'

检索重定向配置，启用重定向

# nginx['client_max_body_size'] = '250m'
 nginx['redirect_http_to_https'] = true
# nginx['redirect_http_to_https_port'] = 80

修改下方crt配置原始为# nginx['ssl_certificate'] = "/etc/gitlab/ssl/#{node['fqdn']}.crt" 修改为nginx['ssl_certificate'] = "/etc/gitlab/ssl/gitlab.fjy8018.top.crt" key、ssl_dhparam同样配置

nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/gitlab.fjy8018.top.key"
nginx['ssl_dhparam'] = "/etc/gitlab/ssl/dhparams.pem" # Path to dhparams.pem, eg. /etc/gitlab/ssl/dhparams.pem

初始化配置文件

gitlab-ctl reconfigure

修改gitlab的nginx配置文件

vi /var/opt/gitlab/nginx/conf/gitlab-http.conf

搜索server_name

 server_name gitlab.fjy8018.top;

并在下方添加重定向HTTP请求

 rewrite ^(.*)$ https://$hosst1 permanent;

使nginx配置生效

[root@localhost ssl]# gitlab-ctl restart

所有服务启动成功

访问成功

Ansible安装配置

Sun, 19 Aug 2018 16:38:00 +0800

ansible官方文档指南

系统预配置

关闭防火墙

[root@localhost ~]# systemctl stop firewalld

关闭防火墙开机启动

[root@localhost ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

关闭强制访问控制安全策略

vi /etc/sysconfig/selinux

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of three two values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted

SELINUX=enforcing改成SELINUX=disabled

reboot使禁用操作生效

查看操作是否生效

[root@localhost ~]# getenforce
Disabled

安装Python3

wget https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tar.xz

解压编译包

tar xf Python-3.7.0.tar.xz

查看文件

[root@localhost ~]# ls
anaconda-ks.cfg Python-3.7.0 Python-3.7.0.tar.xz
[root@localhost ~]# cd Python-3.7.0

安装OpenSSL依赖，否则使用pip3.7会报找不到SSL模块错误

yum install openssl-devel

设定编译参数，/usr/local为安装位置，–with-ensurepip=install指定安装包管理工具，–enable-shared配置Python匹配系统参数

[root@localhost Python-3.7.0]# ./configure --prefix=/usr/local --with-ensurepip=install --enable-shared LDFLAGS="-Wl,-rpath /usr/local/lib" --with-ssl
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking for python3.7... no
checking for python3... no
checking for python... python
checking for --enable-universalsdk... no
checking for --with-universal-archs... no
checking MACHDEP... checking for --without-gcc... no
checking for --with-icc... no
checking for gcc... no
checking for cc... no
checking for cl.exe... no
configure: error: in `/root/Python-3.7.0':
configure: error: no acceptable C compiler found in $PATH
See `config.log' for more details

报错为找不到C编译器，由于系统为最小化安装，没有自带gcc编译器安装gcc

yum install gcc

再次编译，编译通过安装编译文件

 make && make altinstall

日志如下

Traceback (most recent call last):
 File "/root/Python-3.7.0/Lib/runpy.py", line 193, in _run_module_as_main
 "__main__", mod_spec)
 File "/root/Python-3.7.0/Lib/runpy.py", line 85, in _run_code
 exec(code, run_globals)
 File "/root/Python-3.7.0/Lib/ensurepip/__main__.py", line 5, in <module>
 sys.exit(ensurepip._main())
 File "/root/Python-3.7.0/Lib/ensurepip/__init__.py", line 204, in _main
 default_pip=args.default_pip,
 File "/root/Python-3.7.0/Lib/ensurepip/__init__.py", line 117, in _bootstrap
 return _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
 File "/root/Python-3.7.0/Lib/ensurepip/__init__.py", line 27, in _run_pip
 import pip._internal
zipimport.ZipImportError: can't decompress data; zlib not available

make: *** [altinstall] 错误 1

编译报错，找不到zlib，同理，安装zlib

yum -y install zlib*

另外，此时安装的是3.7版本，3.7版本需要一个新的包libffi-devel否则也会编译报错

yum install libffi-devel -y

安装成功

Python3.7对应的包管理工具pip版本也为3.7，查看其路径

[root@localhost Python-3.7.0]# which pip3.7
/usr/local/bin/pip3.7

设置软链接

ln -s /usr/local/bin/pip3.7 /usr/local/bin/pip

安装virtualenv

安装virtualenv，用于隔离Python环境

pip install virtualenv

安装git工具及其依赖

yum -y install git nss curl

创建ansible账户

useradd ansible

切换用户

[root@localhost Python-3.7.0]# su - ansible
[ansible@localhost ~]$

在ansible用户下创建virtualenv实例，用于集成ansible2.5

virtualenv -p /usr/local/bin/python3.7 .py3-a2.5-env

源码安装git-ansible2.5

cd /home/ansible/.py3-a2.5-env

下载源码

git clone https://github.com/ansible/ansible.git

加载virtualenv环境

[ansible@localhost .py3-a2.5-env]$ source /home/ansible/.py3-a2.5-env/bin/activate
(.py3-a2.5-env) [ansible@localhost .py3-a2.5-env]$

安装ansible依赖

pip install paramiko PyYAML jinja2

查看文件

(.py3-a2.5-env) [ansible@localhost .py3-a2.5-env]$ ll
总用量 8
drwxrwxr-x 14 ansible ansible 4096 8月 18 20:24 ansible
drwxrwxr-x 2 ansible ansible 248 8月 18 20:16 bin
drwxrwxr-x 2 ansible ansible 24 8月 18 20:13 include
drwxrwxr-x 3 ansible ansible 23 8月 18 20:13 lib
-rw-rw-r-- 1 ansible ansible 59 8月 18 20:16 pip-selfcheck.json

移动源码至虚拟环境

mv ansible/ .py3-a2.5-env/

进入虚拟环境源码目录

(.py3-a2.5-env) [ansible@localhost .py3-a2.5-env]$ cd .py3-a2.5-env/
(.py3-a2.5-env) [ansible@localhost .py3-a2.5-env]$ ll
总用量 108
drwxrwxr-x 2 ansible ansible 223 8月 18 20:24 bin
drwxrwxr-x 3 ansible ansible 63 8月 18 20:24 changelogs
-rw-rw-r-- 1 ansible ansible 10591 8月 18 20:24 CODING_GUIDELINES.md
drwxrwxr-x 4 ansible ansible 53 8月 18 20:24 contrib
-rw-rw-r-- 1 ansible ansible 35148 8月 18 20:24 COPYING
drwxrwxr-x 7 ansible ansible 71 8月 18 20:24 docs
-rw-rw-r-- 1 ansible ansible 48 8月 18 20:24 docsite_requirements.txt
drwxrwxr-x 4 ansible ansible 192 8月 18 20:24 examples
drwxrwxr-x 5 ansible ansible 305 8月 18 20:24 hacking
drwxrwxr-x 3 ansible ansible 21 8月 18 20:24 lib
drwxrwxr-x 2 ansible ansible 78 8月 18 20:24 licenses
-rw-rw-r-- 1 ansible ansible 13452 8月 18 20:24 Makefile
-rw-rw-r-- 1 ansible ansible 557 8月 18 20:24 MANIFEST.in
-rw-rw-r-- 1 ansible ansible 201 8月 18 20:24 MODULE_GUIDELINES.md
drwxrwxr-x 10 ansible ansible 120 8月 18 20:24 packaging
-rw-rw-r-- 1 ansible ansible 5265 8月 18 20:24 README.rst
-rw-rw-r-- 1 ansible ansible 371 8月 18 20:24 requirements.txt
-rw-rw-r-- 1 ansible ansible 10586 8月 18 20:24 setup.py
-rw-rw-r-- 1 ansible ansible 3287 8月 18 20:24 shippable.yml
drwxrwxr-x 10 ansible ansible 121 8月 18 20:24 test
-rw-rw-r-- 1 ansible ansible 1103 8月 18 20:24 tox.ini

切换版本

(.py3-a2.5-env) [ansible@localhost .py3-a2.5-env]$ git checkout stable-2.5

分支 stable-2.5 设置为跟踪来自 origin 的远程分支 stable-2.5。切换到一个新分支 ‘stable-2.5’

(.py3-a2.5-env) [ansible@localhost ansible]$ git branch
 devel
* stable-2.5

(.py3-a2.5-env) [ansible@localhost ansible]$ pwd
/home/ansible/.py3-a2.5-env/ansible

加载ansible2.5版本

source /home/ansible/.py3-a2.5-env/ansible/hacking/env-setup -q

验证版本

(.py3-a2.5-env) [ansible@localhost ansible]$ ansible --version
ansible 2.5.8 (stable-2.5 b23b1c2ee0) last updated 2018/08/18 20:35:45 (GMT +800)
 config file = None
 configured module search path = ['/home/ansible/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
 ansible python module location = /home/ansible/.py3-a2.5-env/ansible/lib/ansible
 executable location = /home/ansible/.py3-a2.5-env/ansible/bin/ansible
 python version = 3.7.0 (default, Aug 18 2018, 20:09:08) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]

从零开始部署Rancher

Sat, 11 Aug 2018 11:41:00 +0800

基础环境部署

基础环境使用CentOS 7，版本号为1804，部署在本地虚拟机VM上

安装CentOS+docker

CentOS下载地址

使用最小安装，ISO镜像大小为906MB，在VM中导入镜像，给4G内存（Rancher2.0最低要求），安装过程系统配置均为默认配置

安装好后为纯净环境，使用ip addr即可查看ip，使用外部远程连接工具连接虚拟机ip

使用yum install docker安装docker环境

安装好后使用命令systemctl start docker启动docker 使用 docker info或者docker version查看版本

[root@localhost ~]# docker version
Client:
 Version: 1.13.1
 API version: 1.26
 Package version: docker-1.13.1-68.gitdded712.el7.centos.x86_64
 Go version: go1.9.4
 Git commit: dded712/1.13.1
 Built: Tue Jul 17 18:34:48 2018
 OS/Arch: linux/amd64

Server:
 Version: 1.13.1
 API version: 1.26 (minimum version 1.12)
 Package version: docker-1.13.1-68.gitdded712.el7.centos.x86_64
 Go version: go1.9.4
 Git commit: dded712/1.13.1
 Built: Tue Jul 17 18:34:48 2018
 OS/Arch: linux/amd64
 Experimental: false

安装Rancher

为了加速Rancher下载速度，需要配置docker镜像下载地址，此处提供一个阿里云的镜像地址

{
 "registry-mirrors": ["https://fy707np5.mirror.aliyuncs.com"]
}

使用命令编辑docker镜像地址 vi /etc/docker/daemon.json 保存后重载配置systemctl daemon-reload 重启docker服务systemctl restart docker 镜像地址配置完成

访问Rancher官网查看docker命令 Rancher英文官网

Rancher中文官网

访问quickstart查看docker命令

目前Rancher官方开始推荐2.0版本，首次使用推荐使用1.6稳定版

在下方找到1.6文档，使用命令在虚拟机中安装Rancher1.6

sudo docker run -d --restart=unless-stopped -p 8080:8080 rancher/server:stable

配置防火墙

安装后若没有打开8080端口则会报错，配置CentOS防火墙来源：[莫小安-CentOS7使用firewalld打开关闭防火墙与端口][10]

1、firewalld的基本使用启动： systemctl start firewalld 关闭： systemctl stop firewalld 查看状态： systemctl status firewalld 开机禁用： systemctl disable firewalld 开机启用： systemctl enable firewalld

2.systemctl是CentOS7的服务管理工具中主要的工具，它融合之前service和chkconfig的功能于一体。启动一个服务：systemctl start firewalld.service 关闭一个服务：systemctl stop firewalld.service 重启一个服务：systemctl restart firewalld.service 显示一个服务的状态：systemctl status firewalld.service 在开机时启用一个服务：systemctl enable firewalld.service 在开机时禁用一个服务：systemctl disable firewalld.service 查看服务是否开机启动：systemctl is-enabled firewalld.service 查看已启动的服务列表：systemctl list-unit-files|grep enabled 查看启动失败的服务列表：systemctl –failed

3.配置firewalld-cmd

查看版本： firewall-cmd –version 查看帮助： firewall-cmd –help 显示状态： firewall-cmd –state 查看所有打开的端口： firewall-cmd –zone=public –list-ports 更新防火墙规则： firewall-cmd –reload 查看区域信息: firewall-cmd –get-active-zones 查看指定接口所属区域： firewall-cmd –get-zone-of-interface=eth0 拒绝所有包：firewall-cmd –panic-on 取消拒绝状态： firewall-cmd –panic-off 查看是否拒绝： firewall-cmd –query-panic

那怎么开启一个端口呢添加 firewall-cmd –zone=public –add-port=80/tcp –permanent （–permanent永久生效，没有此参数重启后失效）重新载入 firewall-cmd –reload 查看 firewall-cmd –zone= public –query-port=80/tcp 删除 firewall-cmd –zone= public –remove-port=80/tcp –permanent

通过命令放行8080端口

firewall-cmd --zone=public --add-port=8080/tcp --permanent

重载防火墙

firewall-cmd --reload

查看端口放行情况

 firewall-cmd --zone=public --list-ports

设置Rancher

设置Rancher界面语言

通过虚拟机ip+端口号访问Rancher：http://192.168.79.142:8080

默认为英文界面，Rancher官方有中文支持修改右下方语言选项即可

设置Rancher主机

首次访问需要设置主机

按照默认识别的IP即可，点击保存

选择主机类型，包含自定义主机、亚马逊主机等，此处选择自定义主机

依照指示操作，为了确保通信测试，暂时关闭防火墙

为了测试多台Rancher Server和agent服务，使用VM保存快照并复制一份虚拟机启动目前测试：128为server，129为agent，注意此处填的是客户端的地址，非主机地址

设置Rancher客户端

复制Server的配置命令，在agent服务器上运行，等待下载

按照指示，运行后关闭Server主机配置查看主机列表

也可以添加环境、测试环境、生产环境等

SpringCloud中遇到的坑

Tue, 24 Jul 2018 21:34:00 +0800

SpringCloud Config

配置无法读取

问题描述

配置中心的配置文件如下

eureka:
 client:
 service-url:
 defaultZone: http://localhost:8761/eureka/
spring:
 application:
 name: config
 cloud:
 config:
 server:
 git:
 uri: http://XXX/fjy8018/config-repo
 username: 1186032234@qq.com
 password: XXX
 basedir: spring-cloud\config\basedir
server:
 port: 8084

已经通过网页测试该账号和密码可以访问该仓库，并有权限，启动配置中心后通过页面访问提示Cannot clone or checkout repository

解决方法

SpringCloud正式版的配置中心路径必须是带.git结尾的全路径，将配置改为全路径即可

eureka:
 client:
 service-url:
 defaultZone: http://localhost:8761/eureka/
spring:
 application:
 name: config
 cloud:
 config:
 server:
 git:
 uri: http://XXX/fjy8018/config-repo.git
 username: 1186032234@qq.com
 password: XXX
 basedir: spring-cloud\config\basedir
server:
server:
 port: 8084

SpringBoot的一些坑

Sun, 08 Jul 2018 16:03:00 +0800

bindingResult无法捕获校验错误

错误复现

原工程代码，RegisterForm对象设置了校验规则

预期的返回结果

{
 "code": 4,
 "msg": "表单错误",
 "data": null
}

实际执行结果

并且控制台打出了异常堆栈，而且记录的log4j日志也没有显示

原因

经过查阅资料，得到原因

@Valid 和 BindingResult 是一一对应的，如果有多个@Valid，那么每个@Valid后面跟着的BindingResult就是这个@Valid的验证结果，顺序不能乱

解决

故将BindingResult跟在RegisterForm后面即可

public ResultVO doRegister(@Valid RegisterForm registerForm, BindingResult bindingResult, HttpServletRequest request) throws Exception{

再次执行得到预期返回值

控制台也打出预期的日志，并没有显示异常

Docker部署中遇到的坑

Thu, 05 Jul 2018 14:21:00 +0800

镜像无法删除

问题复现

当前的镜像

PS D:\JAVA WORKSPACE\dockerTemp> docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
orderbook latest 6514a4df3367 7 seconds ago 300MB
hub.c.163.com/fjy8018/mysql-orderbooks latest 37a82bac5f71 About an hour ago 407MB
hub.c.163.com/library/tomcat latest 72d2be374029 10 months ago 292MB
hub.c.163.com/library/mysql latest 9e64176cd8a2 14 months ago 407MB

运行一个镜像

PS D:\JAVA WORKSPACE\dockerTemp> docker run -d -p 8888:8080 orderbook:latest
678dcf28c1c3206cd78a824aab51cc5d1f6c333ed85f48d4c364a51526049f20

查看进程，发现进程没有出现（3306是已经启动的MySQL容器），由于容器制作过程出现问题导致进程无法成功启动

PS D:\JAVA WORKSPACE\dockerTemp> docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
43146ddfc771 hub.c.163.com/library/mysql:latest "docker-entrypoint.s…" 2 hours ago Up 2 hours 0.0.0.0:8006->3306/tcp hopeful_euclid

检查镜像

PS D:\JAVA WORKSPACE\dockerTemp> docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
orderbook latest 6514a4df3367 44 seconds ago 300MB
hub.c.163.com/fjy8018/mysql-orderbooks latest 37a82bac5f71 About an hour ago 407MB
hub.c.163.com/library/tomcat latest 72d2be374029 10 months ago 292MB
hub.c.163.com/library/mysql latest 9e64176cd8a2 14 months ago 407MB

删除没有成功运行的镜像报错，报该容器已经被占用

PS D:\JAVA WORKSPACE\dockerTemp> docker rmi 6514a4df3367
Error response from daemon: conflict: unable to delete 6514a4df3367 (must be forced) - image is being used by stopped container 678dcf28c1c3

删除报错中提示的容器ID

PS D:\JAVA WORKSPACE\dockerTemp> docker rmi 678dcf28c1c3
Error: No such image: 678dcf28c1c3

报无该镜像

解决

官方对rm与rmi的解释： rm Remove one or more containers rmi Remove one or more images 可知rm删除的是容器，rmi删除的是镜像，两者的区别如下：

容器（container）的定义和镜像（image）几乎一模一样，也是一堆层的统一视角，唯一区别在于容器的最上面那一层是可读可写的。镜像（Image）就是一堆只读层（read-only layer）的统一视角要点：容器 = 镜像 + 可读层。并且容器的定义并没有提及是否要运行容器。

故对于不在镜像列表的ID，要删除的是容器，使用rm命令 PS D:\JAVA WORKSPACE\dockerTemp> docker rm 678dcf28c1c3 678dcf28c1c3

列出所有的镜像

PS D:\JAVA WORKSPACE\dockerTemp> docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
orderbook latest 6514a4df3367 2 minutes ago 300MB
hub.c.163.com/fjy8018/mysql-orderbooks latest 37a82bac5f71 About an hour ago 407MB
hub.c.163.com/library/tomcat latest 72d2be374029 10 months ago 292MB
hub.c.163.com/library/mysql latest 9e64176cd8a2 14 months ago 407MB

再删除目标镜像 PS D:\JAVA WORKSPACE\dockerTemp> docker rmi 6514a4df3367 Untagged: orderbook:latest Deleted: sha256:6514a4df33678bd98e32b5738150c88c5d72b9a3a4d86d1c370406f0f7f18788 Deleted: sha256:68a70a0ac20590c8412286b3c34ab2bc5c5b7c8845088536f03702185ac34105 Deleted: sha256:bf15e37a31e4266bc8a962a2b11fc3223c07de328ff85b17c70a15653c71061b Deleted: sha256:b447a606517609703f1efe4c560c42b9aeeaee669d22f12a65d2d2929d8ecc20

删除成功
PS D:\JAVA WORKSPACE\dockerTemp> docker images REPOSITORY TAG IMAGE ID CREATED SIZE hub.c.163.com/fjy8018/mysql-orderbooks latest 37a82bac5f71 About an hour ago 407MB hub.c.163.com/library/tomcat latest 72d2be374029 10 months ago 292MB hub.c.163.com/library/mysql latest 9e64176cd8a2 14 months ago 407MB

停止并删除所有容器

批处理命令

docker stop $(docker ps -aq)
docker rm $(docker ps -aq)

自定义docker容器

Wed, 04 Jul 2018 17:06:00 +0800

准备docker环境

去官网下载docker环境，此处使用win10的docker环境，若早期版本的windows要安装额外的工具

检查docker版本

下载基础环境

本次决定搭建的docker为订书系统，其基于jsp编写，运行在tomcat上，数据库依赖MySQL，故需要一个MySQL和tomcat环境（tomcat环境自带Java）去网易镜像中心下载镜像，网易镜像中心与官方镜像中心保持同步更新，而且对于国内来说速度更快

部署MySQL镜像

下载

复制下载地址

mysql镜像下方也有使用说明

以及环境变量的描述，是否需要在创建时建立数据库等

启动

使用命令启动mysql的docker，指定本机8006端口映射到docker的3306端口，同时设置数据库root用户密码为root -d指定为后台运行

docker run -d -p 8006:3306 -e MYSQL_ROOT_PASSWORD=root hub.c.163.com/library/mysql:latest

使用命令查看当前运行的docker

docker ps

测试

使用Navicat连接docker的数据库

建立相关数据库，不使用环境变量创建是为了设置字符集

启用命令行界面创建相关用户并授权

导入数据库文件

部署Tomcat镜像

查看本机的ip

下载

从镜像中心下载tomcat

查看镜像中配置的tomcat目录

查看当前已经下载的镜像

PS D:\JAVA WORKSPACE\dockerTemp> docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
hub.c.163.com/library/tomcat latest 72d2be374029 10 months ago 292MB
hub.c.163.com/library/mysql latest 9e64176cd8a2 14 months ago 407MB

编写Dockerfile文件将工程移到tomcat的webapps目录下

from hub.c.163.com/library/tomcat

MAINTAINER fjy8018 fjy8018@gmail.com

COPY /OrderBooks /usr/local/tomcat/webapps/OrderBooks

修改工程中的数据库地址，地址为ip+暴露的docker端口

使用docker build -t orderbook .命令将docker打包成镜像

启动

使用命令启动带工程的tomcat，指定访问端口为8888

docker run -d -p 8888:8080 orderbook:latest

配置

由于使用了jsp+jdbc作为数据库连接，故需要进入容器内部将jdbc的jar复制到tomcat的lib目录下，否则tomcat会报无法加载驱动，使用命令进入容器内部

docker exec -it {容器ID} /bin/sh

使用exec命令比attach优势在于，exec进入容器后再使用exit命令不会令容器停止进入容器将工程的lib复制到tomcat中的lib

PS D:\JAVA WORKSPACE\dockerTemp> docker exec -it 6fe288a25be0 /bin/sh
# cp /usr/local/tomcat/webapps/OrderBooks/WebRoot/WEB-INF/lib/mysql-connector-java-5.1.45-bin.jar /usr/local/tomcat/lib
# exit
PS D:\JAVA WORKSPACE\dockerTemp> docker restart 6fe288a25be0
6fe288a25be0

测试

外部浏览器访问工程

工程正常访问

提交到网易镜像仓库

官方推送指南

提交修改的镜像

docker commit提交修改的镜像

PS D:\JAVA WORKSPACE\dockerTemp> docker commit 6fe288a25be0 orderbooks
sha256:30c6a878adc80f54118f5c67c39c271baa5dc68006aa26b64ef5281832fb4b37
PS D:\JAVA WORKSPACE\dockerTemp> docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
orderbooks latest 30c6a878adc8 5 seconds ago 301MB
orderbook latest a57e26354272 10 minutes ago 300MB
hub.c.163.com/library/tomcat latest 72d2be374029 10 months ago 292MB
hub.c.163.com/library/mysql latest 9e64176cd8a2 14 months ago 407MB

提交mysql容器的修改 PS D:\JAVA WORKSPACE\dockerTemp> docker commit 43146ddfc771 hub.c.163.com/fjy8018/mysql-orderbooks sha256:37a82bac5f71d54e3ff31b0ddd795604e46d4f2e4dde8a6a73d556cc5f26411a PS D:\JAVA WORKSPACE\dockerTemp> docker images REPOSITORY TAG IMAGE ID CREATED SIZE hub.c.163.com/fjy8018/mysql-orderbooks latest 37a82bac5f71 3 seconds ago 407MB orderbooks latest 30c6a878adc8 13 minutes ago 301MB hub.c.163.com/fjy8018/tomcat-orderbooks latest 30c6a878adc8 13 minutes ago 301MB orderbook latest a57e26354272 23 minutes ago 300MB hub.c.163.com/library/tomcat latest 72d2be374029 10 months ago 292MB hub.c.163.com/library/mysql latest 9e64176cd8a2 14 months ago 407MB

推送镜像

将镜像推送到网易仓库

登录到网易镜像仓库查看，默认推送会设置为private，推送完成

Redis分布式锁在高并发情况下偶然突然失效

Mon, 11 Jun 2018 12:40:00 +0800

背景

原理

在学习Redis分布式锁时编写Service类，其中包含加锁和解锁两个方法，通过调用Redis的SETNX和GETSET方法加锁

源码

为便于查看，加了logback进行日志打印

需要加锁的方法，该方法为模拟抢购场景的实现

交由SpringBoot自动注入写好的RedisLock包（其实问题出在这里，本应该是10*1000，手误写成10*000结果为0，故所有超时为0）

访问Controller，一个为抢购地址，抢购地址访问一次则数量减一，一个为查询地址

压测工具

使用Apache AB模拟高并发访问

并发测试

手动测试

手动访问，测试功能是否正常查询

订购

压测

10个请求，单线程发起测试

数量正常

50个请求，2个线程发起测试

出错

数量显然不对

查看日志，发现后续请求都从判断锁是否过期的方法获得的锁，但执行上数值一致，逻辑也没问题

原因

后来发现在高并发情况下超时时间如果设置过小，多个线程可能获得的时间戳一致，导致加锁无效，即每个线程都在判断时间戳时可以获得锁，导致“超售”

再次压测

数量正常

提高并发量

数量正常，问题解决

elasticsearch原生json操作笔记

Mon, 21 May 2018 07:28:00 +0800

建立结构

建People表

方法：PUT 发送地址：http://192.168.79.138:9200/people

{
	"setting": {
		"number_of_shards": 3,
		"number_of_replicas": 1
	},
	"mappings": {
		"man": {
			"properties": {
				"name": {
					"type": "text"
				},
				"country": {
					"type": "keyword"
				},
				"age": {
					"type": "integer"
				},
				"data":{
					"type": "date",
					"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
				}
			}
		}
	}
}

建book表

方法：PUT 发送地址：http://192.168.79.138:9200/book

{
	"setting": {
		"number_of_shards": 3,
		"number_of_replicas": 1
	},
	"mappings": {
		"novel": {
			"properties": {
				"word_count": {
					"type": "integer"
				},
				"author": {
					"type": "keyword"
				},
				"title": {
					"type": "text"
				},
				"publish_date":{
					"type": "date",
					"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
				}
			}
		}
	}
}

插入数据

指定ID插入数据到people.man

方法：PUT 发送地址http://192.168.79.138:9200/people/man/1

{
	"name": "F嘉阳",
	"country": "China",
	"age": 21,
	"date": "1990-05-18"
}

图形化查询检验

不指定ID插入数据到people.man

方法：PUT 发送地址http://192.168.79.138:9200/people/man/

{
	"name": "FJY",
	"country": "China",
	"age": 28,
	"date": "1990-05-18"
}

图形化查询检验

更新

更新字段数据

方法：POST 发送地址：http://192.168.79.138:9200/people/man/1/_update

{
	"doc": {
		"name": "符嘉阳"
	}
}

图形化查询检验

使用脚本语言更新数据

方法：POST 发送地址：http://192.168.79.138:9200/people/man/1/_update

{
	"script": {
		"lang": "painless",
		"inline": "ctx._source.age += 5"
	}
}

图形化查询检验

使用脚本语言分离参数更新数据

方法：POST 发送地址：http://192.168.79.138:9200/people/man/1/_update

{
	"script": {
		"lang": "painless",
		"inline": "ctx._source.age = params.age",
		"params": {
			"age": 21
		}
	}
}

图形化查询检验

删除

指定ID删除一行数据

方法：DELETE 发送地址：http://192.168.79.138:9200/people/man/1

图形化查询检验

删除表

方法：DELETE 发送地址：http://192.168.79.138:9200/people

查询

指定ID查询

方法：GET 发送地址：http://192.168.79.138:9200/book/novel/H5LbbWMBzFxBPWV2GG9d

查询所有

方法：GET 发送地址：http://192.168.79.138:9200/book/_search { “query”: { “match_all”: {} } }

指定查询结果大小

{
	"query": {
		"match_all": {}
	},
	"from": 1,
	"size": 1
}

查询

{
	"query": {
		"match": {
			"title": "Java"
		}
	}
}

指定查询结果排序方法

{
	"query": {
		"match": {
			"title": "Java"
		}
	},
	"sort": [
		{
			"publish_date": {
				"order": "desc"
			}
		}
	]
}

聚合

{
	"aggs": {
		"group_by_word_count": {
			"terms": {
				"field": "word_count"
			}
		}
	}
}

多字段聚合

{
	"aggs": {
		"group_by_word_count": {
			"terms": {
				"field": "word_count"
			}
		},
		"group_by_author": {
			"terms": {
				"field": "author"
			}
		}
	}
}

聚合

{
	"aggs": {
		"grades_by_word_count": {
			"stats": {
				"field": "word_count"
			}
		}
	}
}

模糊匹配(默认匹配法则)

{
	"query": {
		"match": {
			"title": "Java虚拟机"
		}
	}
}

习语匹配

{
	"query": {
		"match_phrase": {
			"title": "Java虚拟机"
		}
	}
}

多字段模糊匹配查询

{
	"query": {
		"multi_match": {
			"query": "F嘉阳",
			"fields": ["author","title"]
		}
	}
}

语法查询

{
	"query": {
		"query_string": {
			"query": "ElasticSearch AND 入门"
		}
	}
}

{
	"query": {
		"query_string": {
			"query": "(ElasticSearch AND 入门) OR PS"
		}
	}
}

语法查询指定字段

{
	"query": {
		"query_string": {
			"query": "ElasticSearch OR 瓦",
			"fields": ["title","author"]
		}
	}
}

结构化查询(指定字段)

{
	"query": {
		"term": {
			"word_count": 1000
		}
	}
}

{
	"query": {
		"term": {
			"author": "F嘉阳"
		}
	}
}

结构化查询(指定范围)

gte->大于等于 lte->小于等于 e->equal(等于)

{
	"query": {
		"range": {
			"word_count": {
				"gte": 1000,
				"lte": 2000
			}
		}
	}
}

结构化查询(指定日期范围)

{
	"query": {
		"range": {
			"publish_date": {
				"gte": "2018-01-01",
				"lt": "2018-12-31"
			}
		}
	}
}

结构化查询(指定当前时间)

{
	"query": {
		"range": {
			"publish_date": {
				"gte": "2018-01-01",
				"lte": "now"
			}
		}
	}
}

子条件查询，使用filter

使用filter过滤器后产生缓存，下次查询速度更快

{
	"query": {
		"bool": {
			"filter": {
				"term": {
					"word_count": 1000
				}
			}
		}
	}
}

复合条件查询-固定分数查询

{
	"query": {
		"constant_score": {
			"filter": {
				"match": {
					"title": "Java"
				}
			}
		}
	}
}

复合条件查询-指定分数查询

{
	"query": {
		"constant_score": {
			"filter": {
				"match": {
					"title": "Java"
				}
			},
			"boost": 2
		}
	}
}

复合条件查询-布尔查询-指定应当满足的条件(满足其一即可)

{
	"query": {
		"bool": {
			"should": [
					{
						"match": {
							"author": "F嘉阳"
						}
					},
					{
						"match": {
							"title": "Java"
						}
					}
				]
		}
	}
}

复合条件查询-布尔查询-指定必须满足的条件(两个都必须匹配)

{
	"query": {
		"bool": {
			"must": [
					{
						"match": {
							"author": "F嘉阳"
						}
					},
					{
						"match": {
							"title": "Java"
						}
					}
				]
		}
	}
}

复合条件查询-布尔查询-过滤器

{
	"query": {
		"bool": {
			"must": [
					{
						"match": {
							"author": "F嘉阳"
						}
					},
					{
						"match": {
							"title": "PS"
						}
					}
				],
				"filter": [
						{
							"term": {
								"word_count": 700
							}
						}
					]
		}
	}
}

复合条件查询-must_not

{
	"query": {
		"bool": {
			"must_not": {
				"term": {
					"author": "F嘉阳"
				}
			}
		}
	}
}

elasticsearch分布式环境搭建和插件安装

Thu, 17 May 2018 18:01:00 +0800

环境搭建

安装包和插件下载

下载elasticsearch

通过官网下载elasticsearch 官网下载页面此处下载的为elasticsearch-6.2.4.tar.gz

下载elasticsearch-head插件

下载elasticsearch-head插件可以在浏览器中以图形化的方式显示下载elasticsearch中的数据 GitHub下载页面

安装node环境

elasticsearch-head插件依赖node环境来启动，可以通过官方node.js安装包安装，也可以通过宝塔自带的PM2管理器安装，此处安装宝塔的PM2管理器

安装后检查当前环境中的node版本，若装好后node命令无法使用可以尝试重启服务器或者让配置环境变量生效

 [root@localhost elasticsearch]# node -v
 v8.11.2

将两个下载好的文件用宝塔上传到CentOS 7环境中并解压

单节点环境配置

elasticsearch单节点配置

配置文件目录为elasticsearch-6.2.4/config 修改elasticsearch.yml

 #虚拟机IP
 network.host: 192.168.79.138
 #es端口
 http.port: 9200
 
 #防止跨源请求阻塞
 http.cors.enabled: true
 http.cors.allow-origin: "*"
 
 #主节点名称
 cluster.name: Fjiayang
 #节点名称
 node.name: master
 #是否为主节点
 node.master: true

若虚拟机或者服务器内存较小，则还要配置jvm.options，将默认1g内存设置为512m或者更小

 -Xms512m
 -Xmx512m

elasticsearch-head插件安装

进入elasticsearch-head安装目录，执行node的安装命令

 [root@localhost elasticsearch]# cd elasticsearch-head-master
 [root@localhost elasticsearch-head-master]# npm install

集群环境配置

集群环境只需要解压多份es压缩包，并按如下对elasticsearch.yml配置文件进行简单配置即可，此处搭建了一个主节点，两个从节点从节点1

 #子节点IP
 network.host: 192.168.79.138
 #子节点端口
 http.port: 8200
 #主节点IP
 discovery.zen.ping.unicast.hosts: ["192.168.79.138"]
 #主节点名称
 cluster.name: Fjiayang
 #子节点名称
 node.name: slave1

从节点2

 #子节点IP
 network.host: 192.168.79.138
 #子节点端口
 http.port: 8300
 #主节点IP
 discovery.zen.ping.unicast.hosts: ["192.168.79.138"]
 #主节点名称
 cluster.name: Fjiayang
 #子节点名称
 node.name: slave2

服务启动

elasticsearch启动

elasticsearch默认不支持root用户启动，故要新建用户并对运行目录授予权限才能启动es

 [root@localhost elasticsearch]# adduser es
 [root@localhost elasticsearch]# passwd es
 更改用户 es 的密码 。
 新的 密码：
 无效的密码： 密码未通过字典检查 - 过于简单化/系统化
 重新输入新的 密码：
 passwd：所有的身份验证令牌已经成功更新。

在root用户下对es目录所有权进行更改

 [root@localhost www]# chown -R es /www/elasticsearch
 [root@localhost www]# ll
 总用量 4
 drwxrwxrwx 5 es www 151 5月 16 21:10 elasticsearch
 drwxr-xr-x 6 root root 4096 5月 7 09:01 Recycle_bin
 drwxr-xr-x. 14 root root 179 5月 7 09:20 server
 drwxrwxrwx. 3 root root 116 5月 6 15:41 wwwlogs
 drwxr-xr-x. 4 root root 38 5月 6 15:41 wwwroot

切换到新建的用户es，用/bin/elasticsearch启动es服务，若加上-d则为后台启动，即/bin/elasticsearch -d

 [root@localhost www]# su es
 [es@localhost elasticsearch]$ cd elasticsearch/elasticsearch-6.2.4
 [es@localhost elasticsearch-6.2.4]$ ls
 bin config data hs_err_pid4546.log lib LICENSE.txt logs modules NOTICE.txt plugins README.textile
 [es@localhost elasticsearch-6.2.4]$ ./bin/elasticsearch -d

如果配置了集群则继续对每个从节点逐一启动即可

elasticsearch-head插件启动

进入elasticsearch-head目录，使用npm run start启动node服务即可

服务检验

首先使用宝塔开放相关防火墙端口，

直接访问es服务，浏览器默认显示为原生json格式，此处使用chrome浏览器的JSON viewer进行了美化

访问elasticsearch-head插件，此处截图为配置好集群后的截图

elasticsearch环境搭建完成！

hadoop分布式搭建的一些坑

Sun, 22 Apr 2018 10:39:00 +0800

DataNode没启动

这种情况多出现于该机器之前配置过hadoop，没清理干净的情况，即仍有文件目录，导致新hadoop的clusterID与旧的不一致，导致HDFS无法启动，报JavaIO异常

 2018-04-22 00:21:15,868 WARN org.apache.hadoop.hdfs.server.common.Storage: java.io.IOException: Incompatible clusterIDs in /www/hadoop/app/tmp/dfs/data: namenode clusterID = CID-6c7766c6-b260-42b6-8c03-07dda802c0cf; datanode clusterID = CID-a53df7ad-e593-4a29-ad15-a93486c54776
 2018-04-22 00:21:15,868 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to hadoop0/192.168.79.129:8020. Exiting. 
 java.io.IOException: All specified directories are failed to load.
 	at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:478)
 	at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1394)
 	at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1355)
 	at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:317)
 	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:228)
 	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:829)
 	at java.lang.Thread.run(Thread.java:745)
 2018-04-22 00:21:15,869 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (Datanode Uuid unassigned) service to hadoop0/192.168.79.129:8020

解决方案

清理所有节点的旧文件存储目录，重新再根节点格式化存储空间

 rm -rf app/
 hdfs namenode -format

重新启动HDFS

 [hadoop@hadoop0 bin]$ cd ../sbin
 [hadoop@hadoop0 sbin]$ ./start-all.sh

查看进程，DataNode已启动

 [hadoop@hadoop0 sbin]$ jps
 37618 DataNode
 37783 SecondaryNameNode
 37932 ResourceManager
 37485 NameNode
 38045 NodeManager
 38365 Jps

NodeManager没启动/启动后进程消失

该情况由于yarn-site.xml文件配置有误，导致从节点找不到主节点，报JavaNet异常

 org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindException: Problem binding to [hadoop0:0] java.net.BindException: 无法指定被请求的地址; For more details see: http://wiki.apache.org/hadoop/BindException
 	at org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
 	at org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
 	at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceStart(ContainerManagerImpl.java:406)
 	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
 	at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
 	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:266)
 	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
 	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:475)
 	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:521)

解决方案

重新配置yarn-site.xml，确保其有下面这一段，并且使用SSH 主机名能够访问该节点

 <!--ResourceManager主机名-->
 <property>
 	<name>yarn.resourcemanager.hostname</name>
 	<value>hadoop0</value>
 </property>

YARN运算IO错误

该错误由于机器内存过小，却执行复杂的运算，导致内存不足，报JavaIO异常，例如在2G内存的机器上执行 hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar pi 4 5运算

 Job Finished in 34.889 seconds
 java.io.FileNotFoundException: File does not exist: hdfs://192.168.79.129:8020/user/hadoop/QuasiMonteCarlo_1524323663509_1454922894/out/reduce-out
 at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1219)
 at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1211)
 at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
 at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1211)
 at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1750)
 at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1774)
 at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:314)
 at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
 at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
 at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
 at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
 at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

YARN也显示工作失败

解决方案

调整虚拟内存，或者减小运算精度，例如设置精度为hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar pi 2 3则一般不会有问题

机器时间不同步

该问题出现在MapReduce测试时，若主从节点机器时间不一致则会报同步错误

 18/04/21 23:55:20 INFO mapreduce.Job: Task Id : attempt_1524320196806_0003_m_000000_2, Status : FAILED
 Container launch failed for container_1524320196806_0003_01_000004 : org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container.
 This token is expired. current time is 1524333912535 found 1524326717165
 Note: System times on machines may be out of sync. Check system time and time zones.
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
 at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:168)
 at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
 at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:159)
 at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:379)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)

解决方案

首先检查各节点的时间主节点

 [hadoop@hadoop0 testFile]$ timedatectl status
 Local time: 六 2018-04-21 23:57:21 CST
 Universal time: 六 2018-04-21 15:57:21 UTC
 RTC time: 六 2018-04-21 15:44:44
 Time zone: Asia/Shanghai (CST, +0800)
 NTP enabled: yes
 NTP synchronized: no
 RTC in local TZ: no
 DST active: n/a

从节点1

[hadoop@hadoop1 hadoop]$ timedatectl status
 Local time: 日 2018-04-22 02:06:49 CST
 Universal time: 六 2018-04-21 18:06:49 UTC
 RTC time: 六 2018-04-21 17:54:14
 Time zone: Asia/Shanghai (CST, +0800)
 NTP enabled: yes
NTP synchronized: no
 RTC in local TZ: no
 DST active: n/a

从节点2

[hadoop@hadoop2 hadoop]$ timedatectl status
 Local time: 日 2018-04-22 02:07:00 CST
 Universal time: 六 2018-04-21 18:07:00 UTC
 RTC time: 日 2018-04-22 04:00:48
 Time zone: Asia/Shanghai (CST, +0800)
 NTP enabled: yes
NTP synchronized: no
 RTC in local TZ: no
 DST active: n/a

三个节点显然时间不一致，使用ntpdate工具同步各机器时间

[root@hadoop2 hadoop]# yum -y install ntp ntpdate
已加载插件：fastestmirror
base | 3.6 kB 00:00:00
epel/x86_64/metalink | 7.7 kB 00:00:00
epel | 4.7 kB 00:00:00
extras | 3.4 kB 00:00:00
updates | 3.4 kB 00:00:00
(1/4): epel/x86_64/updateinfo | 916 kB 00:00:01
(2/4): extras/7/x86_64/primary_db | 185 kB 00:00:01
(3/4): epel/x86_64/primary_db | 6.3 MB 00:00:01
(4/4): updates/7/x86_64/primary_db | 6.9 MB 00:00:03
Determining fastest mirrors
 * base: centos.ustc.edu.cn
 * epel: mirror01.idc.hinet.net
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
软件包 ntp-4.2.6p5-25.el7.centos.2.x86_64 已安装并且是最新版本
软件包 ntpdate-4.2.6p5-25.el7.centos.2.x86_64 已安装并且是最新版本
无须任何处理
[root@hadoop2 hadoop]# ntpdate cn.pool.ntp.org
22 Apr 12:02:55 ntpdate[36396]: step time server 108.59.2.24 offset 35618.521668 sec

将时间偏差写入机器时间

[root@hadoop2 hadoop]# hwclock --systohc
[root@hadoop2 hadoop]# date
2018年 04月 22日 星期日 12:03:33 CST

问题解决

Hadoop分布式搭建测试

Sat, 21 Apr 2018 18:10:00 +0800

搭建环境

Windows VM虚拟机

虚拟机配置 CPU:7700 单核 RAM:2G SSD:20G OS:CentOS 7.4

环境配置

修改主机名

因为三台虚拟机均为一台虚拟机克隆而成，且源主机名为localhost，先计划将三台主机分别设置为hadoop0，hadoop1，hadoop2 IP、主机名、职能分别为： 192.168.79.129 hadoop0 NameNode DataNode ResourceManager NodeManager 192.168.79.130 hadoop1 DataNode NodeManager 192.168.79.131 hadoop2 DataNode NodeManager

CentOS 7使用hostnamectl set-hostname hadoop0设置主机名，其他几台同样操作，输入后可通过hostnamectl status查看是否修改成功

 [root@hadoop0 ~]# hostnamectl status
 Static hostname: hadoop0
 Icon name: computer-vm
 Chassis: vm
 Machine ID: 4eddbba909c84e159cfa474604412c3c
 Boot ID: df37e835fde94d039153028c9479b68e
 Virtualization: vmware
 Operating System: CentOS Linux 7 (Core)
 CPE OS Name: cpe:/o:centos:centos:7
 Kernel: Linux 3.10.0-693.el7.x86_64
 Architecture: x86-64

由于hadoop0是根节点，用于资源调度，故其需要具有所有DataNode的访问地址和权限，进入/etc/hosts文件，添加如下内容将主机名解析到对应的IP

 192.168.79.129 hadoop0
 192.168.79.130 hadoop1
 192.168.79.131 hadoop2

SSH免密码登录

进入当前用户的根目录/home/hadoop/.ssh/，在该目录下执行ssh-keygen -t rsa生成主机公钥，过程中弹出的除了 Overwrite (y/n)? y选择y确认覆盖之外，其余全为空值，则可生成免密码登录的公钥 【注意】Ubuntu系统操作略有特殊，建议使用CentOS系统进行操作

[hadoop@hadoop0 root]$ cd /home/hadoop/.ssh/
[hadoop@hadoop0 .ssh]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hadoop/.ssh/id_rsa):
/home/hadoop/.ssh/id_rsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/hadoop/.ssh/id_rsa.
Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:RjyGCGedx9xzxXG97KGgjXMIZ/pg7Q+tLU1nJoxIhm8 hadoop@hadoop0
The key's randomart image is:
+---[RSA 2048]----+
| . o. + . oo.o|
| + .oo+ o . ...|
| . o.= o . .|
| . * + . + |
| + S B . o .|
| E B.* = . |
| o +.=.= |
| ++. |
| .oo |
+----[SHA256]-----+

使用ssh-copy-id将本地主机的公钥复制到远程主机的authorized_keys文件上，此时先对本地主机hadoop0授权

[hadoop@hadoop0 .ssh]$ ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop0
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/hadoop/.ssh/id_rsa.pub"
The authenticity of host 'hadoop0 (fe80::e99d:ad75:28e9:9276%ens33)' can't be established.
ECDSA key fingerprint is SHA256:GEEH9kBVcDuaYpJZLNsaWZA3O47+HdqIbd91fGAWcew.
ECDSA key fingerprint is MD5:cd:df:9e:8c:92:3a:9f:ae:b0:10:9b:b6:25:9c:4d:c6.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
hadoop@hadoop0's password:

Number of key(s) added: 1

Now try logging into the machine, with: "ssh 'hadoop0'"
and check to make sure that only the key(s) you wanted were added.

测试SSH免密码登录成功

 [hadoop@hadoop0 /]$ ssh hadoop0
 Last login: Sat Apr 21 17:18:37 2018
 [hadoop@hadoop0 ~]$ exit
 登出
 Connection to hadoop0 closed.

对另外两台主机访问授权 hadoop1

 [hadoop@hadoop0 root]$ ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop1
 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/hadoop/.ssh/id_rsa.pub"
 The authenticity of host 'hadoop1 (192.168.79.130)' can't be established.
 ECDSA key fingerprint is SHA256:GEEH9kBVcDuaYpJZLNsaWZA3O47+HdqIbd91fGAWcew.
 ECDSA key fingerprint is MD5:cd:df:9e:8c:92:3a:9f:ae:b0:10:9b:b6:25:9c:4d:c6.
 Are you sure you want to continue connecting (yes/no)? yes
 /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
 /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
 hadoop@hadoop1's password:
 
 Number of key(s) added: 1
 
 Now try logging into the machine, with: "ssh 'hadoop1'"
 and check to make sure that only the key(s) you wanted were added.

hadoop2

 [hadoop@hadoop0 root]$ ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop2
 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/hadoop/.ssh/id_rsa.pub"
 The authenticity of host 'hadoop2 (192.168.79.131)' can't be established.
 ECDSA key fingerprint is SHA256:GEEH9kBVcDuaYpJZLNsaWZA3O47+HdqIbd91fGAWcew.
 ECDSA key fingerprint is MD5:cd:df:9e:8c:92:3a:9f:ae:b0:10:9b:b6:25:9c:4d:c6.
 Are you sure you want to continue connecting (yes/no)? yes
 /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
 /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
 hadoop@hadoop2's password:
 
 Number of key(s) added: 1
 
 Now try logging into the machine, with: "ssh 'hadoop2'"
 and check to make sure that only the key(s) you wanted were added.

对另外两台主机做类似操作，即仅对本地主机授权，无需对hadoop0主机授权，操作后可查看authorized_keys文件内容，当前主机名即可

环境变量

三台主机均由之前的单节点主机克隆而来，故宝塔、java、hadoop环境是自带的 Java8

 [hadoop@hadoop0 .ssh]$ java -version
 java version "1.8.0_121"
 Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
 Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
 [hadoop@hadoop0 .ssh]$ javac -version
 javac 1.8.0_121

hadoop

 [hadoop@hadoop0 .ssh]$ cd /www/hadoop
 [hadoop@hadoop0 hadoop]$
 [hadoop@hadoop0 hadoop]$ ls
 hadoop-2.6.0-cdh5.7.0 hadoop-2.6.0-cdh5.7.0.tar.gz testFile tmp
 [hadoop@hadoop0 hadoop]$ source ~/.bash_profile

$HADOOP_HOME

 [hadoop@hadoop0 hadoop]$ echo $HADOOP_HOME
 /www/hadoop/hadoop-2.6.0-cdh5.7.0

hadoop配置

若之前配置过hadoop，建议将原目录删除后重新解压，主节点和从节点的目录都要删除干净，否则会导致大量问题 配置主节点core-site.xml

 <configuration>
 <property>
 <name>fs.defaultFS</name>
 <value>hdfs://192.168.79.129:8020</value>
 </property>
 </configuration>

配置HDFS文件存放目录和节点数hdfs-site.xml

 <property>
 <name>dfs.namenode.name.dir</name>
 <value>/www/hadoop/app/tmp/dfs/name</value>
 </property>
 <property>
 <name>dfs.datanode.data.dir</name>
 <value>/www/hadoop/app/tmp/dfs/data</value>
 </property>

配置YARN的yarn.xml，删除单节点的配置，添加如下字段

 <configuration>
 
 <!-- Site specific YARN configuration properties -->
 <property>
 <name>yarn.nodemanager.aux-services</name>
 <value>mapreduce_shuffle</value>
 </property>
 
 	<property>
 		<name>yarn.log-aggregation-enable</name>
 		<value>true</value>
 	</property>
 
 <!--ResourceManager主机名-->
 <property>
 		<name>yarn.resourcemanager.hostname</name>
 		<value>hadoop0</value>
 	</property>
 
 </configuration>

配置从节点slave，默认slave为localhost

 [hadoop@hadoop0 hadoop]$ cat slaves
 localhost

加入host配置

 [hadoop@hadoop0 hadoop]$ cat slaves
 hadoop0
 hadoop1
 hadoop2

额外配置，启用jobhistory

 <configuration>
 <property>
 <name>mapreduce.framework.name</name>
 <value>yarn</value>
 </property>
 
 <property>
 		<name>mapreduce.jobhistory.address</name>
 		<value>192.168.79.129:10020</value>
 		<description>MapReduce JobHistory Server IPC host:port</description>
 	</property>
 
 	<property>
 		<name>mapreduce.jobhistory.webapp.address</name>
 		<value>192.168.79.129:19888</value>
 		<description>MapReduce JobHistory Server Web UI host:port</description>
 	</property>
 
 	<property>
 		<name>mapreduce.jobhistory.done-dir</name>
 		<value>/history/done</value>
 	</property>
 
 	<property>
 		<name>mapreduce.jobhistory.intermediate-done-dir</name>
 		<value>/history/done_intermediate</value>
 	</property>
 
 </configuration>

【注意】配置好后将单节点时的/www/hadoop/hadoop-2.6.0-cdh5.7.0/logs文件夹删除或者直接重新解压hadoop的压缩包防止出现文件夹权限导致的错误

资源分发

将主节点的所有配置分发到从节点，避免重复配置。在主节点执行如下命令即可

 scp -r /www/hadoop hadoop@hadoop1:/www
 scp -r /www/hadoop hadoop@hadoop2:/www

分发完成

去根节点查看文件是否已经传到，分别执行source ~/.bash_profile以启用hadoop命令

注意分发前执行一次hadoop命令是报未找到命令，说明从节点的hadoop旧根目录是删除干净的，分发完成后再次执行命令就有效了

NameNode格式化

格式化只对主节点执行即可

 hdfs namenode -format

格式化成功

hadoop分布式启动

主节点启动，在/sbin目录下，执行./start-all.sh

 [hadoop@hadoop0 bin]$ cd ../sbin
 [hadoop@hadoop0 sbin]$ ./start-all.sh
 This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
 18/04/21 22:16:18 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 Starting namenodes on [hadoop0]

从节点的datanode和namenode逐一启动

 hadoop0: starting namenode, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-namenode-hadoop0.out
 hadoop1: starting datanode, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-datanode-hadoop1.out
 hadoop0: starting datanode, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-datanode-hadoop0.out
 hadoop2: starting datanode, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-datanode-hadoop2.out
 Starting secondary namenodes [0.0.0.0]

主节点secondarynamenode启动

 0.0.0.0: starting secondarynamenode, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-secondarynamenode-hadoop0.out
 18/04/21 22:16:34 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

启动yarn和resourcemanager、nodemanager

 starting yarn daemons
 starting resourcemanager, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/yarn-hadoop-resourcemanager-hadoop0.out
 hadoop1: starting nodemanager, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/yarn-hadoop-nodemanager-hadoop1.out
 hadoop2: starting nodemanager, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/yarn-hadoop-nodemanager-hadoop2.out
 hadoop0: starting nodemanager, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/yarn-hadoop-nodemanager-hadoop0.out

hadoop分布式启动验证

命令行方式

使用jps查看进程，以下进程少一个都不行，若缺少，查看另一篇文章寻找解决办法hadoop分布式搭建的一些坑

[[hadoop分布式搭建的一些坑]]

启动主节点后从节点的HDFS会自动启动，进入从节点，使用jps查看，有两个进程即成功

 [hadoop@hadoop2 hadoop]$ jps
 29876 Jps
 29575 DataNode
 29692 NodeManager

webUI方式

使用webUI确认所有进程成功启动，查看HDFS节点

查看YARN

已经有三个从节点注册到YARN

分布式集群测试

分布式文件测试

上传测试文件再查看节点情况初始节点为空

 [hadoop@hadoop0 hadoop-2.6.0-cdh5.7.0]$ ls
 bin cloudera examples include libexec logs README.txt share
 bin-mapreduce1 etc examples-mapreduce1 lib LICENSE.txt NOTICE.txt sbin src
 [hadoop@hadoop0 hadoop-2.6.0-cdh5.7.0]$ hadoop fs -ls /
 18/04/21 22:33:27 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

新建数据目录

 [hadoop@hadoop0 hadoop-2.6.0-cdh5.7.0]$ hadoop fs -mkdir /data
 18/04/21 22:33:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 [hadoop@hadoop0 hadoop-2.6.0-cdh5.7.0]$ hadoop fs -ls /
 18/04/21 22:33:45 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 Found 1 items
 drwxr-xr-x - hadoop supergroup 0 2018-04-21 22:33 /data

上传小文件

 [hadoop@hadoop0 hadoop-2.6.0-cdh5.7.0]$ hadoop fs -put README.txt /data
 18/04/21 22:34:13 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 [hadoop@hadoop0 hadoop-2.6.0-cdh5.7.0]$ hadoop fs -ls /data
 18/04/21 22:34:26 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 Found 1 items
 -rw-r--r-- 3 hadoop supergroup 1366 2018-04-21 22:34 /data/README.txt

上传大文件

 [hadoop@hadoop0 hadoop]$ ll
 总用量 304284
 drwxrwxr-x 3 hadoop hadoop 17 4月 21 22:16 app
 drwxrwxrwx 15 www www 253 4月 21 22:08 hadoop-2.6.0-cdh5.7.0
 -rwxrwxrwx 1 www www 311585484 3月 31 23:06 hadoop-2.6.0-cdh5.7.0.tar.gz
 drwxr-xr-x 2 www www 301 4月 11 06:31 testFile
 [hadoop@hadoop0 hadoop]$ hadoop fs -put hadoop-2.6.0-cdh5.7.0.tar.gz /data
 18/04/21 22:35:59 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 [hadoop@hadoop0 hadoop]$ hadoop fs -ls /data
 18/04/21 22:36:19 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 Found 2 items
 -rw-r--r-- 3 hadoop supergroup 1366 2018-04-21 22:34 /data/README.txt
 -rw-r--r-- 3 hadoop supergroup 311585484 2018-04-21 22:36 /data/hadoop-2.6.0-cdh5.7.0.tar.gz

通过webUI查看文件情况

分布式MapReduce测试

测试MapReduce，进入/www/hadoop/hadoop-2.6.0-cdh5.7.0/share/hadoop/mapreduce目录，执行pi运算hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar pi 2 3 【注意】由于此处是虚拟机，不建议将后面的精度设置过大，否则会报IO错误 进入YARN的web界面查看

运行成功

分布式日志离线处理测试

上传日志数据 [hadoop@hadoop0 testFile]$ hadoop fs -put access.log /data 18/04/21 23:42:13 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable 执行

 hadoop jar hadoopstudy-0.6-jar-with-dependencies.jar com.fjy.hadoop.hadoop.project.LogApp /data/access.log /logaccess/out

Web查看工作详情

查看结果

题外话

其实同时运行三台虚拟机是很吃配置的，特别是内存，对于hadoop来说内存太小很多都会报异常，因此本次的测试机器配置还可以的，三台虚拟机均运行在480G SATA3的固态上 CPU:7700 4core 8thread RAM:16G DDR4 Dual Channal SSD:128+480 nvme+sata3 HDD:9T sata3 Eth:1000M I219V 但在实际运行中依然顶着内存瓶颈，多次使用了系统盘的交换分区

MapReduce对网站日志进行分析

Fri, 20 Apr 2018 22:58:00 +0800

代码

GitHub链接开源工具类UserAgentParser地址

日志来源

根据之前的作业提交系统项目的后台获取，为了本地测试，故先从数据库中备份下来并做了适当的处理，仅保留UserAgent的相关信息

本地文件测试

使用开源工具类解析UserAgent中的信息

@Test
 public void testReadFile() throws Exception {
 String path = "F:\\JAVA Workspace\\hadoopstudy\\access.log";
 BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(new File(path))));

 String line = "";
 int count = 0;

 //模拟MapReduce存储
 Map<String, Integer> browerMap = new HashMap<String, Integer>();

 UserAgentParser userAgentParser = new UserAgentParser();

 while (line != null) {
 line = reader.readLine();
 count++;
 if (StringUtils.isNotBlank(line)) {
 String source = line;
 UserAgent agent = userAgentParser.parse(source);
 //测试
 String browser = agent.getBrowser();
 String engine = agent.getEngine();
 String engineVersion = agent.getEngineVersion();
 String os = agent.getOs();
 String platform = agent.getPlatform();
 boolean mobile = agent.isMobile();
 Integer browserValue = browerMap.get(browser);
 
 if (browserValue != null) {
 browerMap.put(browser, browerMap.get(browser) + 1);
 } else {
 browerMap.put(browser, 1);
 }

 //输出解析的信息
 System.out.println(browser + "," + engine + "," + engineVersion + "," + os + "," + platform + "," + mobile);
 }

 }
 System.out.println("总记录数：" + count);
 System.out.println("====================================");
 for (Map.Entry<String,Integer>entry :browerMap.entrySet()){
 System.out.println(entry.getKey()+":"+entry.getValue());
 }
 }

执行单元测试

使用MapReduce统计

代码以WordCount为原型，结合本地测试的代码编写 … boolean mobile = agent.isMobile();

 if (mobile){
 mobileuser = "手机用户";
 }else {
 mobileuser = "非手机用户";
 }

 //通过上下文把map的处理结果输出
 context.write(new Text(browser), one);
...

打包运行

将工程用mvn assembly:assembly命令把插件一起打包，上传至虚拟机，同时将log上传至HDFS的根目录 [hadoop@localhost testFile]$ hadoop fs -put access.log / 18/04/11 06:12:14 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable [hadoop@localhost testFile]$ hadoop fs -ls / 18/04/11 06:12:21 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable Found 9 items -rw-r–r– 1 hadoop supergroup 73 2018-04-11 00:30 /PartitionerTest.txt -rw-r–r– 1 hadoop supergroup 26888 2018-04-11 06:12 /access.log drwxr-xr-x - hadoop supergroup 0 2018-04-08 10:56 /hdfsapi -rw-r–r– 1 hadoop supergroup 60 2018-04-10 23:03 /hello.txt drwxrwx— - hadoop supergroup 0 2018-04-11 01:28 /history drwxr-xr-x - hadoop supergroup 0 2018-04-11 00:35 /output drwxr-xr-x - hadoop supergroup 0 2018-04-08 10:35 /test drwx—— - hadoop supergroup 0 2018-04-11 01:36 /tmp drwxr-xr-x - hadoop supergroup 0 2018-04-09 20:11 /user 运行

结果与本地测试一致 [hadoop@localhost testFile]$ hadoop fs -ls /logaccess/browserout 18/04/11 06:15:18 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable Found 2 items -rw-r–r– 1 hadoop supergroup 0 2018-04-11 06:14 /logaccess/browserout/_SUCCESS -rw-r–r– 1 hadoop supergroup 32 2018-04-11 06:14 /logaccess/browserout/part-r-00000 [hadoop@localhost testFile]$ hadoop fs -text /logaccess/browserout/part-r-00000 18/04/11 06:15:32 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable Chrome 191 Firefox 10 Unknown 1 增加其他属性重新运行MapReduce，结果如下 [hadoop@localhost testFile]$ hadoop fs -ls /logaccess 18/04/11 06:33:11 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable Found 2 items -rw-r–r– 1 hadoop supergroup 0 2018-04-11 06:32 /logaccess/_SUCCESS -rw-r–r– 1 hadoop supergroup 192 2018-04-11 06:32 /logaccess/part-r-00000 [hadoop@localhost testFile]$ hadoop fs -text /logaccess/part-r-00000 18/04/11 06:33:21 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable 20100101 10 537.36 191 604.3.5 1 Android 48 Chrome 191 Firefox 10 Gecko 10 Linux 48 Unknown 1 Webkit 192 Windows 296 Windows 7 10 iPhone 1 iPhone OS 11.1 1 手机用户 49 非手机用户 153 日志离线统计完成！

单节点MapReduce开发

Tue, 17 Apr 2018 13:03:00 +0800

官方原理解析

MapReduce演进

MapReduce2.x的架构

MapReduce1.x的架构，一旦JobTracker挂了，整个分布式节点就崩溃了，因此被淘汰

开发WordCount

WordCount原理

数据输入到MapReduce，每个MapReduce任务都被初始化为一个Job，每个Job又可以分为两种阶段：map阶段和reduce阶段。map方法接收一个<key,value>形式的输入，key表示词组偏移量，value表示该偏移量对应的值，然后同样产生一个<key,value>形式的中间输出，Hadoop函数接收一个如<key,(list of values)>形式的输入，list of values即Java中的集合，用于表示相同偏移量下的字符统计数量，然后对这个value集合进行处理，每个Reduce方法产生0或1个输出，reduce的输出也是<key,value>形式的。

如果文本为

Hadoop Hello Hadoop

实际过程大概为：

Mapping：

<Hadoop ,1>，<Hello,1>，<Hadoop ,1>

Shuffling：

<Hadoop ,1>，<Hadoop ,1>
<Hello,1>

Reducing：

<Hadoop ,2>
<Hello,1>

多节点下的并行计算原理示意图

开发代码

查看源码

通过查看Mapper类的源码，发现官方已经定义的run方法，包含了所有基础的Mapper操作（模板方法设计模式），主要操作在map中，因此，只要重写map方法即可。

同理Reduce类中也有类似的定义

重写方法

这里只列举部分代码，其余所有代码可去本人的GitHub中查看：GitHub链接 /** * Map：读取输入文件 * Text:类似字符串 */ public static class MyMapper extends Mapper<LongWritable, Text, Text, LongWritable> {

 LongWritable one = new LongWritable(1);

 /**
 * @param key 偏移量
 * @param value 每行的字符串
 * @param context 上下文
 * @throws IOException
 * @throws InterruptedException
 */
 @Override
 protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
 /*super.map(key, value, context);*/

 //接收到每一行数据
 String line = value.toString();

 //按照指定分隔符进行拆分
 /*line.split("\t");//以Tab分隔*/
 String[] words = line.split(" ");//以空格分隔

 for (String word : words) {
 //通过上下文把map的处理结果输出
 context.write(new Text(word), one);
 }
 }
}

运行

通过宝塔面板上传到服务器，确保此时Hadoop所有服务均为启动状态

在HDFS中预先创建好要测试的文件，通过绝对路径的方法hadoop fs -ls hdfs://192.168.79.129:8020/确认文件是可以访问的

输入运行命令即可运行

 hadoop jar hadoopstudy-1.0-SNAPSHOT.jar com.fjy.hadoop.mapreduce.WordCountApp hdfs://192.168.79.129:8020/hello.txt hdfs://192.168.79.129:8020/output/wc

启动至FileInputFormat

 [hadoop@localhost testFile]$ hadoop jar hadoopstudy-1.0-SNAPSHOT.jar com.fjy.hadoop.mapreduce.WordCountA2.168.79.129:8020/hello.txt hdfs://192.168.79.129:8020/output/wc
 18/04/10 09:57:43 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 18/04/10 09:57:44 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
 18/04/10 09:57:45 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
 18/04/10 09:57:45 INFO input.**FileInputFormat**: Total input paths to process : 1

执行Job

 18/04/10 09:57:46 INFO mapreduce.JobSubmitter: number of splits:1
 18/04/10 09:57:46 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1523322221448_0001
 18/04/10 09:57:47 INFO impl.YarnClientImpl: Submitted application application_1523322221448_0001
 18/04/10 09:57:47 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1523322221448_0001/
 18/04/10 09:57:47 INFO mapreduce.Job: Running job: job_1523322221448_0001
 18/04/10 09:57:56 INFO mapreduce.Job: Job job_1523322221448_0001 running in uber mode : false
 18/04/10 09:57:56 INFO mapreduce.Job: map 0% reduce 0%
 18/04/10 09:58:04 INFO mapreduce.Job: map 100% reduce 0%
 18/04/10 09:58:10 INFO mapreduce.Job: map 100% reduce 100%
 18/04/10 09:58:12 INFO mapreduce.Job: Job job_1523322221448_0001 completed successfully
 18/04/10 09:58:12 INFO mapreduce.Job: Counters: 49
 File System Counters
 FILE: Number of bytes read=39
 FILE: Number of bytes written=222915
 FILE: Number of read operations=0
 FILE: Number of large read operations=0
 FILE: Number of write operations=0
 HDFS: Number of bytes read=113
 HDFS: Number of bytes written=17
 HDFS: Number of read operations=6
 HDFS: Number of large read operations=0
 HDFS: Number of write operations=2
 Job Counters

Map方法执行

 Launched map tasks=1
 Launched reduce tasks=1
 Data-local map tasks=1
 Total time spent by all maps in occupied slots (ms)=5920
 Total time spent by all reduces in occupied slots (ms)=3321
 Total time spent by all map tasks (ms)=5920
 Total time spent by all reduce tasks (ms)=3321
 Total vcore-seconds taken by all map tasks=5920
 Total vcore-seconds taken by all reduce tasks=3321
 Total megabyte-seconds taken by all map tasks=6062080
 Total megabyte-seconds taken by all reduce tasks=3400704
 Map-Reduce Framework
 Map input records=1
 Map output records=2
 Map output bytes=29
 Map output materialized bytes=39

Reduce 方法执行

 Input split bytes=101
 Combine input records=0
 Combine output records=0

 Reduce input groups=2
 Reduce shuffle bytes=39
 Reduce input records=2
 Reduce output records=2
 Spilled Records=4
 Shuffled Maps =1
 Failed Shuffles=0
 Merged Map outputs=1
 GC time elapsed (ms)=115
 CPU time spent (ms)=1080
 Physical memory (bytes) snapshot=300331008
 Virtual memory (bytes) snapshot=5493092352
 Total committed heap usage (bytes)=165810176

Shuffle 方法执行

 Shuffle Errors
 BAD_ID=0
 CONNECTION=0
 IO_ERROR=0
 WRONG_LENGTH=0
 WRONG_MAP=0
 WRONG_REDUCE=0
`File Input`和`File Output` 方法执行 
 File Input Format Counters
 Bytes Read=12
 File Output Format Counters
 Bytes Written=17

运行结果如图

进一步优化

使用Combiner

使用Combiner组件可以在Mapping时就先进行一次聚合，再发送到Shuffling继续运算，从而提高运算效率在主方法中进行如下配置即可

 //通过job设置combiner处理类，逻辑上与reduce一致，注意，如果要计算平均数等不能使用Combiner!
 job.setCombinerClass(MyReducer.class);

编写完后使用Maven重新打包

 mvn clean package -DskipTests

上传到服务器后再次执行计算命令

 hadoop jar hadoopstudy-0.3.jar com.fjy.hadoop.mapreduce.WordCountApp hdfs://192.168.79.129:8020/hello.txt hdfs://192.168.79.129:8020/output/wc

查看图形界面可观察是否完成

通过查看代码可发现Combine input records已经不为0(之前未使用Combiner 时该值为0)，说明使用成功

 Map-Reduce Framework
 Map input records=4
 ...
 Combine input records=9
 Combine output records=5
 ...
 Physical memory (bytes) snapshot=298844160
 Virtual memory (bytes) snapshot=5493088256
 Total committed heap usage (bytes)=165810176
 Shuffle Errors
 BAD_ID=0
 ...

查看结果

 [hadoop@localhost testFile]$ hadoop fs -cat /output/wc/part-r-00000
 18/04/10 23:06:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 ! 1
 Hadoop 3
 Hello 2
 MapReduce 2
 YARN 1

使用Partitioner组件

作用

Partitioner决定MapTask输出的数据交由哪个ReduceTask处理，其默认实现为分发的key的hash值对ReduceTask个数取模

文件准备

在宝塔测试目录/www/hadoop/testFile下准备好PartitionerTest.txt文件，文件内容如图，将其上传到HDFS根目录中

 [hadoop@localhost root]$ cd /www/hadoop/testFile
 [hadoop@localhost testFile]$ ls
 PartitionerTest.txt hello.txt
 [hadoop@localhost testFile]$ hadoop fs -put PartitionerTest.txt /
 18/04/11 00:30:31 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 [hadoop@localhost testFile]$ hadoop fs -ls /
 18/04/11 00:30:39 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 Found 7 items
 -rw-r--r-- 1 hadoop supergroup 73 2018-04-11 00:30 /PartitionerTest.txt
 drwxr-xr-x - hadoop supergroup 0 2018-04-08 10:56 /hdfsapi
 -rw-r--r-- 1 hadoop supergroup 60 2018-04-10 23:03 /hello.txt
 drwxr-xr-x - hadoop supergroup 0 2018-04-10 23:05 /output
 drwxr-xr-x - hadoop supergroup 0 2018-04-08 10:35 /test
 drwx------ - hadoop supergroup 0 2018-04-09 20:11 /tmp
 drwxr-xr-x - hadoop supergroup 0 2018-04-09 20:11 /user
 [hadoop@localhost testFile]$ hadoop fs -text /PartitionerTest.txt
 18/04/11 00:30:59 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 xiaomi 200
 huawei 300
 xiaomi 100
 huawei 200
 iphone 300
 iphone 200
 sony 50

代码编写

主要对分隔符和上下文内容进行修改，map类中修改为

 //每一个空格是一个手机品牌，另一个是销售数量
 context.write(new Text(words[0]), new LongWritable(Long.parseLong(words[1])));

新增Partitioner处理类

 /**
 * Partitioner处理类
 */
 public static class MyPartitioner extends Partitioner<Text,LongWritable>{
 @Override
 public int getPartition(Text key, LongWritable value, int i) {

 if ("xiaomi".equals(key.toString())) {
 return 0;//若为xiaomi则交由0 ReduceTask处理
 }
 if ("huawei".equals(key.toString())) {
 return 1;//若为huawei则交由1 ReduceTask处理
 }
 if ("iphone".equals(key.toString())) {
 return 2;//若为iphone则交由2 ReduceTask处理
 }

 return 3;//若为其他，则交由3 ReduceTask处理
 }
 }

最后在驱动中配置Partitioner

 //设置job的Partition
 job.setPartitionerClass(MyPartitioner.class);
 //设置四个reducer，每个分区一个，否则Partitioner配置不生效
 job.setNumReduceTasks(4);

通过复制类名全路径（以IDEA为例），修改运行命令

此时，运行命令为

 hadoop jar hadoopstudy-0.4.jar com.fjy.hadoop.mapreduce.WordCountPartitionerApp hdfs://192.168.79.129:8020/PartitionerTest.txt hdfs://192.168.79.129:8020/output/partitioner

执行

再次用Maven打包并上传到测试目录运行运行前：

运行中：

运行结束

查看结果内容：

 [hadoop@localhost testFile]$ hadoop fs -ls /output/partitioner
 18/04/11 00:36:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 Found 5 items
 -rw-r--r-- 1 hadoop supergroup 0 2018-04-11 00:35 /output/partitioner/_SUCCESS
 -rw-r--r-- 1 hadoop supergroup 11 2018-04-11 00:35 /output/partitioner/part-r-00000
 -rw-r--r-- 1 hadoop supergroup 11 2018-04-11 00:35 /output/partitioner/part-r-00001
 -rw-r--r-- 1 hadoop supergroup 11 2018-04-11 00:35 /output/partitioner/part-r-00002
 -rw-r--r-- 1 hadoop supergroup 8 2018-04-11 00:35 /output/partitioner/part-r-00003
 [hadoop@localhost testFile]$ hadoop fs -text /output/partitioner/part-r-00000
 18/04/11 00:36:53 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 xiaomi 300
 [hadoop@localhost testFile]$ hadoop fs -text /output/partitioner/part-r-00001
 18/04/11 00:36:58 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 huawei 500
 [hadoop@localhost testFile]$ hadoop fs -text /output/partitioner/part-r-00002
 18/04/11 00:37:05 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 iphone 500
 [hadoop@localhost testFile]$ hadoop fs -text /output/partitioner/part-r-00003
 18/04/11 00:37:12 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 sony 50

结果正确！

开启JobHistory

JobHistory作用

记录已经运行完的MapReduce信息到指定的HDFS目录下，但默认不开启因此此时不配置直接访问如下

![jobhistory][33]

配置JobHistory

在mapred-site.xml加入配置

 <!-- jobhistory地址-->
	<property>
		<name>mapreduce.jobhistory.address</name>
		<value>192.168.79.129:10020</value>
		<description>MapReduce JobHistory Server IPC host:port</description>
	</property>
 <!-- jobhistory web的地址-->
	<property>
		<name>mapreduce.jobhistory.webapp.address</name>
		<value>192.168.79.129:19888</value>
		<description>MapReduce JobHistory Server Web UI host:port</description>
	</property>
 <!-- 作业运行完存放地址-->
	<property>
		<name>mapreduce.jobhistory.done-dir</name>
		<value>/history/done</value>
	</property>
 <!-- 作业运行中存放地址-->
	<property>
		<name>mapreduce.jobhistory.intermediate-done-dir</name>
		<value>/history/done_intermediate</value>
	</property>

配置完后重启YARN

 [hadoop@localhost mapreduce]$ cd /www/hadoop/hadoop-2.6.0-cdh5.7.0/sbin
 [hadoop@localhost sbin]$ ./stop-yarn.sh
 stopping yarn daemons
 stopping resourcemanager
 localhost: stopping nodemanager
 no proxyserver to stop
 [hadoop@localhost sbin]$ jps
 2353 DataNode
 80224 Jps
 2228 NameNode
 2522 SecondaryNameNode
 [hadoop@localhost sbin]$ ./start-yarn.sh
 starting yarn daemons
 starting resourcemanager, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/yarn-hadoop-resourcemanager-localhost.localdomain.out
 localhost: starting nodemanager, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/yarn-hadoop-nodemanager-localhost.localdomain.out
 [hadoop@localhost mapreduce]$ jps
 2353 DataNode
 80336 ResourceManager
 2228 NameNode
 81510 Jps
 2522 SecondaryNameNode
 63582 JobHistoryServer
 80447 NodeManager

启动jobhistory服务

 [hadoop@localhost sbin]$ ./mr-jobhistory-daemon.sh start historyserver
 starting historyserver, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/mapred-hadoop-historyserver-localhost.localdomain.out

运行PI测试

 [hadoop@localhost sbin]$ cd /www/hadoop/hadoop-2.6.0-cdh5.7.0/share/hadoop/mapreduce
 [hadoop@localhost mapreduce]$ hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar pi 1 2

访问报错

进入日志查看，发现仍然无法访问

报未开启聚合的错误，同时发现指向跳转链接为localhost

增加配置

因此在yarn-site.xml加入配置

 <!-- 开启YARN log的聚合功能-->
	<property>
		<name>yarn.log-aggregation-enable</name>
		<value>true</value>
	</property>

同时，为了让虚拟机的跳转链接不为localhost，此处再加入一段配置，制定IP

 <property> 
 <name>yarn.nodemanager.hostname</name> 
 <value>192.168.79.129</value> 
 </property>

重启服务

再次重启YARN，并同时重启jobhistory服务

 [hadoop@localhost sbin]$ ./mr-jobhistory-daemon.sh stop historyserver
 stopping historyserver
 [hadoop@localhost sbin]$ ./stop-yarn.sh
 stopping yarn daemons
 stopping resourcemanager
 localhost: stopping nodemanager
 no proxyserver to stop
 [hadoop@localhost sbin]$ ./start-yarn.sh
 starting yarn daemons
 starting resourcemanager, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/yarn-hadoop-resourcemanager-localhost.localdomain.out
 localhost: starting nodemanager, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/yarn-hadoop-nodemanager-localhost.localdomain.out
 [hadoop@localhost sbin]$ ./mr-jobhistory-daemon.sh start historyserver
 starting historyserver, logging to /www/hadoop/hadoop-2.6.0-cdh5.7.0/logs/mapred-hadoop-historyserver-localhost.localdomain.out

【注意】此时若只重启YARN就算配置正确，仍然可能出现Aggregation is not enabled的错误

验证

再次执行PI运算

 [hadoop@localhost sbin]$ cd /www/hadoop/hadoop-2.6.0-cdh5.7.0/share/hadoop/mapreduce
 [hadoop@localhost mapreduce]$ hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar pi 1 2

查看Logs记录

已经有日志记录，配置成功！

阿里云CDN加速的一些经历

Mon, 09 Apr 2018 22:54:00 +0800

问题产生

最近有一个新的业务系统上线运行，源站使用8085端口，通过Nginx反向代理到80和443端口，为了保证全国的延迟在50ms以内，故启用CDN加速，仅对js和css等静态文件加速，其余文件自动回源，然而一旦开启了静态文件CDN后，网页便无法渲染（Vue），通过控制台查看，报重定向（301）过多的错误

一旦关闭CDN对js和css的加速后再清除浏览器缓存就立刻可以访问

寻找解决办法

经过网上查阅资料无果后向阿里云提交工单咨询

过程十分曲折，在阿里云工程师的指引下反复修改CDN各种配置，从上午一直修改到晚上10点，更换了3个工程师（从工号判断），其中还包括阿里工程师直接打电话询问问题情况（好评）测试途径包括但不限于：跟随回源协议、抓包等

目前折衷方案

根据阿里的建议，现在只能暂停对静态文件的加速，全部回源到源站

造成的影响

2018年4月9日全天整个业务系统通过域名访问处于崩溃状态，对所有需要访问的客户重定向到IP+端口号的地址，并且该地址还一直被微信拦截（微信拦截非80端口的网址）

HDFS伪分布式单节点搭建

Sat, 07 Apr 2018 13:01:00 +0800

环境

系统：CentOS Linux 7.4.1708 (Core) 客户端：putty 服务器：虚拟机1core、2G RAM 同时搭建了宝塔面板（自动配置了java8）

Hadoop下载

CDH版本Hadoop下载

为了统一版本，全部使用hadoop-2.6.0-cdh5.7.0

点击可进入官方推荐配置页

解压并配置

将下载好的文件通过宝塔上传到服务器，这里选择上传到/www/hadoop目录

解压tar包

配置

新建用户

新建hadoop用户，忽略密码强度提示

 adduser hadoop
 passwd hadoop

为方便远程连接和学习，同时授予root权限，输入whereis sudoers查看权限控制文件位置

 [root@fjy ~]# whereis sudoers
 sudoers: /etc/sudoers /etc/sudoers.d /usr/libexec/sudoers.so /usr/share/man/man5/sudoers.5.gz

使用宝塔找到并修改/etc/sudoers文件

配置新用户免密码远程登录

切换至hadoop用户，使用 ssh-keygen -t rsa生成私钥信息，位置在/home/hadoop/，并且忽略所有提示输入的信息

查看生成的文件，该SSH文件夹为隐藏文件夹，故用ls -a查看，可见自动生成了.ssh目录和id_rsa,id_rsa.pub两个文件

复制公钥到.ssh目录

 cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys

查看JAVA目录

因为是宝塔自动配置的java8，所以要首先查看当前java环境变量位置，使用如下命令

 update-alternatives --config java

此时宝塔自动将java配置到/usr/java/jdk1.8.0_121/jre/bin/java目录即/usr/java/jdk1.8.0_121为java环境变量的目录

配置hadoop的java目录

使用宝塔进入hadoop目录下的etc/hadoop/hadoop-env.sh，对应此处的绝对路径应为/www/hadoop/hadoop-2.6.0-cdh5.7.0/etc/hadoop 使用宝塔编辑文件，加入java环境变量位置

配置分布式文件参数

打开core-site.xml，配置如下

 <configuration>
 <property>
 <name>fs.defaultFS</name>
 <!-- 配置主机IP-->
 <value>hdfs://192.168.79.129:8020</value>
 </property>
 
 <!-- 配置临时文件位置-->
 <property>
 <name>hadoop.tmp.dir</name>
 <value>/www/hadoop/tmp</value>
 </property>
 </configuration>

打开hdfs-site.xml，配置如下

 <configuration>
 <property>
 <name>dfs.replication</name>
 <!-- 配置副本数量-->
 <value>1</value>
 </property>
 </configuration>

启动

初始化

进入/www/hadoop/hadoop-2.6.0-cdh5.7.0/bin目录，使用如下语句初始化Hadoop，该操作仅执行一次即可，以后使用无需重复执行，否则执行后会删除当前hadoop上的所有文件

 hadoop namenode -format

滚动均为info级别的日志

可看到已初始化成功

启动HDFS

切换到hadoop用户，进入/www/hadoop/hadoop-2.6.0-cdh5.7.0/sbin目录

输入./start-dfs.sh启动HDFS服务

【注意】，要提前在root用户下对/www/hadoop授予hadoop用户读写权限，否则无法启动

使用jps查看当前的java进程，若出现如下四个进程表示HDFS服务启动成功

外部访问浏览器客户端

首先使用宝塔开放50070端口

使用外部浏览器访问ip:50070可查看如下页面，则表示HDFS已成功启动

hdfs命令

配置hdfs命令

使用vi ~/.bash_profile进入配置文件，按如下配置

 export HADOOP_HOME=/www/hadoop/hadoop-2.6.0-cdh5.7.0
 export PATH=$HADOOP_HOME/bin:$PATH

使配置文件生效source ~/.bash_profile 输入hdfs回车测试命令是否生效，若出现如下画面则表示命令已经生效

命令测试

先在本地创建hello.txt文件，使用 hadoop fs -put hello.txt /命令将文件推送到hdfs根目录下

查看服务器根目录hadoop fs -ls /，已经有该文件，hadoop伪分布式配置完成

Typecho Pinghsu 主题加入备案信息

Sat, 17 Mar 2018 00:01:00 +0800

修改前

一般安装好typecho和主题后默认的底部信息如下，此时发现没有显示ICP备案信息

修改目的

根据国家法律法规要求备案后，是需要在页面底部增加ICP备案信息和链接的，否则将可能会面临相应的处罚

修改教程

首先以admin账号登录控制台，选择修改外观

选择编辑当前外观，打开footer.php

<p>ICP<a href="http://www.miitbeian.gov.cn/" target="_blank"> 粤ICP备18026XXX号</a></p>

最后点击保存即可

最终效果

访问有ICP备案号的博客链接

班级订书系统

Fri, 02 Mar 2018 10:00:00 +0800

订书系统介绍

订书系统（点击访问）访问后使用学号+姓名即可登录

登录后能够查看全班订书情况，总价为服务器自动计算

点击添加/修改即可为自己增加或修改订书信息

同时每位同学可查看后台提供的书籍价格和对应的课程名

也可查看实时公告信息

后台人员登录后可以修改价格，查看各类订书总数，对订书时间进行控制，当时间超过所设定日期后则不允许订书

后台也可以查看所有IP登录日志和设备信息，用于判断异地登录和系统审计，顺带一提，后面真有一次有人盗用账号进行登录修改，通过IP审计的方式找出登录地点和操作人，成功解决非法修改信息的问题

经过两个版本的更新后，系统增加了支付信息，后台可以根据支付宝记录更改支付状态

用户通过点击价格表，扫描其中的支付宝二维码即可支付书费，支付一段时间后经过确认会更新支付状态

开发进度

经过最后两次更新后，因多种原因，开发人员打算放弃开发和维护，愿有意接手维护和开发的人员与作者联系

已知BUG

价格计算页面因二进制移位问题，导致其中一位同学的价格产生多位小数的错误

作业提交系统介绍

Tue, 27 Feb 2018 19:50:00 +0800

起因

在2017年下半学年收集作业的过程中，遇到经常需要统一命名格式的情况，十分繁琐，也有出现许多深夜、凌晨提交作业的情况，同时也有不少要求找回以往提交的作业的情况，故考虑开发作业提交系统，满足作业收集需求

系统成品

作业提交系统正式版（点击访问正式版网站本体）

作业提交系统测试服务器版本（点击访问测试服务器网站本体）

系统使用说明（点击访问）

系统优点

系统主要解决以下几大核心问题

全天候接受作业提交并及时作出响应
作业自动按学号+姓名命名
作业按列表进行展示
用户可下载以往提交的作业和模板
后台自动获取所有未提交人员名单

系统延迟和访问速度

在购买了阿里云CDN加速后，通过在线网络测试工具得到如下结果全球的平均延迟在50ms以内

同时进行国内多地访问时的下载速度能稳定在100K以上，若遇到打开缓慢请稍等片刻或者刷新页面

开发准备

确定立项后首先对所需架构进行选型，考虑范围包括：

传统Java MVC、Struct、SpringMVC、SpringBoot、Mybatis、Hibernate bootstrap4.0、Vue、ElementUI、jQuery、原生ajax、axios、vue-resource thymeleaf、JSP

最后根据各方优缺点决定使用Java开发 Java版本（使用了java8 的新特性，故不对低版本的jdk兼容）

Java 8或以上

后台使用

Spring Boot2.0RC2

数据库层使用

SpringJpa

前端界面组件使用

ElementUI

前端底层数据响应式框架

Vue ajax使用Vue官方推荐的库 axios

开发过程

开发系统生命周期选型采用螺旋模型，通过反复迭代的方式进行持续改进由于切换新的架构（以前订书系统是纯jsp处理所有业务，Java MVC对于注解使用十分少），同时注重对数据库的范式设计（3NF），因此在开发准备花了不少时间，以及在后续开发遇到不少问题，耗时极大，根据私人Git仓库GitLab的记录，2018年1月29号开始代码编写V0.1，直到2018年2月27号版本号V2.2 RC1网站正式上线公测持续接近1个月时间，目前（2018年2月28日）仍在对bug进行测试和修复 GitLab截图

后台部分代码截图

所有工程文件使用Git进行版本配置管理，并上传到私人代码仓库

所有jar包使用maven进行统一管理

下载的Controller

所有Controller

通过使用枚举对异常统一处理，并编写自定义异常类

对日志进行科学管理，使用SLF4j、Logback 日志框架对所有info和error日志进行每天一个文件的日志输出

单元测试

前端部分代码截图

使用Spring Boot官方推荐的thymeleaf模板引擎，其性能比JSP更优，而且也有与JSTL类似的语法，可以轻松实现组件化和动态更新数据，并能够保证前端所有页面统一为纯HTML

axios异步获取后台传来的json数据，并使用Vue做数据绑定

前端使用饿了么的ElementUI组件进行快速开发，遗憾的是未使用ES6和Webpack

访问权限控制

通过使用Spring Boot拦截器，实现访问控制列表（Access Control List，ACL）与强制访问控制(Mandatory Access Control——MAC)结合的访问控制机制，对所有非注册用户和非管理员用户进行分辨和拦截

开发文档截图

后台提供的API文档

Restful风格的API由后台提供，前端可直接使用，实现前后端分离开发

后台提供的错误码文档

数据字典

数据库ER图

按照3NF规范进行开发，充分使用数据库视图作为外模式进行访问

服务器配置

硬件服务器

服务器采用由阿里云购买的轻量应用服务器，配置如下

1核CPU Intel Xeon E5-2682 v4
2G DDR4 内存
1M上行带宽
VPC专有网络 
40GB系统盘 SSD云盘

硬件服务器评价

整体配置基本满足系统运行需求，根据Beta测试阶段的数据，高并发时单核CPU瓶颈会显现，内存维持50%左右的水平，磁盘读写也令人满意，上行带宽略有不足，但由于为了保证系统高并发下的稳定性优先，以及系统主要用于用户上传，服务器下载速度没有瓶颈即可

服务器的安全策略

服务器的安全策略为服务器每天凌晨自动备份网站存储的文件和数据库到阿里云OSS和又拍云FTP存储空间，确保收到攻击后能以最快速度恢复所有用户文件和数据

★[2018/03/14 04:31:01] 数据库[db_imis2]已成功备份到阿里云OSS,用时[0.21]秒
|---保留最新的[3]份备份
|---文件名:Db_db_imis2_20180314_043101.sql.gz
|---已清理过期备份文件：Db_db_imis2_20180311_043102.sql.gz
★[2018/03/15 04:31:02] 数据库[db_imis2]已成功备份到阿里云OSS,用时[0.21]秒
|---保留最新的[3]份备份
|---文件名:Db_db_imis2_20180315_043101.sql.gz
|---已清理过期备份文件：Db_db_imis2_20180312_043101.sql.gz

★[2018/03/12 04:30:01] 网站[cms.fjy8018.xin]已成功备份到阿里云OSS,用时[0.23]秒
|---保留最新的[2]份备份
|---文件名:Web_cms.fjy8018.xin_20180312_043001.tar.gz

Web服务器

考虑到业务需要轻量级，占有内存少，并发能力强的软件平台，故服务器使用nginx1.12作为Web服务器，并通过配置实现反向代理和负载均衡

upstream cms-host{
server cms.fjy8018.xin:8085; #负载均衡
}

 server {
 listen 80;
 listen 443 ssl;
 server_name cms.fjy8018.xin; #域名
 index index.html; #指定的server的root的访问页面
 root /index; #指定的server的root目录

 if ($server_port !~ 443){
 rewrite ^(/.*)$ https://$host$1 permanent;
 } 
 location /cms {
 proxy_pass http://cms.fjy8018.xin:8085;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header X-Forwarded-Proto $scheme;
 proxy_set_header X-Forwarded-Port $server_port;
 # root html;
 # index index.html index.htm;
 }
 }

但由于没有使用分布式文件系统，故实际负载仍然为一台阿里云服务器

域名备案

为了保证微信能够直接访问，故通过购买万网域名fjy8018.xin，在通信管理局进行ICP备案，经过6天的等待，终于获得ICP备案和备案号

启用HTTPS

通过免费获取TrustAsia证书，并在nginx中加入pem文件路径配置，让网址成功启用HTTPS安全协议 #HTTP_TO_HTTPS_END ssl_certificate XXX/cms/full_chain.pem; ssl_certificate_key XXX/cms/private.key; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5:!RC4:!DHE; ssl_prefer_server_ciphers on; ssl_session_cache shared:SSL:10m; ssl_session_timeout 10m; error_page 497 https://$host$request_uri;

 #SSL-END

开源想法

暂无对外公开源码的想法，若本班有感兴趣的可以一同交流学习，如有兴趣接手维护或者开发新的网页端、安卓端和IOS端的也可以和本人联系，本人将提供后台API文档。

作业提交系统使用说明

Tue, 27 Feb 2018 15:58:00 +0800

地址

不推荐使用手机访问或者使用微信内置浏览器访问作业提交系统（点击访问）

系统介绍（点击查看）

注册

点击注册选项卡进行注册，若用户名已存在会返回相应的提示，若注册用户非15信管2班学生则不允许注册

注册时务必牢记密码，任何人无法通过后台查询用户的密码，若容易忘记密码，建议登录后立即进入个人中心设置密保问题

作业上传

点击图中箭头位置的下拉按钮，展开作业列表可见作业详情

点击模板下载下方的链接可下载模板文件（edge浏览器下载文件名会乱码，文件内容不受影响）

点击选取文件按钮可选择上传文件

点击上传到服务器即可上传，若要取消上传则点击选中文件右侧的“×”即可

上传成功后会自动提示成功消息，并于1秒后自动刷新已提交文件列表

若文件已存在，则自动将源文件后缀加上.日期.bak，新文件自动按照学号+姓名命名 注意：操作按钮中的“删除”操作不会有“是否确认删除的提示”，请谨慎操作

修改个人信息和密保问题

点击个人中心，可在里面重置个人信息和密码

个人中心里可设置或修改密保问题，用于找回密码，问题答案可放心设置，网站对所有问题答案均进行SHA安全加密

关于页面

点击关于按钮和查看网站相关信息

点击“查看更新日志”可看到网站历史版本和所有内部技术更新信息

意见反馈

进入意见反馈和提交对网站的意见和建议以及遇到的bug，欢迎多多反馈，当然也可以给出对该网站的评价~

登出

建议所有用户使用完后主动点击登出按钮销毁会话，登出后若不主动进行登录操作，任何人无法看到主页内容

已知兼容性问题

已对chrome进行完美测试，测试通过已对QQ浏览器、edge浏览器、火狐浏览器进行登录测试，测试通过已对360极速浏览器进行登录测试，必须在地址栏右侧选择“极速模式”才能正常访问已对微信内置浏览器进行登录测试，微信会对非80端口的网站进行拦截，严重影响使用体验，不推荐使用微信内置浏览器进行访问

网站未对手机显示进行优化，请使用电脑访问以获得最佳体验

已知bug

edge浏览器下载文件标题乱码

版权

该网站完全由符嘉阳自主开发，运行于国内家庭服务器中

作业提交系统

Sun, 04 Feb 2018 12:48:00 +0800

预告

2018-2-4 作业提交系统（班级管理系统）正式立项预告图

使用宝塔运维Ubuntu 16.04

Tue, 23 Jan 2018 13:11:00 +0800

系统环境

硬件：树莓派3

CPU：四核1.2GHz Broadcom BCM2837 x64 RAM：1GB TF容量：32G U1速度操作系统：ubuntu-mate-16.04.2-desktop-armhf-raspberry-pi 通过简单的方法可烧录TF卡，系统自动安装配置

终端输入sudo raspi-config启动树莓派配置工具，启动ssh服务并开机启动操作步骤参考：玩转树莓派之Ubuntu MATE for the Raspberry 系统安装

安装宝塔

官方安装要求安装要求：内存：128M以上，推荐512M以上（纯面板约占系统10M内存）硬盘：100M以上可用硬盘空间（纯面板约占20M磁盘空间）系统：暂时只对 CentOS 6.x / 7.x /Ubuntu/Debian 确保是干净的操作系统，没有安装过其它环境带的Apache/Nginx/php/MySQL

下面使用putty登录PI，安装宝塔

自动运行如下

root@fjy-desktop:/# wget -O install.sh http://download.bt.cn/install/install-ubuntu.sh && sudo bash install.sh –2018-01-23 13:13:41– http://download.bt.cn/install/install-ubuntu.sh Resolving download.bt.cn (download.bt.cn)… 103.224.251.67 Connecting to download.bt.cn (download.bt.cn)|103.224.251.67|:80… connected. HTTP request sent, awaiting response… 200 OK Length: 13572 (13K) [application/octet-stream] Saving to: ‘install.sh’

install.sh 100%[===================>] 13.25K –.-KB/s in 0s

2018-01-23 13:13:41 (52.5 MB/s) - ‘install.sh’ saved [13572/13572]

+———————————————————————- | Bt-WebPanel 5.x FOR Ubuntu/Debian +———————————————————————- | Copyright © 2015-2017 BT-SOFT(http://www.bt.cn) All rights reserved. +———————————————————————- | The WebPanel URL will be http://SERVER_IP:8888 when installed. +———————————————————————- 下面选择确定，输入小写y Do you want to install Bt-Panel to the /www directory now?(y/n): y 然后等待系统自动安装依赖

安装完成后会显示如下信息，提供了出事登录用户名和IP地址 ================================================================== Congratulations! Install succeeded! ================================================================== Bt-Panel: http://113.83.XX.XXX:8888 username: admin password: 0df18173 Warning: If you cannot access the panel, release the following port (8888|888|80|443|20|21) in the security group ================================================================== Time consumed: 12 Minute!

但ip地址一般都是不对的，请自行输入ip addr查询主机的ip地址

 root@fjy-desktop:/# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
 inet 127.0.0.1/8 scope host lo
 valid_lft forever preferred_lft forever
 inet6 ::1/128 scope host
 valid_lft forever preferred_lft forever
2: enxb827eb9e1f8b: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
link/ether b8:27:eb:9e:1f:8b brd ff:ff:ff:ff:ff:ff
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
 link/ether b8:27:eb:cb:4a:de brd ff:ff:ff:ff:ff:ff
 inet 192.168.31.9/24 brd 192.168.31.255 scope global dynamic wlan0
 valid_lft 30658sec preferred_lft 30658sec
 inet6 fe80::ab44:81:1f49:ce98/64 scope link
 valid_lft forever preferred_lft forever

由上可知192.168.31.9就是我树莓派的局域网地址，因此在浏览器登录的地址为http://192.168.31.9:8888

输入默认账号密码即可登录，登录后进入如下页面，提示安装全套web服务组件

这里选择安装LAMP组件套装，因为后面打算安装私有云服务（nextcloud），nextcloud对nginx支持并不友好，需要负载均衡的可以安装LNMP

点击一键安装即可安装状态可以点击左上角的标号查看

点击监软件管理，可以安装其他软件，包括Tomcat、gitlab等，安装Tomcat还会自动配置java和javac环境，可以说是非常的方便

以后要搭建java web服务，因此这里选择安装Tomcat，版本选择8

最后点击监控面板，打开服务器状态监控

一段时间后可查看服务器状态曲线图

有时候安装完成后发现MySQL服务无法被宝塔安装，那么可以到putty中使用命令行自行安装，输入sudo apt-get install mysql-server

root@fjy-desktop:/home/fjy# apt-get install mysql-server 正在读取软件包列表… 完成正在分析软件包的依赖关系树正在读取状态信息… 完成将会同时安装下列软件： libaio1 libcgi-fast-perl libcgi-pm-perl libevent-core-2.0-5 libfcgi-perl libhtml-template-perl mysql-client-5.7 mysql-client-core-5.7 mysql-common mysql-server-5.7 mysql-server-core-5.7 建议安装： libipc-sharedcache-perl mailx tinyca 下列【新】软件包将被安装： libaio1 libcgi-fast-perl libcgi-pm-perl libevent-core-2.0-5 libfcgi-perl libhtml-template-perl mysql-client-5.7 mysql-client-core-5.7 mysql-common mysql-server mysql-server-5.7 mysql-server-core-5.7 升级了 0 个软件包，新安装了 12 个软件包，要卸载 0 个软件包，有 249 个软件包未被升级。需要下载 17.7 MB 的归档。解压缩后会消耗 140 MB 的额外空间。您希望继续执行吗？ [Y/n] Y 之后会弹出如下对话框，输入默认数据库管理员密码即可

有时候安装后会发现Tomcat无法正常启动，进入putty用命令javac，发现未安装jdk

root@fjy-desktop:/var/lib/mysql# javac The program ‘javac’ can be found in the following packages:

default-jdk

ecj

gcj-5-jdk

openjdk-8-jdk-headless

gcj-4.8-jdk

gcj-4.9-jdk Try: apt install root@fjy-desktop:/var/lib/mysql# java -version openjdk version “1.8.0_151” OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12) OpenJDK Client VM (build 25.151-b12, mixed mode)

则输入sudo apt-get install openjdk-8-jdk安装jdk环境

root@fjy-desktop:/var/lib/mysql# apt-get install openjdk-8-jdk 正在读取软件包列表… 完成正在分析软件包的依赖关系树正在读取状态信息… 完成将会同时安装下列软件： fonts-dejavu-extra libgif7 libice-dev libpthread-stubs0-dev libsm-dev libx11-dev libx11-doc libxau-dev libxcb1-dev libxdmcp-dev libxt-dev openjdk-8-jdk-headless openjdk-8-jre x11proto-core-dev x11proto-input-dev x11proto-kb-dev xorg-sgml-doctools xtrans-dev 建议安装： libice-doc libsm-doc libxcb-doc libxt-doc openjdk-8-demo openjdk-8-source visualvm icedtea-8-plugin 下列【新】软件包将被安装： fonts-dejavu-extra libgif7 libice-dev libpthread-stubs0-dev libsm-dev libx11-dev libx11-doc libxau-dev libxcb1-dev libxdmcp-dev libxt-dev openjdk-8-jdk openjdk-8-jdk-headless openjdk-8-jre x11proto-core-dev x11proto-input-dev x11proto-kb-dev xorg-sgml-doctools xtrans-dev 升级了 0 个软件包，新安装了 19 个软件包，要卸载 0 个软件包，有 249 个软件包未被升级。需要下载 11.8 MB 的归档。解压缩后会消耗 61.9 MB 的额外空间。您希望继续执行吗？ [Y/n] Y

手动安装JDK后应去Tomcat配置文件中更改JAVA_HOME的路径以确保Tomcat正常运行查看当前系统jre和jdk路径，输入update-alternatives --config java和update-alternatives --config javac

root@fjy-desktop:/etc# update-alternatives –config java 链接组 java (提供 /usr/bin/java)中只有一个候选项：/usr/lib/jvm/java-8-openjdk-ar mhf/jre/bin/java 无需配置。 root@fjy-desktop:/etc# update-alternatives –config javac 链接组 javac (提供 /usr/bin/javac)中只有一个候选项：/usr/lib/jvm/java-8-openjdk- armhf/bin/javac 无需配置。输入root@fjy-desktop:/etc# vi /etc/init.d/tomcat进入Tomcat配置文件找到JAVA_HOME，修改为系统配置的jdk路径默认为 JAVA_HOME=/usr/java/jdk CATALINA_HOME=/www/server/tomcat 改成 JAVA_HOME=JAVA_HOME=/usr/lib/jvm/java-8-openjdk-armhf/jre CATALINA_HOME=/www/server/tomcat 保存退出即可