Posted on | In |

这里用一个简单spring boot例子，运行部署Job后自动触发Automaion job。

Automation

简单的用Python + Requests + Tox做个验证。

  • 自动化脚本：

    1
    2
    3
    4
    5
    6
    7
    8
    9
    import requests

    r = requests.get('http://localhost:8000')

    print(r.status_code)
    print(r.text)

    assert r.status_code == 200
    assert 'Hello World!' in r.text

  • Job配置

  • 结果验证

Posted on | In |

系统的把基础知识又过了一遍，算是个简单笔记吧。

简单数据类型

String - 大小写
title(), upper(), lower()

String - 连接
+

String - 空白
\n, \t, lstrip(), rstrip(), strip()
len(), split(), Str()

列表

正向0，反向-1
append(), insert()
remove(), del(), pop(),
sort(), reverse(), sorted(),
遍历 - for in

数字列表
range(), min(), max(), sum()

列表解析
切片 - 切片表示法[:] 创建列表的副

字典

遍历字典，Key，Value
Set剔除重复项

函数

可变参数允许你传入0个或任意个参数，这些可变参数在函数调用时自动组装为一个tuple。而关键字参数允许你传入0个或任意个含参数名的参数，这些关键字参数在函数内部自动组装为一个dict。
一个星（）：表示接收的参数作为元组来处理
两个星（*）：表示接收的参数作为字典来处理

类

init()
将实例用作属性

文件管理

读取模式 （’r’ ）、写入模式 （’w’ ）、附加模式 （’a’ ）或让你能够读取和写入文件的模式（’r+’ ）。如果你省略了模式实参，Python将以默认的只读模式打开文件。

Posted on | In |

前面已经实现最基本的当有Code Commit，会自动运行Jenkins Job，并把Artifact Upload到Artifactory上去。这中间的Version是手动来更改的，最好也跟着一并做成全自动化的。

Goal

Build成功后，自动更新Verion Number在Pom文件，并上传至Github。

Implementation

  1. 通过Shell Script对Pipeline做了一些修改，增加获得Version，更改Version，上传Github

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    node {
       def mvnHome
       
       stage('Checkout and Preparation') { // for display purposes
          // Get some code from a GitHub repository
          git credentialsId: 'ee7a726b-e672-424d-9b78-e8015668d55a', url: 'git@github.com:yongfeiuall/artifactory-maven-example.git'
          // Get the Maven tool.
          // ** NOTE: This 'M3' Maven tool must be configured
          // **       in the global configuration.           
          mvnHome = tool 'M3'
       }
       
       stage('Get current version') {
            sh "'${mvnHome}/bin/mvn' help:evaluate -Dexpression=project.version | grep -e '^[^\\[]' > version.out"
            version = readFile("version.out").trim()
            echo "Version is ${version}"
    		sh 'cat version.out |awk -F "-" \'{print $1}\' > release.out'
            release = readFile("release.out").trim()
    		echo "Release version is ${release}"
            
            sh 'cat release.out|awk \'{n=split($1,A,".");i=2;major=A[1];while(i<n) major = major "." A[i++];print major}\' > major.out'
            sh 'cat release.out|awk \'{n=split($1,A,".");print A[n]}\' > minor.out'
            major = readFile("major.out").trim()
            minor = readFile("minor.out").trim() as Integer
        }
    	
       stage('Build') {
          // Run the maven build
          sh "'${mvnHome}/bin/mvn' clean install -DskipTests"
       }
       
       stage('Results') {
          archive 'target/*.jar'
       }
        
        stage('Update pom with next development version') {
            sh "'${mvnHome}/bin/mvn' versions:set -DnewVersion=${major}.${minor+1}-SNAPSHOT -DoldVersion=* -DgroupId=* -DartifactId=* -DgenerateBackupPoms=false"
            sh script:'git status'
            modifiedFiles = sh(returnStdout: true, script: "git ls-files -m").trim()
            echo modifiedFiles
            if(modifiedFiles) {
                echo "commit required"
                fileAdded = sh(returnStdout: true, script: "git add .")
                sh(script: "git commit -m 'RELEASE_BUILD: changing to next development'")
                sh(script: "git push --set-upstream origin master")
              
                echo "commit completed"
            } else {
                echo "commit not required"
            }
        }
    }

  2. 运行Jenkins Job

  3. Github验证

Posted on | In |

Goal

When Github have commit，will trigger Jenkins job to upload artifact to artifactory.

Prerequisites

Artifactory installation and running，please refer: Artifactory

Config Artifactory with Jenkins

Jenkins > Manage Jenkins > Configure System

Update Pipeline Script

Add script for uploading Artifact

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
node {
   def mvnHome
   def baseArtifactoryUrl = 'com/izheyi/yongfeiuall'
   def artifactoryServer = Artifactory.server 'artifact-jenkins'
   
   stage('Preparation') { // for display purposes
      // Get some code from a GitHub repository
      git credentialsId: 'ee7a726b-e672-424d-9b78-e8015668d55a', url: 'https://github.com/yongfeiuall/artifactory-maven-example.git'
      // Get the Maven tool.
      // ** NOTE: This 'M3' Maven tool must be configured
      // **       in the global configuration.           
      mvnHome = tool 'M3'
   }
   stage('Build') {
      // Run the maven build
      if (isUnix()) {
         sh "'${mvnHome}/bin/mvn' -Dmaven.test.failure.ignore clean package"
      } else {
         bat(/"${mvnHome}\bin\mvn" -Dmaven.test.failure.ignore clean package/)
      }
   }
    stage('publish release build to artifactory') {
        def uploadSpec = """
            {
                "files": [{
                    "pattern": "target/*.jar",
                    "target": "libs-release-local/${baseArtifactoryUrl}/test/"
                }]
            }
        """
        def buildInfo = artifactoryServer.upload(uploadSpec)
        artifactoryServer.publishBuildInfo buildInfo
    }
}

Verify Uploadiing

  1. Run jenkins job
  2. Check on Artifactory

Posted on | In |

Goal

When Github have commit，will trigger Jenkins job run automatically。

Prerequisites

Note: GitHub would not know how to contact “localhost” or 127.0.0.1: what GitHub would consider “local” is not your local machine.

Use Ngrok to expose our local host to the Internet.

  1. Download ngrok
  2. Config and run
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    [root@Automation src]# ./ngrok authtoken 3rsqkJTsPKW432XeLzEZ2_2uVdSyDqgLsTuSSb7FhaH
    Authtoken saved to configuration file: /root/.ngrok2/ngrok.yml
    [root@Automation src]# ./ngrok http 8080

    ngrok by @inconshreveable                                                                                                                                                   (Ctrl+C to quit)
                                                                                                                                                                                                
    Session Status                online                                                                                                                                                        
    Account                       yongfeiuall (Plan: Free)                                                                                                                                      
    Version                       2.2.8                                                                                                                                                         
    Region                        United States (us)                                                                                                                                            
    Web Interface                 http://127.0.0.1:4040                                                                                                                                         
    Forwarding                    http://5a6d5b8a.ngrok.io -> localhost:8080                                                                                                                    
    Forwarding                    https://v.ngrok.io -> localhost:8080                                                                                                                   
                                                                                                                                                                                                
    Connections                   ttl     opn     rt1     rt5     p50     p90                                                                                                                   
                                  35      0       0.00    0.00    5.37    14.10

Create a Personal Access Token in GitHub


Copy the token

Set Up Jenkins Access to GitHub with Token

Add Token to Jenkins

Jenkins > Credentials

Config Access to Github

Jenkins > Manage Jenkins > Configure System

Config the Webhook

Jenkins does not automatically configure webhooks after config pipeline, need to run the job the first time.

In order to connect to Github, need to use the below format:
git credentialsId: 'xxxxxxxxxxxxx', url: 'https://github.com/yongfeiuall/artifactory-maven-example.git'

Then config this job again and save it without any updating, go to Github to check webhook in Repository setting page.

Check trigger automatically

Update script and commit to Github, Jenkins job run automatically

Download the ArtifactoryJenkins-0.0.1-SNAPSHOT.jar file, and extract source code, build successfully.

Posted on | In |

实现环境

通过Jenkins Job从Github获取代码并用Maven Build。

说明： 这里只用了一个Server, 用到的工具都安装在此

Jenkins安运行，请参考Jenkins安装

  1. 在Github创建一个Repository

    代码：

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    package artifactory;

    public class MavenSample {

    	public static void main(String[] args) {
    		// TODO Auto-generated method stub
    		
    		System.out.println("This for testing artifactory, github and jenkins");

    	}
    }

  2. 安装/升级Maven，并做以下配置(Global Tool Configuration)

  3. 安装/升级Git，并做以下配置(Global Tool Configuration)
  4. 配置JDK(Global Tool Configuration)

  5. 创建一个Jenkins Pipeline Job

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    node {
       def mvnHome
       stage('Preparation') { // for display purposes
          // Get some code from a GitHub repository
          git 'https://www.github.com/yongfeiuall/artifactory-maven-example.git'
          // Get the Maven tool.
          // ** NOTE: This 'M3' Maven tool must be configured
          // **       in the global configuration.           
          mvnHome = tool 'M3'
       }
       stage('Build') {
          // Run the maven build
          if (isUnix()) {
             sh "'${mvnHome}/bin/mvn' -Dmaven.test.failure.ignore clean package"
          } else {
             bat(/"${mvnHome}\bin\mvn" -Dmaven.test.failure.ignore clean package/)
          }
       }
       stage('Results') {
          archive 'target/*.jar'
       }
    }

  6. 运行Job

最基本的配置好了，接下来会配置当Github有Commit时Job自动运行。

Posted on | In |

有感于最近花太多时间在手机上，现在的人对于手机的依赖是不是跟之前的鸦片一样？

我们生在这个瞬息万变的时代，科技的创新和发展，作为载体，特别是手机，给我们的工作和生活带来了很多便利，也让我们的生活节奏越来越快。因为快，特别是在大城市，我们模糊了生活和工作的界线，我们会焦虑，就会不断的去学习新知识，了解新科技，生怕自己被这个时代落下。

见到过开车玩手机的，吃饭之前不让TA先拍个照都感觉对不起TA，甚至在地铁上的劝导口号都成了别看手机，下来再看。不管走到哪里，放眼望去，基本都是低头族，手机俨然成了生活的必需品。

不要拿手机和Pad去哄孩子，其实大家都知道手机的危害，更不要沉迷于手机让孩子觉得手机比她重要，有太多的案例由于家长玩手机而造成不可挽回的悲剧。请放下手机，全身心的去陪伴孩子，有时候真的感觉到孩子在某一瞬间就长大了，你现在不想陪她，等你想陪她的时候，晚了。

请放下手机，它不是你生活的唯一。去看看这个世界，去拥抱大自然，多一些与人的交流，感受一下生活的真实和生命的美好。

科技的发展对于我们，我们的下一代，带来了便利，但是对我们的上一代，有时候看起来是有点残酷的。

父母对新技术的拒绝和不知所措，不会用手机微信和网络支付，甚至不会连Wifi，我们应该理解，在他们那个年代技术和知识没有爆炸增长，他们过着甚至可以一眼看到头的生活。我们应该给予他们耐心，就像网上流传的手绘微信使用说明，家里电脑桌面上特制的图标和文字。在快速发展的技术面前我们是那么渺小，不能够带着所有人一起赶上这个时代，但不管技术发展多快多好，在家人和亲情面前它才是那渺小的一方。

Posted on | In |

Jenkins 是一款流行的开源持续集成（Continuous Integration）工具，广泛用于项目开发，具有自动化构建、测试和部署等功能。

可以.war的方式来运行Jenkins

  1. 从Jenkins下载jenkins.war.
  2. 在目录下运行：java -jar jenkens.war

启动后会有这个错：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Upgrading Jenkins. Failed to update the default Update Site 'default'. Plugin upgrades may fail.
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
	at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1917)
	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:301)
	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:295)
	at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1369)
	at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:156)
	at sun.security.ssl.Handshaker.processLoop(Handshaker.java:925)
	at sun.security.ssl.Handshaker.process_record(Handshaker.java:860)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1043)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1343)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1371)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1355)
	at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:563)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1511)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:14

不能够安装Plugin。

解决方法：
把HTTPS换成HTTP(/root/.jenkins/hudson.model.UpdateCenter.xml)

1
2
3
4
5
6
7
<?xml version='1.0' encoding='UTF-8'?>
<sites>
  <site>
    <id>default</id>
    <url>https://updates.jenkins.io/update-center.json</url>
  </site>
</sites>

Posted on | In |

Jfrog的Artifactory是一款Maven仓库服务端软件，可以用来在内网搭建maven仓库，供公司内部公共库的上传和发布，以提高公共代码使用的便利性。

用Open Source进行学习，从这里下载JFrog。安装相对比较简单，Linux常规安装，默认安装到/opt/jfrog目录下。（需要先安装JDK1.8）

安装成功后，执行service artifactory start，默认会在8081端口开启服务。这时从浏览器里访问即可。

Posted on | In |

Loops

Use meta-parameter count. Here we use this example to create 3 s3 bucket with different name.

  1. Create config file

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    # variable
    variable "bucket_names" {
    	description = "Create s3 bucket with these names"
    	type = "list"
    	default = ["hulu-1", "hulu-2", "hulu-3"]
    }

    provider "aws" {
    	region = "ap-northeast-1"
    }

    # S3
    # you can use count.index to get the index of each “iteration” in the “loop”.
    # you can accomplish the same thing by using count and two interpolation functions, element and length:
    # "${element(LIST, INDEX)}"
    # "${length(LIST)}"
    resource "aws_s3_bucket" "terraform_loop" {
    	count = "${length(var.bucket_names)}"
    	bucket = "${element(var.bucket_names, count.index)}"

    	versioning {
    		enabled = true
    	}
    }

    # output
    # which bucket name you want to return by specifying its index in the list: 
    # ${TYPE.NAME.INDEX.ATTRIBUTE}"
    # when you use the '*' character, you get back a list, so you need to wrap the output variable with brackets
    output "1st_bucket_names" {
    value = ["${aws_s3_bucket.terraform_loop.0.bucket}"]
    }
    output "bucket_names" {
    value = ["${aws_s3_bucket.terraform_loop.*.bucket}"]
    }

  2. Run terraform init, terraform plan and terraform apply commands

    Note that since the ‘*’ syntax returns a list, you can combine it with other
    interpolation functions, such as element.

If-statements

You accomplish the same thing by using the count parameter and taking advantage of two properties:

  1. In Terraform, if you set a variable to a boolean true (that is, the word true without any quotes around it), it will be coerced into a 1, and if you set it to a boolean false, it will be coerced into a 0.

  2. If you set count to 1 on a resource, you get one copy of that resource; if you set count to 0, that resource is not created at all.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    # variable
    variable "enable_s3" {
    	description = "Create s3 bucket with these names"
    }

    # S3
    resource "aws_s3_bucket" "terraform_loop" {
    	count = "${var.enable_s3)}"
    	bucket = "if condition"

    	versioning {
    		enabled = true
    	}
    }

    apply:

    1
    2
    3
    4
    5
    6
    7
    8
    provider "aws" {
    	region = "ap-northeast-1"
    }

    module "module-test" {
    	source = "../../module/s3"
    	bucket_name = true
    }

To handle more complicated cases, you can again use the count parameter, but this time, rather than setting it to a boolean variable, you set it to the value returned by a conditional. Conditionals in Terraform use the same ternary syntax available in many programming
languages:
"${CONDITION ? TRUEVAL : FALSEVAL}"
For example, a more verbose way to do the simple if-statement from the previous section is as follows:
count = “${var.enable_s3 ? 1 : 0}”

If-else-statements

You can also use count to implement this:

1
2
3
count = "${var.enable_s3)}"

count = "${1 - var.enable_s3)}"

you can take advantage of the concat interpolation function to implement complicated If-Else-Statements:
"${concat(LIST1, LIST2, ...)}"

Zero-downtime deployment

This is exactly what the create_before_destroy lifecycle setting does!

As an added bonus, if something went wrong during the deployment, Terraform will automatically roll back!