对于测试来说，更关注于功能（接口或UI）测试的覆盖率，一样也可以通过Jacoco来实现。

我们需要这两个jar： jacocoagent.jar和jacococli.jar。

一样是和Git，Jenkins集成，说一下思路

1. 同样mvn install，打包生成jar包

2. 项目启动的时候，配置jacoco agent

   1	java -javaagent:/jacocoagent.jar=includes=*,output=tcpserver,address=localhost,port=11111 -jar ./target/project-0.0.1-SNAPSHOT.jar

3. 启动后，生成报告

   1	java -jar jacococli.jar dump --address localhost --port 11111 --destfile ./jacoco-demo.exec

   1	java -jar jacococli.jar report ./jacoco-demo.exec --classfiles ./target/classes --html ./report

   没有做任何测试，覆盖率极低。

4. 做一些测试之后，生成报告

   反映出实时的测试覆盖率。

以上只是简单例子，也可以在Tomcat里配置。

介绍

JaCoCo是一个开源的覆盖率工具，它针对的开发语言是java，其使用方法很灵活，可以嵌入到Ant、Maven中；可以作为Eclipse插件，可以使用其JavaAgent技术监控Java程序等等。官网(Jacoco)[https://www.eclemma.org/jacoco/]

覆盖率计数器

1. 行覆盖

2. 类覆盖

3. 方法覆盖

4. 分支覆盖

5. 指令覆盖

6. 圈复杂度

原理

Jacoco使用插桩的方式来记录覆盖率数据。
插桩模式有两种：

1. on-the-fly模式

JVM通过 -javaagent参数指定jar文件启动代理程序，代理程序在ClassLoader装载一个class前判断是否修改class文件，并将探针插入class文件，探针不改变原有方法的行为，只是记录是否已经执行。

2. offline模式

在测试之前先对文件进行插桩，生成插过桩的class或jar包，测试插过桩的class和jar包，生成覆盖率信息到文件，最后统一处理，生成报告。

集成Maven

下载配置JDK，Maven，Jacoco

Manven配置(Maven Plug-in)[https://www.eclemma.org/jacoco/trunk/doc/maven.html]

用下载的Jacoco的example里的例子演示

1
2
3
4
5
6
7
8
9

C:\Test\Jacoco>mvn install
[INFO] Scanning for projects...
[INFO]
[INFO] ----------------< org.jacoco:org.jacoco.examples.maven >----------------
[INFO] Building JaCoCo Maven plug-in example 0.8.6
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.6:prepare-agent (default-prepare-agent) @ org.jacoco.examples.maven ---
[INFO] argLine set to -

Maven构建成功之后，生成代码覆盖报告（\target\site\jacoco）

集成Git和Jenkins

1. 把example上传到Github上

2. 配置Jenkins，回传报告

   

3. 配置Jacoco Coverage Report

   

4. Coverage Trend

   

从方法网络取得，仅供学习研究，请勿用于商业用途！

1. 安装Pycharm

2. 下载

   链接: https://pan.baidu.com/s/1W6k0RRB45iJ15L6_7HEMKA 提取码: fti8

3. 打开Pycharm， 把下载的zip拖拽到IDE窗口，重启Pycharm

4. 在弹出的JetbrainsAgent Helper对话框中，点击‘为Pycharm安装’

5. 重启Pycharm即可

从方法网络取得，仅供学习研究，请勿用于商业用途！

1. 下载

   链接: https://pan.baidu.com/s/14Cc_nqyNNu5Ol1yzmMZwQw 提取码: 2dav

2. 解压安装

   安装 navicat for mysql

3. 激活

   打开Patch.ext，选择navicat for mysql安装目录下的navicat.exe即可。

We could define one online report to know test status easily. here just setup some widget. use the below tech:

1. Python
2. Jira
3. Xray
4. Highcharts

Total Test Execution

Jira Rest API: https://xxxxxx/jira/rest/api/2/issue/ISR-11732

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75

"customfield_10931": {
          "count": 17,
          "statuses": [
              {
                  "id": 0,
                  "name": "PASS",
                  "description": "The test run has passed",
                  "isFinal": true,
                  "color": "#95C160",
                  "isNative": true,
                  "statusCount": 8,
                  "statusPercent": 47.05882352941177
              },
              {
                  "id": 3,
                  "name": "FAIL",
                  "description": "The test run has failed",
                  "isFinal": true,
                  "color": "#D45D52",
                  "isNative": true,
                  "statusCount": 1,
                  "statusPercent": 5.882352941176471
              },
              {
                  "id": 4,
                  "name": "ABORTED",
                  "description": "The test run was aborted",
                  "isFinal": true,
                  "color": "#111111",
                  "isNative": true,
                  "statusCount": 0,
                  "statusPercent": 0
              },
              {
                  "id": 1000,
                  "name": "N/A",
                  "description": "The test can't be executed or is not relevant",
                  "isFinal": true,
                  "color": "#000000",
                  "isNative": false,
                  "statusCount": 8,
                  "statusPercent": 47.05882352941177
              },
              {
                  "id": 2,
                  "name": "EXECUTING",
                  "description": "The test run is currently being executed",
                  "isFinal": false,
                  "color": "#F1E069",
                  "isNative": true,
                  "statusCount": 0,
                  "statusPercent": 0
              },
              {
                  "id": 1001,
                  "name": "BLOCKED",
                  "description": "The test run is currently blocked",
                  "isFinal": false,
                  "color": "#1F39B8",
                  "isNative": false,
                  "statusCount": 0,
                  "statusPercent": 0
              },
              {
                  "id": 1,
                  "name": "TODO",
                  "description": "The test run has not started",
                  "isFinal": false,
                  "color": "#A2A6AE",
                  "isNative": true,
                  "statusCount": 0,
                  "statusPercent": 0
              }
          ]
      }

Test Execution Summary

Xray Rest API: https://xxxxxx/jira/rest/raven/1.0/api/testexec/ISR-11759/test

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74

[
    {
        "id": 4389836,
        "status": "PASS",
        "key": "ISR-10928",
        "rank": 1
    },
    {
        "id": 4389837,
        "status": "PASS",
        "key": "ISR-10927",
        "rank": 2
    },
    {
        "id": 4389838,
        "status": "PASS",
        "key": "ISR-10926",
        "rank": 3
    },
    {
        "id": 4389839,
        "status": "PASS",
        "key": "ISR-10925",
        "rank": 4
    },
    {
        "id": 4389840,
        "status": "PASS",
        "key": "ISR-10935",
        "rank": 5
    },
    {
        "id": 4389841,
        "status": "PASS",
        "key": "ISR-10934",
        "rank": 6
    },
    {
        "id": 4389842,
        "status": "PASS",
        "key": "ISR-10933",
        "rank": 7
    },
    {
        "id": 4389843,
        "status": "PASS",
        "key": "ISR-10932",
        "rank": 8
    },
    {
        "id": 4389844,
        "status": "PASS",
        "key": "ISR-10931",
        "rank": 9
    },
    {
        "id": 4389845,
        "status": "PASS",
        "key": "ISR-10930",
        "rank": 10
    },
    {
        "id": 4389846,
        "status": "PASS",
        "key": "ISR-10938",
        "rank": 11
    },
    {
        "id": 4389847,
        "status": "FAIL",
        "key": "ISR-10937",
        "rank": 12
    }
]

Test Active Defect

Jira Rest API: https://xxxxxx/jira/rest/api/2/search?jql=status not in (Closed, Resolved) AND issue in defectsCreatedDuringTestExecution(ISR-11759)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

{
    "expand": "names,schema",
    "startAt": 0,
    "maxResults": 50,
    "total": 1,
    "issues": [
        {
            "expand": "operations,versionedRepresentations,editmeta,changelog,renderedFields",
            "id": "5925696",
            "self": "https://xxxxxx/jira/rest/api/2/issue/5925696",
            "key": "ISR-11944",
            "fields": {
	"priority": {
                    "self": "https://xxxxxx/jira/rest/api/2/priority/4",
                    "iconUrl": "https://xxxxxx/jira/images/icons/priorities/low.svg",
                    "name": "Low",
                    "id": "4"
	}
            }
        }
    ]
}

Report:

工作当中，发现用户在测试过程当中都是手工检查两个Excel文件的数据是否一致，有的都有好几万行，非常耗时也可能会遗漏一些数据。

中间也试了Spreadsheet Compare这个工具，有时候对比达不到想要的效果，就想着用Python写一个小程序来实现两个Excel的数据对比，思路：

1. 用户不懂技术，实现一个简单界面让用户去选择文件
2. 对比数据，把结果写到一个新Excel里，并标注不同的数据

工具实现

通过tkinter和pandas来实现，代码如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88

from tkinter import *
from tkinter.filedialog import *
import pandas as pd

root = Tk()
root.title("Compare Excel")
root.geometry('600x250')


def select_compare_file():
    name = askopenfilename()
    file1_text.set(name)

Button(root, text="Select Compare File", command=select_compare_file).pack()
file1_text = StringVar()
file1 = Entry(root, textvariable=file1_text, width=150)
file1_text.set(" ")
file1.pack()


Label(root, text="").pack()


def select_to_file():
    a = askopenfilename()
    file2_text.set(a)
Button(root, text="Select To File", command = select_to_file).pack()
file2_text = StringVar()
file2 = Entry(root, textvariable=file2_text, width=150)
file2_text.set(" ")
file2.pack()


Label(root, text="").pack()

def compare_excel(compare, to):
    f1 = pd.read_excel(compare)
    f2 = pd.read_excel(to)

    # Perform Diff
    dfDiff = f1.copy()
    for row in range(dfDiff.shape[0]):
        for col in range(dfDiff.shape[1]):
            value_OLD = f1.iloc[row, col]
            try:
                value_NEW = f2.iloc[row, col]
                if value_OLD == value_NEW:
                    dfDiff.iloc[row, col] = f2.iloc[row, col]
                else:
                    dfDiff.iloc[row, col] = ('{}→{}').format(value_OLD, value_NEW)
            except:
                dfDiff.iloc[row, col] = ('{}→{}').format(value_OLD, 'NaN')

    # Save output and format
    writer = pd.ExcelWriter('compare_result.xlsx', engine='xlsxwriter')

    dfDiff.to_excel(writer, sheet_name='DIFF', index=False)

    # get xlsxwriter objects
    workbook = writer.book
    worksheet = writer.sheets['DIFF']
    worksheet.hide_gridlines(2)

    # define formats
    highlight_fmt = workbook.add_format({'font_color': '#FF0000', 'bg_color': '#6A93B0'})

    # set format over range
    ## highlight changed cells
    worksheet.conditional_format('A1:ZZ1000', {'type': 'text',
                                               'criteria': 'containing',
                                               'value': '→',
                                               'format': highlight_fmt})
    # save
    writer.save()
    print('Compare Done.')


def on_click():
    f1 = file1_text.get()
    f2 = file2_text.get()

    compare_excel(f1, f2)

    Label(root, text="Compare done, please see the result: 'compare_result.xlsx'").pack()


Button(root, text="Start Compare", command=on_click).pack()
root.mainloop()

验证

1. 准备对比文件

   • 文件1

     1 2 3 4 5

     学号 姓名 年龄 年级 1 tom 12 1 2 jim 33 8 3 Green 21 2 4 Hello 11 1

   • 文件2

     1 2 3 4 5

     No. Name Age Grade 1 Tom 12 1 2 Jimmy 23 8 3 Green 21 2 4 Hello 11 1

2. 运行程序，选择文件，对比

   

3. 查看结果

   

进一步

用户不会直接用命令行运行脚本，把脚本封装成一个bat文件，直接双击使用。

Jmeter原生的监控确实是丑了点，且很难实时的压测过程中的性能状态，用下面的方式来代替原生的监控展示。

InfluxDB来存Jmeter运行数据，用Grafana来展示数据。Jmeter引入Backend Listener，压测的过程中实时把统计数据发送到InfluxDB中，通过Grafana可视化的展示出来。

InfluxDB

一个时序数据库，不做详细说明，可参见官网InfluxDB

1. 从官网下载

2. 修改配置文件，influxdb.conf

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

   [meta] # Where the metadata/raft database is stored dir = "C:/influxdb/meta" # Automatically create a default retention policy when creating a database. # retention-autocreate = true # If log messages are printed for the meta service # logging-enabled = true ############################ [data] # The directory where the TSM storage engine stores TSM files. dir = "C:/influxdb/data" # The directory where the TSM storage engine stores WAL files. wal-dir = "C:/influxdb/wal" ############################ [http] # Determines whether HTTP endpoint is enabled. enabled = true # Determines whether the Flux query endpoint is enabled. # flux-enabled = false # Determines whether the Flux query logging is enabled. # flux-log-enabled = false # The bind address used by the HTTP service. bind-address = ":8086"

1. 创建相应Folder： data, meta, wal

2. 双击influxd.exe，运行influxDB

   1 2 3 4 5 6 7 8 9 10

   8888888 .d888 888 8888888b. 888888b. 888 d88P" 888 888 "Y88b 888 "88b 888 888 888 888 888 888 .88P 888 88888b. 888888 888 888 888 888 888 888 888 8888888K. 888 888 "88b 888 888 888 888 Y8bd8P' 888 888 888 "Y88b 888 888 888 888 888 888 888 X88K 888 888 888 888 888 888 888 888 888 Y88b 888 .d8""8b. 888 .d88P 888 d88P 8888888 888 888 888 888 "Y88888 888 888 8888888P" 8888888P" 2020-06-23T06:34:00.421889Z info InfluxDB starting {"log_id": "0NZyzDPG000", "version": "1.8.0", "branch": "1.8", "commit": "781490de48220d7695a05c29e5a36f550a4568f5"}

Jmeter

1. 在InfluxDB创建DB

   1 2 3 4 5 6 7 8 9

   Connected to http://localhost:8086 version 1.8.0 InfluxDB shell version: 1.8.0 > create database jmeter > show databases name: databases name ---- _internal jmeter

2. 配置 Jmeter Backend Listerner

   

Grafana

1. 配置数据源

   

2. 搭建Dashboard

   Import Dashboard: 5496

Verify

运行Jmeter脚本，查看数据展示：

From last blog, we integrate RF, Jenkins and Xray, but for now, we usually use ‘Pipeline’ way for better management. Give it a try…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

pipeline {
    agent any
    stages {  
        stage('Test Execution'){
            steps{
                bat 'robot db.robot'
            }
        }
        
        stage('Publish RF Test Result'){
             steps {
				step([
                     $class: 'RobotPublisher',
                     disableArchiveOutput: false,
                     logFileName: 'log.html',
                     otherFiles: '',
                     outputFileName: 'output.xml',
                     outputPath: '',
                     passThreshold: 100,
                     reportFileName: 'report.html',
                     unstableThreshold: 0
				])
			}
		 }
        
        stage('Import Result To Xray') {
            steps {
                step([
                    $class: 'XrayImportBuilder',
                    endpointName: '/robot',
                    fixVersion: '',
                    importFilePath: 'output.xml',
                    importToSameExecution: 'true',
                    projectKey: 'ISR',
                    serverInstance: '39c998d7-e25b-4cc7-81d6-dfe14d970cac',
                    testPlanKey: '',
                    testExecKey: 'ISR-8352'
				])
            }
        }
    }s
}

After run job, the result as below:

Currently, we use Xray in Jira for test management, now we want to implement our automation as below:

1. use Robotframework design script
2. use Jenkins trigger automaion running
3. return running result to xray and update execution status

Pre-Condition

1. Create one test case, one execution, and add test case to execution, the default status as below:
   
2. RF demo script:
   

Install and config xray jenkins plugin

Use offline way to install, download plugin here: xray-connector, and install plugin as below:

Then, we need to configure Xray plug-in to communicate with JIRA, navigate to Manage Jenkins > Configure System > Xray for Jira configuration.

Config jenkins job to run automaiton test and return report to xray

1. Just use RF command to run demo script, config as below:
   
2. Retrun results to Xray, define in Post-build actioas - “Xray: Results Import Task”.
   
3. Config task with below params:
   

   - Import to Same Test Execution – When this option is checked, if you are importing multiple execution report files using a glob expression, the results will be imported to the same Test Execution
   - Execution Report File (file path with file name) – location and name of the Robot Framework tests output file. 
   - Project Key – key of the JIRA project. This is a required parameter
   - Test Execution Key – JIRA issue key where results will be imported. 
   - Test Plan Key – JIRA issue key of the Test Plan. 
   - Test Environments – list of Environments (separated by “,”) where tests are executed.
   

4. Run jenkins job
5. Back to Jira Xray, the status update from TODO to PASS automatically
   
6. Go to Execution detail page to view automaion detail result
   

Finally

Other things we could do:

1. Create test exceution in xray automatically after automation running
2. Even create test case after automation running

上篇中介绍了Prometheus， Grafana，并实现了实时监控Windows，回顾：Prometheus + Grafana 实时监控Windows

本篇来介绍一下Linux。

node exporter安装配置

1. 下载 node exporter

   1	wget https://github.com/prometheus/node_exporter/releases/download/v0.18.1/node_exporter-0.18.1.linux-amd64.tar.gz

2. 解压

   1	tar -zxvf node_exporter-0.18.1.linux-amd64.tar.gz

3. 添加系统服务 (vi /usr/lib/systemd/system/node_exporter.service)

   1 2 3 4 5 6 7 8 9

   [Unit] Description=https://prometheus.io [Service] Restart=on-failure ExecStart=/opt/node_exporter-0.18.1.linux-amd64/node_exporter [Install] WantedBy=multi-user.target

4. 启动服务

   1 2	systemctl start node_exporter systemctl status node_exporter

5. 验证

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

   [root@izheyi node_exporter-0.18.1.linux-amd64]# curl http://localhost:9100/metrics # HELP go_gc_duration_seconds A summary of the GC invocation durations. # TYPE go_gc_duration_seconds summary go_gc_duration_seconds{quantile="0"} 0 go_gc_duration_seconds{quantile="0.25"} 0 go_gc_duration_seconds{quantile="0.5"} 0 go_gc_duration_seconds{quantile="0.75"} 0 go_gc_duration_seconds{quantile="1"} 0 go_gc_duration_seconds_sum 0 go_gc_duration_seconds_count 0 # HELP go_goroutines Number of goroutines that currently exist. # TYPE go_goroutines gauge go_goroutines 6 # HELP go_info Information about the Go environment. # TYPE go_info gauge go_info{version="go1.12.5"} 1

Prometheus配置

1. 添加prometheus相关配置， prometheus.yml

   1 2 3

   - job_name: 'linux' static_configs: - targets: ['192.168.174.128:9100']

2. 重启prometheus

3. 验证

   

Grafana配置

1. 配置Prometheus数据源

2. 搭建Prometheus仪表盘
   Import Dashboard: 8919

3. 验证