2019 - Flag
<<<<<<< HEAD
4月初的时候看了一下学历教育,今年大部分的时间应该都会在这个考试上了,以下计划相应调整。
cf95e98… switch pc
2018年自我感觉还是可以的,2019年又开始了,好多人都在朋友圈里发看书的Flag了,也为自己立一些Flag:
- 读书
基于去年的不足,今年细化,每月一本书,读书笔记到Blog。- PMP
这个东西几年前有看过,一直没付诸行动,挤时间,逼自己一把,今年拿下证书。- 工作
a. 保持技术的更新,有自己的想法和应对方法。
b. 更全面的看待,处理事情。- 英语
a. 再看一下英语基础,试着找一下快速提升的方法。
b. 搞定Friends前三季。- 生活
a. 控制脾气,控制脾气,控制脾气。
b. 放下手机,活的仪式感一些。
<<<<<<< HEAD
c. 培养一个兴趣,报个钢琴班。- Blog
隔周一更。
奖励
奖励
- 拿到PMP证书,看音乐会。
- 国内游。
cf95e98… switch pc
2018 - 总结
2018年马上过完,年初的时候有个工作计划 2018 - 学习计划,也简单做个总结。
工作
- 随着公司Agile的转型成功,还是学到了很多东西,结果重要,但过程同样重要。
- 年终最后一次上线竟然出现了几个Critical的Issues,新项目和现有的6个项目都有交集,诸多原因,确实也遗漏了一些东西。通过Retro知道了不足,以及How could be better in next stage。但也有一些体会:Agile强调整个Team对质量负责,真正出了问题,还是QA背锅,终究是没有逃脱0和-1游戏规则。有些事情虽不爽,但是要敢于承认错误,力争把工作做到挑不出毛病。某些人,劝你评价别人的时候请先看看自己。唯有自己强大,其他都是浮云。
- 今年学了不少新技术,在测试应用到的技术这块有了一个很明显的提升。
- 基于AWS,Salt,Atrifactory,Terraform,Jenkins,Github实现CI/CD
- 基于SSM的java项目开发流程和实现
- Zookeeper,Dubbo和Docker的基本应用
- 基于Pact的CDC测试
- 性能测试的一些思考
- 通过一些渠道也知道了在技术的全局观上还有待于提高,项目和质量的整体把控上也有欠缺,也要提高一下技术快速落地的能力。
- 参加了一些课程,学到了不少新的思路和知识。
其他
- 基本能保持每周一次的Blog更新,内容多是学习笔记。
- 孩子教育是个很大的事情,还得继续努力。
- 力争生活和工作的Balance。
- 对自己也有一次奖励,去青岛玩了一趟。
遗憾/不足
- 2018只剩下个位数,读的书也只有个位数,来年需要一个更详细一点的计划。
- 英语,跟着公司还花钱报了个课程,效果不是很明显,进步不大,需要反思。
- 十一之后,一直不在状态,要做阶段性总结反思,把目标分拆的更细小,严格要求自己,并做好时间管理。
- 情绪的掌控有待提高,团队管理和沟通要进一步加强。
- 技术很重要,但工作不仅仅是技术。
CDC - Pact:Verify Provider Ways
There are many ways can be used to verify Provider, maven, junit, spring…..
Only list 2 ways below to check the verify process.
Build Plug-in
We use maven plug-in to verify Provider, you can refer the details in previous article Pact: Simple Usage.
The biggest disadvantage using this way is that we need to have provider running and pre-condition set up, e.g., prepare testing data.
Testing Framework
For JVM based provider, we can use pact junit provider runner to verify it. we can use junit framework, and don’t need to run provider, verify the exact provider code instead of all, use the JUnit setup and tear-down as well as a state annotation, and can also using mocking framework.
Add mockito & pact-jvm-provider-spring to maven pom
1
2
3
4
5
6
7
8
9
10
11
12
13<!-- https://mvnrepository.com/artifact/org.mockito/mockito-all -->
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>1.10.19</version>
<scope>test</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/au.com.dius/pact-jvm-provider-spring -->
<dependency>
<groupId>au.com.dius</groupId>
<artifactId>pact-jvm-provider-spring_2.12</artifactId>
<version>3.5.24</version>
</dependency>Consumer test
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64public class ConsumerTest {
@Rule
public PactProviderRuleMk2 mockProvider = new PactProviderRuleMk2("test_provider", "localhost", 8080, this);
@Pact(provider="test_provider", consumer="test_consumer")
public RequestResponsePact getPersonAge(PactDslWithProvider builder) {
Map<String, String> headers = new HashMap<String, String>();
headers.put("Content-Type", "application/json");
DslPart resbody = new PactDslJsonBody()
.stringType("name", "zhangsan")
.integerType("age", 4)
.asBody();
return builder
.uponReceiving("A request for age for person zhangsan")
.path("/person/zhangsan")
.method("GET")
.willRespondWith()
.status(200)
.headers(headers)
.body(resbody)
.toPact();
}
@Pact(provider="test_provider", consumer="test_consumer")
public RequestResponsePact getUser(PactDslWithProvider builder) {
Map<String, String> headers = new HashMap<String, String>();
headers.put("Content-Type", "application/json");
DslPart resbody = new PactDslJsonBody()
.stringType("id", "1")
.stringType("name", "tanghulu")
.integerType("age", 4)
.asBody();
return builder
.given("user 1 exist")
.uponReceiving("A request go get user 1")
.path("/user")
.method("GET")
.willRespondWith()
.status(200)
.headers(headers)
.body(resbody)
.toPact();
}
@Test
@PactVerification(value="test_provider", fragment="getPersonAge")
public void doTest(){
Integer age = new GetPersonAge(mockProvider.getUrl()).checkAge("zhangsan");
System.out.println("Test result, age = " + age);
assertTrue(age <= 6);
}
@Test
@PactVerification(fragment="getUser")
public void getUser(){
String user = new GetUser(mockProvider.getUrl()).getUser();
System.out.println("Test result, user: " + user);
assertTrue(user.contains("tanghulu" ));
}Verify provider - state and mock
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32@RunWith(RestPactRunner.class) // Say JUnit to run tests with custom Runner
@Provider("test_provider") // Set up name of tested provider
@PactFolder("../pact-consumer/target/pacts") // Point where to find pacts (See also section Pacts source in documentation)
@VerificationReports({"console", "markdown"})
public class VerifyState {
@Mock
private ProviderService providerService;
@InjectMocks
private ProviderController providerController = new ProviderController();
@Before //Method will be run before each test of interaction
public void before() {
//initialize your mocks using your mocking framework
MockitoAnnotations.initMocks(this);
//configure the MockMvcTarget with your controller and controller advice
target.setControllers(providerController);
}
@State("user 1 exist")
public void addUser(){
when(providerService.getUser())
.thenReturn(new ProviderModel("1", "tanghulu", 5));
}
@TestTarget // Annotation denotes Target that will be used for tests
public final MockMvcTarget target = new MockMvcTarget(); // Out-of-the-box implementation of Target (for more information take a look at Test Target section)
}
CDC - Pact:Broker Basic
The Pact Broker is an application for sharing for consumer driven contracts and verification results.
- The consumer CI build generates the pacts during execution of its isolated tests, and then publishes the generated pacts.
- The provider CI retrieves the pacts, performs the verification locally, and then publishes the verification results back to the broker.
- Consumer and provider CI deployment builds check with the broker before deploying to ensure the application version they are about deploy will be compatible with the versions of the other applications that are already in that environment.
We can use many approaches to deposite pact files and verification results, i use docker as an example,
Docker Setup
Get info from pact_broker-docker.
- Install docker.
Download
docker-compose.yml
and modify.1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26version: "3"
services:
postgres:
image: postgres
healthcheck:
test: psql postgres --command "select 1" -U postgres
ports:
- "5432:5432"
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
POSTGRES_DB: postgres
broker_app:
image: dius/pact-broker
links:
- postgres
# If you remove nginx, enable the following
ports:
- "80:80"
environment:
PACT_BROKER_DATABASE_USERNAME: postgres
PACT_BROKER_DATABASE_PASSWORD: password
PACT_BROKER_DATABASE_HOST: postgres
PACT_BROKER_DATABASE_NAME: postgresRun
docker-compose up
command to get a running Pact Broker and a clean Postgres database.- Verify.
Pact Broker Usage
consumer
Add the below configs in consumer pom.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15<plugin>
<!-- mvn pact:publish -->
<groupId>au.com.dius</groupId>
<artifactId>pact-jvm-provider-maven_2.11</artifactId>
<version>3.5.24</version>
<configuration>
<!-- <pactDirectory>../pacts</pactDirectory> Defaults to ${project.build.directory}/pacts -->
<pactBrokerUrl>http://192.168.220.139:80</pactBrokerUrl>
<projectVersion>${project.version}</projectVersion> <!-- Defaults to ${project.version} -->
<trimSnapshot>true</trimSnapshot> <!-- Defaults to false -->
<tags>
<tag>cdc/pact_broker</tag>
</tags>
</configuration>
</plugin>execute
mvn pact:publish
to publish the pact files.
provider
Add the below configs in provider pom.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19<plugin>
<groupId>au.com.dius</groupId>
<artifactId>pact-jvm-provider-maven_2.12</artifactId>
<version>3.5.24</version>
<configuration>
<serviceProviders>
<!-- You can define as many as you need, but each must have a unique name -->
<serviceProvider>
<name>test_provider</name>
<!-- All the provider properties are optional, and have sensible defaults (shown below) -->
<protocol>http</protocol>
<host>localhost</host>
<port>8080</port>
<path>/</path>
<pactBrokerUrl>http://192.168.220.139:80/</pactBrokerUrl>
</serviceProvider>
</serviceProviders>
</configuration>
</plugin>Execute
mvn pact:verify
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24D:\Java\pact\pact-jvm\pact-provider>mvn pact:verify
[INFO] Scanning for projects...
[INFO]
[INFO] ----------------------< com.izheyi:pact-provider >----------------------
[INFO] Building pact-provider 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- pact-jvm-provider-maven_2.12:3.5.24:verify (default-cli) @ pact-provider ---
Verifying a pact between test_consumer and test_provider
[from Pact Broker http://192.168.220.139/pacts/provider/test_provider/consumer/test_consumer/version/0.0.1]
A request for age for person zhangsan
returns a response which
has status code 200 (OK)
includes headers
"Content-Type" with value "application/json" (OK)
has a matching body (OK)
[WARNING] Skipping publishing of verification results as it has been disabled (pact.verifier.publishResults is not 'true')
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.702 s
[INFO] Finished at: 2018-11-18T13:12:24+08:00
[INFO] ------------------------------------------------------------------------We can see there has one
WARNING
, Publishing of verification results will be skipped unless this property is set to ‘true’, so we need to add the below config.1
2
3<configuration>
<pact.verifier.publishResults>true</pact.verifier.publishResults>
</configuration>Execute
mvn pact:verify
again.
CDC - Pact: Simple Usage
Usage: Java + Sprint Boot + Maven + Junit,please refer for detail:Github - cdc-pact
pact-provider
Implemetion: return name and age(<6) by passing parameter name
Provider Model
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21public class ProviderModel {
String name;
Integer age;
public ProviderModel(String name, int age) {
this.name = name;
this.age = age;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Integer getAge() {
return age;
}
public void setAge(Integer age) {
this.age = age;
}
}Provider Control
1
2
3
4
5
6
7
8
9
10@RestController
public class ProviderController {
@RequestMapping("/person/{name}")
public ProviderModel person(@PathVariable String name){
int age = new Random().nextInt(6);
return new ProviderModel(name, age);
}
}
pact-consumer
pom.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.izheyi</groupId>
<artifactId>pact-consummer</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>pact-consummer</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.6</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.json/json -->
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20180813</version>
</dependency>
<!-- https://mvnrepository.com/artifact/au.com.dius/pact-jvm-consumer-junit -->
<dependency>
<groupId>au.com.dius</groupId>
<artifactId>pact-jvm-consumer-junit_2.12</artifactId>
<version>3.5.24</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>call provider(test)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45public class GetPersonAge{
int port = 8080;
public GetPersonAge(String url) {
// TODO Auto-generated constructor stub
}
public GetPersonAge() {
// TODO Auto-generated constructor stub
}
public static void main(String[] args) {
Integer age = new GetPersonAge().checkAge("zhangsan");
System.out.println("Age is: " + age);
}
public Integer checkAge(String name) {
try {
String url = String.format("http://localhost:%d/person/%s", port, name);
System.out.println("URL is: " + url);
HttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new HttpGet(url);
String json = null;
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
if (entity != null) {
json = EntityUtils.toString(entity, "UTF-8").trim();
}
System.out.println("Json is: " + json);
JSONObject jsonObject = new JSONObject(json);
String age = jsonObject.get("age").toString();
return new Integer(age);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
return null;
}
}
}consumer pact test
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34public class ConsumerTest {
@Rule
public PactProviderRuleMk2 mockProvider = new PactProviderRuleMk2("test_provider", "localhost", 8080, this);
@Pact(provider="test_provider", consumer="test_consumer")
public RequestResponsePact createPact(PactDslWithProvider builder) {
Map<String, String> headers = new HashMap<String, String>();
headers.put("Content-Type", "application/json");
DslPart resbody = new PactDslJsonBody()
.stringType("name", "zhangsan")
.integerType("age", 5)
.asBody();
return builder
.uponReceiving("A request for age for person zhangsan")
.path("/person/zhangsan")
.method("GET")
.willRespondWith()
.status(200)
.headers(headers)
.body(resbody)
.toPact();
}
@Test
@PactVerification()
public void doTest(){
Integer age = new GetPersonAge(mockProvider.getUrl()).checkAge("zhangsan");
System.out.println("Test result, age = " + age);
assertTrue(age <= 6);
}
}
cdc process with pact
- Run Junit pact test in consumer.
Generate contract file(test_consumer-test_provider.json) to trget automatically.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55{
"provider": {
"name": "test_provider"
},
"consumer": {
"name": "test_consumer"
},
"interactions": [
{
"description": "A request for age for person zhangsan",
"request": {
"method": "GET",
"path": "/person/zhangsan"
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"body": {
"name": "zhangsan",
"age": 5
},
"matchingRules": {
"body": {
"$.name": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.age": {
"matchers": [
{
"match": "integer"
}
],
"combine": "AND"
}
}
}
}
}
],
"metadata": {
"pactSpecification": {
"version": "3.0.0"
},
"pact-jvm": {
"version": "3.5.24"
}
}
}Add the below in provider pom
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26<plugin>
<groupId>au.com.dius</groupId>
<artifactId>pact-jvm-provider-maven_2.12</artifactId>
<version>3.5.24</version>
<configuration>
<serviceProviders>
<!-- You can define as many as you need, but each must have a unique name -->
<serviceProvider>
<name>test_provider</name>
<!-- All the provider properties are optional, and have sensible defaults (shown below) -->
<protocol>http</protocol>
<host>localhost</host>
<port>8080</port>
<path>/</path>
<consumers>
<!-- Again, you can define as many consumers for each provider as you need, but each must have a unique name -->
<consumer>
<name>test_consumer</name>
<!-- currently supports a file path using pactFile or a URL using pactUrl -->
<pactFile>../pact-consumer/target/pacts/test_consumer-test_provider.json</pactFile>
</consumer>
</consumers>
</serviceProvider>
</serviceProviders>
</configuration>
</plugin>Start the provider application.
- Run
mvn pact:verify
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24D:\Java\pact\pact-jvm\pact-provider>mvn pact:verify
[INFO] Scanning for projects...
[INFO]
[INFO] ----------------------< com.izheyi:pact-provider >----------------------
[INFO] Building pact-provider 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- pact-jvm-provider-maven_2.12:3.5.24:verify (default-cli) @ pact-provider ---
Verifying a pact between test_consumer and test_provider
[Using File ..\pact-consumer\target\pacts\test_consumer-test_provider.json]
A request for age for person zhangsan
returns a response which
has status code 200 (OK)
includes headers
"Content-Type" with value "application/json" (OK)
has a matching body (OK)
[WARNING] Skipping publishing of verification results as it has been disabled (pact.verifier.publishResults is not 'true')
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.577 s
[INFO] Finished at: 2018-11-13T09:09:01+08:00
[INFO] ------------------------------------------------------------------------
CDC - 介绍及工具
随着Microservices的流行,把系统拆分成一个个服务,服务之间会有很多依赖关系,但是每个之间又是相互独立的。当一个服务修改时,怎么保证其它调用它的服务没有受到影响呢?
CDC
契约测试 ,又称之为 消费者驱动的契约测试(Consumer-Driven Contracts,简称CDC),根据 消费者驱动契约 ,我们可以将服务分为消费者端和生产者端,而消费者驱动的契约测试的核心思想在于是从消费者业务实现的角度出发,由消费者自己会定义需要的数据格式以及交互细节,并驱动生成一份契约文件。然后生产者根据契约文件来实现自己的逻辑,并在持续集成环境中验证。
- 单元测试 单元测试针对代码单元(通常是类)的测试,单元测试的价值在于能提供最快的反馈。另外好的单元测试还可以帮助你改善设计,在你的团队掌握TDD的前提下,单元测试能辅助重构,帮助改善代码整洁度。
- API测试 API测试是针对业务接口进行的测试,主要测内部接口功能实现是否完整,比如说内部逻辑是不是正常,异常处理是不是正确。
- 契约测试 契约测试其实是为了测试服务之间连接或者说接口调用的正确性,为了验证服务提供者的功能是不是真正能够满足消费者的需求。它其实体现了测试前移的思想,把本来要通过集成测试才能验证的工作化作单元测试和接口测试,用更轻量的方式快速进行验证。
- 集成测试 它从用户的角度验证整个功能的正确性,测的是端到端的流程,并且加入用户场景和数据,验证整个过程是不是OK,它的价值业务价值最高,是验证一个完整的流程。
Pact
Pact 提供API支持CDC Testing,在Consumer端提供Mock服务秋模拟Provider并生成Contract信息,在Provider端使用生成的Contract信息来验证Provider的服务。
Pact工作流程 how pact works:
Consumer
Install zabbix through docker
Zabbix is an open-source ultimate enterprise-level software designed for real-time monitoring of millions of metrics collected from tens of thousands of servers, virtual machines and network devices.
Deploy
Deploy Zabbix by running the below bash script.1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40#!/bin/bash
# Deploy a mysql container for zabbix to use.
docker run --name zabbix-mysql-server --hostname zabbix-mysql-server \
-e MYSQL_ROOT_PASSWORD="123456" \
-e MYSQL_USER="zabbix" \
-e MYSQL_PASSWORD="123456" \
-e MYSQL_DATABASE="zabbix" \
-p 3306:3306 \
-d mysql:5.7 \
--character-set-server=utf8 --collation-server=utf8_bin
# Deploy the zabbix-server application container
docker run --name zabbix-server-mysql --hostname zabbix-server-mysql \
--link zabbix-mysql-server:mysql \
-e DB_SERVER_HOST="mysql" \
-e MYSQL_USER="zabbix" \
-e MYSQL_DATABASE="zabbix" \
-e MYSQL_PASSWORD="123456" \
-v /etc/localtime:/etc/localtime:ro \
-v /data/docker/zabbix/alertscripts:/usr/lib/zabbix/alertscripts \
-v /data/docker/zabbix/externalscripts:/usr/lib/zabbix/externalscripts \
-p 10051:10051 \
-d \
zabbix/zabbix-server-mysql
# Deploy the webserver frontend.
docker run --name zabbix-web-nginx-mysql --hostname zabbix-web-nginx-mysql \
--link zabbix-mysql-server:mysql \
--link zabbix-server-mysql:zabbix-server \
-e DB_SERVER_HOST="mysql" \
-e MYSQL_USER="zabbix" \
-e MYSQL_PASSWORD="123456" \
-e MYSQL_DATABASE="zabbix" \
-e ZBX_SERVER_HOST="zabbix-server" \
-e PHP_TZ="Asia/Shanghai" \
-p 8000:80 \
-p 8443:443 \
-d \
zabbix/zabbix-web-nginx-mysql
Verify
After running the script, we can access it by http://192.168.220.138:8000/
in browser:
The default credentials are: Admin/zabbix