Skip to content

Commit 914a9d6

Browse files
ashishranjan738ashishranjan738
authored
Updates Readme for running loadtests (#362)
This commit updates readme for running loadtests. Signed-off-by: Ashish Ranjan <[email protected]>
1 parent 8fa7490 commit 914a9d6

File tree

3 files changed

+180
-1
lines changed

3 files changed

+180
-1
lines changed

‎infrastructure/README.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,27 @@ The application is caching IAM policies of two components as static files in `li
111111
These IAM policies are pinned to a version and downloaded form Github. When the version of these components change
112112
in an upgrade, the cache needs to be updated by modifying the version values in `cache-iam-policies.sh` and executing file.
113113

114+
### [Optional] Creating Amazon Managed Prometheus Workspace:
115+
```
116+
AMP_WORKSPACE_NAME=<workspace-name>
117+
AMP_WORKSPACE_ID=`aws amp create-workspace --alias $AMP_WORKSPACE_NAME --output text --query workspaceId`
118+
echo $AMP_WORKSPACE_ID # this amp workspace can be used to remote write cluster metrics from test tasks. Ref: https://github.com/awslabs/kubernetes-iteration-toolkit/blob/main/tests/pipelines/eks/awscli-cl2-load-with-addons.yaml#L17
119+
```
120+
121+
### [Optional] Creating Amazon Managed Grafana Dashboard. [Ref](https://docs.aws.amazon.com/grafana/latest/userguide/AMP-adding-AWS-config.html)
122+
* Open the Amazon Managed Grafana console at https://console.aws.amazon.com/grafana/.
123+
* In the upper left corner of the page, choose the menu icon and then choose All workspaces.
124+
* Choose the name of the workspace. (use the workspace id and name you got from the previous command)
125+
* If you didn't choose to use service-managed permissions for this workspace when you created it, then change from using customer-managed permissions to use service-managed permissions to ensure that the proper IAM roles and policies are enabled for using the AWS data source configuration option in the Grafana workspace console. To do so, choose the edit icon by IAM role and then choose Service managed, Save changes. For more information, see Amazon Managed Grafana permissions and policies for AWS data sources (https://docs.aws.amazon.com/grafana/latest/userguide/AMG-manage-permissions.html).
126+
* Choose the Data sources tab. Then select the check box for Amazon Managed Service for Prometheus, and choose Actions, Enable service-managed policy.
127+
* Choose the Data sources tab again, and then choose Configure in Grafana in the Amazon Managed Service for Prometheus row.
128+
* Sign into the Grafana workspace console using IAM Identity Center if necessary.
129+
* In the left navigation bar in the Grafana workspace console, choose the AWS icon and then choose AWS services, Prometheus.
130+
* Select the Region that you want Amazon Managed Grafana to search to discover Amazon Managed Service for Prometheus workspaces, and then select the accounts and Amazon Managed Service for Prometheus workspaces that you want to add, and then choose Add data source.
131+
132+
### Installing tekton tasks and pipelines
133+
Follow the steps from here: https://github.com/awslabs/kubernetes-iteration-toolkit/tree/main/tests
134+
114135
### Context Parameters:
115136

116137
| Context Param | Description | Default | | |

‎tests/README.md‎

Lines changed: 65 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,74 @@
11
# Kubernetes Iteration Toolkit Tests with Tekton
22

33
### Overview:
4-
Tekton is a powerful and flexible open-source framework for creating CI/CD systems, allowing developers to build, test, and deploy across cloud providers and on-premise systems.
4+
Tekton is a powerful and flexible open-source framework for creating CI/CD systems, allowing developers to build, test, and deploy across cloud providers and on-premise systems. This documents steps for setting up kit load test tasks and pipelines in tekton infrastructure.
55

6+
### Prerequisites
7+
* A kubernetes cluster with tekton infrastructure configured. [Ref](https://github.com/awslabs/kubernetes-iteration-toolkit/tree/main/infrastructure) for setting up KITInfra.
8+
* [Kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl-macos/)
69

710

11+
### Installing tekton pipelines
12+
```
13+
cd tests
14+
kubectl apply -n tekton-pipelines -f tasks -R
15+
kubectl apply -n tekton-pipelines -f pipelines -R
16+
```
17+
18+
### Running a single pipelinerun
19+
```
20+
cat <<EOF | kubectl apply -f -
21+
apiVersion: tekton.dev/v1beta1
22+
kind: PipelineRun
23+
metadata:
24+
generateName: awscli-eks-load-sample
25+
spec:
26+
pipelineRef:
27+
name: awscli-eks-cl2loadtest
28+
timeout: 9h0m0s
29+
workspaces:
30+
- name: source
31+
emptyDir: {}
32+
- name: config
33+
volumeClaimTemplate:
34+
spec:
35+
accessModes:
36+
- ReadWriteOnce
37+
storageClassName: gp2
38+
resources:
39+
requests:
40+
storage: 1Gi
41+
- name: results
42+
emptyDir: {}
43+
params:
44+
- name: cluster-name
45+
value: "awscli-eks-load-100"
46+
- name: desired-nodes
47+
value: "100"
48+
- name: pods-per-node
49+
value: "10"
50+
- name: nodes-per-namespace
51+
value: "100"
52+
- name: cl2-load-test-throughput
53+
value: "20"
54+
- name: results-bucket
55+
value: kit-eks-scalability/kit-eks-5k/$(date +%s)
56+
- name: slack-message
57+
value: "You can monitor here - https://tekton.scalability.eks.aws.dev/#/namespaces/tekton-pipelines/pipelineruns ;5k node "
58+
- name: slack-hook
59+
value: <slack hook url. Pass empty if you don't have one>
60+
- name: vpc-cfn-url
61+
value: "https://raw.githubusercontent.com/awslabs/kubernetes-iteration-toolkit/main/tests/assets/amazon-eks-vpc.json"
62+
podTemplate:
63+
nodeSelector:
64+
kubernetes.io/arch: amd64
65+
serviceAccountName: tekton-pipelines-executor
66+
EOF
67+
```
68+
69+
### Scheduling recurrent test runs.
70+
For scheduling recurrent test runs a cron job is used to trigger the pipeline from the pipelinerun template. Reffer to the for a sample cron job and pipelinerun temaple. [example.yaml](pipelineruns%2Feks%2Fexample.yaml)
71+
872
### Test Images
973

1074
Tekton tasks can leverage image like clusterloader2 to perform various kinds of tests like load, pod-density on K8s cluster on KIT infra.

‎tests/pipelineruns/eks/example.yaml‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
---
2+
apiVersion: batch/v1beta1
3+
kind: CronJob
4+
metadata:
5+
name: awscli-eks-load-5k-example
6+
namespace: tekton-pipelines
7+
spec:
8+
schedule: "13 15 * * *"
9+
jobTemplate:
10+
spec:
11+
template:
12+
spec:
13+
containers:
14+
- name: curl
15+
image: curlimages/curl
16+
args: ["curl", "-X", "POST", "--data", "{}", "el-awscli-eks-load-5k.tekton-pipelines.svc.cluster.local:8080"]
17+
restartPolicy: Never
18+
---
19+
apiVersion: triggers.tekton.dev/v1alpha1
20+
kind: TriggerBinding
21+
metadata:
22+
name: awscli-eks-load-5k
23+
namespace: tekton-pipelines
24+
spec:
25+
params:
26+
- name: servicerole
27+
value: <service role ARN. Example of Service Role- https://github.com/awslabs/kubernetes-iteration-toolkit/blob/main/tests/assets/eks_service_role.json>
28+
---
29+
apiVersion: triggers.tekton.dev/v1alpha1
30+
kind: EventListener
31+
metadata:
32+
name: awscli-eks-load-5k
33+
namespace: tekton-pipelines
34+
spec:
35+
serviceAccountName: tekton-triggers
36+
triggers:
37+
- name: cron
38+
bindings:
39+
- ref: awscli-eks-load-5k
40+
template:
41+
ref: awscli-eks-load-5k
42+
---
43+
apiVersion: triggers.tekton.dev/v1alpha1
44+
kind: TriggerTemplate
45+
metadata:
46+
name: awscli-eks-load-5k
47+
namespace: tekton-pipelines
48+
spec:
49+
resourcetemplates:
50+
- apiVersion: tekton.dev/v1beta1
51+
kind: PipelineRun
52+
metadata:
53+
generateName: awscli-eks-load-5k-
54+
spec:
55+
pipelineRef:
56+
name: awscli-eks-cl2loadtest-with-addons
57+
timeout: 9h0m0s
58+
workspaces:
59+
- name: source
60+
emptyDir: {}
61+
- name: config
62+
volumeClaimTemplate:
63+
spec:
64+
accessModes:
65+
- ReadWriteOnce
66+
storageClassName: gp2
67+
resources:
68+
requests:
69+
storage: 1Gi
70+
- name: results
71+
emptyDir: {}
72+
params:
73+
- name: cluster-name
74+
value: "awscli-eks-load-100"
75+
- name: desired-nodes
76+
value: "100"
77+
- name: pods-per-node
78+
value: "10"
79+
- name: nodes-per-namespace
80+
value: "100"
81+
- name: cl2-load-test-throughput
82+
value: "20"
83+
- name: results-bucket
84+
value: <replace it a s3 bucket to store test results>
85+
- name: slack-message
86+
value: "You can monitor here - https://tekton.scalability.eks.aws.dev/#/namespaces/tekton-pipelines/pipelineruns ;5k node "
87+
- name: slack-hook
88+
value: <slack hook url>
89+
- name: vpc-cfn-url
90+
value: "https://raw.githubusercontent.com/awslabs/kubernetes-iteration-toolkit/main/tests/assets/amazon-eks-vpc.json"
91+
podTemplate:
92+
nodeSelector:
93+
kubernetes.io/arch: amd64
94+
serviceAccountName: tekton-pipelines-executor

0 commit comments

Comments
 (0)