Monolith setup using AWS EKS
Primeiros Passos Documentação Guias Arquitetura API FAQ

Monolith setup using AWS EKS

This guide will help you to setup ThingsBoard in monolith mode using AWS EKS. See monolithic architecture page for more details about how it works. The advantage of monolithic deployment via K8S comparing to Docker Compose is that in case of AWS instance outage, K8S will restart the service on another instance. We will use Amazon RDS for managed PostgreSQL.

Install and configure tools

To deploy ThingsBoard on EKS cluster you’ll need to install kubectl, eksctl and awscli tools.

Afterwards you need to configure Access Key, Secret Key and default region. To get Access and Secret keys please follow this guide. The default region should be the ID of the region where you’d like to deploy the cluster.

1

aws configure

Step 1. Clone ThingsBoard CE K8S scripts repository

1
2

git clone https://github.com/thingsboard/thingsboard-ce-k8s.git
cd thingsboard-ce-k8s/aws/monolith

Step 2. Configure and create EKS cluster

In the cluster.yml file you can find suggested cluster configuration. Here are the fields you can change depending on your needs:

  • region - should be the AWS region where you want your cluster to be located (the default value is us-east-1)
  • availabilityZones - should specify the exact IDs of the region’s availability zones (the default value is [us-east-1a,us-east-1b,us-east-1c])
  • instanceType - the type of the instance with TB node (the default value is m5.xlarge)

Note: if you don’t make any changes to instanceType and desiredCapacity fields, the EKS will deploy 1 node of type m5.xlarge.

The following command will create new VPC for your ThingsBoard cluster. This guide assumes you will create new VPC. Although it is fine to use existing VPC and subnets as well. You can find more information about configuring VPC for eksctl aqui.

Command to create AWS cluster:

1

eksctl create cluster -f cluster.yml

Step 3. Create AWS load-balancer controller

Once the cluster is ready you must create AWS load-balancer controller. You can do it by following this guide. The cluster provisioning scripts will create several load balancers:

  • “tb-http-loadbalancer” - AWS ALB that is responsible for the web UI, REST API and HTTP transport;
  • “tb-mqtt-loadbalancer” - AWS NLB that is responsible for the MQTT transport;
  • “tb-coap-loadbalancer” - AWS NLB that is responsible for the CoAP transport;

Provisioning of the AWS load-balancer controller is a very important step that is required for those load balancers to work properly.

Step 4. Amazon PostgreSQL DB Configuration

You’ll need to set up PostgreSQL on Amazon RDS. One of the ways to do it is by following this guide.

Note: Make sure your database is accessible from the cluster, one of the way to achieve this is to create the database in the same VPC and subnets as ThingsBoard cluster.

aqui you should choose VPC with the name of your cluster:

image

aqui you should choose security group corresponding to the one on the screen:

image

Note: in order to make PostgreSQL more secure you may create the separate security group, configure access only to the 5432 port and from the ThingsBoard nodes. This can be achieved if you assigned security group to the node node-group in the cluster.yml file.

Make sure that thingsboard database is created along with PostgreSQL instance (or create it afterwards).

image

On AWS Console get the Endpoint of the RDS PostgreSQL and paste it to SPRING_DATASOURCE_URL in the tb-node-db-configmap.yml instead of your_url:

image

Also, you’ll need to set SPRING_DATASOURCE_USERNAME and SPRING_DATASOURCE_PASSWORD with PostgreSQL username and password corresponding.

Step 5. Configure secure HTTP connection

Use AWS Certificate Manager to create or import SSL certificate. After creation/import you’ll need to uncomment the ‘alb.ingress.kubernetes.io/certificate-arn’ setting and paste certificate’s ARN instead of YOUR_HTTPS_CERTIFICATE_ARN in the routes.yml file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

...
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: thingsboard
  name: tb-http-loadbalancer
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    # Uncomment the following line to enable HTTPS. Don't forget to replace YOUR_CERTIFICATE_ARN with the correct value
    # See https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-https-listener.html#https-listener-certificates for more info
    # alb.ingress.kubernetes.io/certificate-arn: YOUR_CERTIFICATE_ARN
...

Step 6. Configure secure MQTT connection

One-way TLS

The simplest way to configure MQTTS is to make your MQTT load balancer (AWS NLB) to act as a TLS termination point. This way we setup the one-way TLS connection, where the traffic between your devices and load balancers is encrypted, and the traffic between your load balancer and MQTT Transport is not encrypted. There should be no security issues, since the ALB/NLB is running in your VPC. The only major disadvantage of this option is that you can’t use “X.509 certificate” MQTT client credentials, since information about client certificate is not transferred from the load balancer to the ThingsBoard MQTT Transport service.

To enable the one-way TLS:

  • Use AWS Certificate Manager to create or import SSL certificate.
  • Locate 2 places marked as “Uncomment the following lines to enable one-way MQTTS” in the “routes.yml” and paste certificate’s ARN instead of YOUR_MQTTS_CERTIFICATE_ARN.

Two-way TLS

The more complex way to enable MQTTS is to obtain valid (signed) TLS certificate and configure it in the MQTT Transport. The main advantage of this option is that you may use it in combination with “X.509 certificate” MQTT client credentials.

To enable the two-way TLS:

Follow this guide to create a .jks file with the SSL certificate. Afterwards, you need to set MQTT_SSL_KEY_STORE_PASSWORD and MQTT_SSL_KEY_PASSWORD environment variables in the tb-services.yml file to the corresponding key-store and certificate key passwords.

You’ll need to create a config-map with your JKS file, you can do it by calling command:

1
2

kubectl create configmap tb-mqtts-config \
             --from-file=server.jks=YOUR_JKS_FILENAME.jks -o yaml --dry-run=client | kubectl apply -f -

where YOUR_JKS_FILENAME is the name of your .jks file. Then, uncomment all sections in the ‘tb-node.yml’ file that are marked with “Uncomment the following lines to enable two-way MQTTS”. Also, uncomment sections in the ‘routes.yml’ file that is marked with the same “Uncomment the following lines to enable two-way MQTTS” comment.

Step 7. Installation

Edit “tb-node-db-configmap.yml” and replace YOUR_RDS_ENDPOINT_URL and YOUR_RDS_PASSWORD with the values you have obtained during step 4.

Execute the following command to run the initial setup of the database. This command will launch short-living ThingsBoard pod to provision necessary DB tables, indexes, etc

1

 ./k8s-install-tb.sh --loadDemo

Where:

  • --loadDemo - optional argument. Whether to load additional demo data.

After this command finish you should see the next line in the console:

1

Installation finished successfully!

Otherwise, please check if you set the PostgreSQL URL in the tb-node-db-configmap.yml correctly.

Step 8. Starting

Execute the following command to deploy resources:

1

 ./k8s-deploy-resources.sh

After few minutes you may call kubectl get pods. If everything went fine, you should be able to see tb-node-0 pod in the READY state.

Step 9. Validate the setup

Validate Web UI access

Now you can open ThingsBoard web interface in your browser using DNS name of the load balancer.

You can see DNS name (the ADDRESS column) of the HTTP load-balancer using command:

1

kubectl get ingress

You should see the similar picture:

image

Use the following default credentials:

  • System Administrator: sysadmin@thingsboard.org / sysadmin

If you installed DataBase with demo data (using --loadDemo flag) you can also use the following credentials:

  • Tenant Administrator: tenant@thingsboard.org / tenant
  • Customer User: customer@thingsboard.org / customer

Validate MQTT/CoAP access

To connect to the cluster via MQTT or COAP you’ll need to get corresponding service, you can do it with command:

1

kubectl get service

You should see the similar picture:

image

There are two load-balancers:

  • tb-mqtt-loadbalancer-external - for MQTT protocol
  • tb-coap-loadbalancer-external - for COAP protocol

Use EXTERNAL-IP field of the load-balancers to connect to the cluster.

Troubleshooting

In case of any issues you can examine service logs for errors. For example to see ThingsBoard node logs execute the following command:

1

kubectl logs -f tb-node-0

Or use kubectl get pods to see the state of the pods. Or use kubectl get services to see the state of all the services. Or use kubectl get deployments to see the state of all the deployments. See kubectl Cheat Sheet command reference for details.

Upgrading to new ThingsBoard version

Merge your local changes with the latest release branch from the repo you have used in the Step 1.

In case when database upgrade is needed, execute the following commands:

1

 ./k8s-upgrade-tb.sh --fromVersion=[FROM_VERSION]

Where:

  • FROM_VERSION - from which version upgrade should be started. See Upgrade Instructions for valid fromVersion values.

Note: You may optionally stop the tb-node pods while you run the upgrade of the database. This will cause downtime, but will make sure that the DB state will be consistent after the update. Most of the updates do not require the tb-nodes to be stopped.

Once completed, execute deployment of the resources again. This will cause rollout restart of the thingsboard components with the newest version.

1

./k8s-deploy-resources.sh

Cluster deletion

Execute the following command to delete all ThingsBoard pods:

1

./k8s-delete-resources.sh

Execute the following command to delete all ThingsBoard pods and configmaps:

1

./k8s-delete-all.sh

Execute the following command to delete EKS cluster (you should change the name of the cluster and zone):

1

eksctl delete cluster -r us-east-1 -n thingsboard -w