CRUD micro-service

This post covers the CRUD microservice creation using Golang\Cassandra and packaging micro-service as docker container and pushing to Git\Docker repository.

The CRUD micro-service is deployed over AWS and integrated with AWS ELB and Region53 as http://deepagar.net:8080/api/v1/policies/common

Create a micro-service using Golang and Cassandra

Follow the steps at rest microservice with gin

This page also covers pushing the micro-service to github repository at crud-service

Create the Dockerfile container using above micro-service

Create the Dockerfile project

Push the Dockerfile project to Git

Create a repo

https://github.com/deepagargit/crud-service

Clone the repo

git clone https://github.com/deepagargit/crud-service

List the current branch

$ git branch -a

* develop

remotes/origin/HEAD -> origin/develop

remotes/origin/develop

Switch to develop branch

$ git checkout develop

Already on 'develop'

Create docker folder

$ mkdir docker

$ cd docker

Copy dockerfile and other project files

$ ls -R

cassandra crud crud-cassandra

./cassandra:

./crud:

./crud-cassandra:

build.sh Dockerfile ReadMe src

./crud-cassandra/src:

cassandra.yaml datastax.repo epel7.repo main.go start.sh supervisord.conf

Add files to git

Touch the gitkeep file in empty folder.

touch .gitkeep

git add .

git status

# On branch develop

# Changes to be committed:

# (use "git reset HEAD <file>..." to unstage)

# new file: cassandra/.gitkeep

# new file: crud-cassandra/Dockerfile

# new file: crud-cassandra/ReadMe

# new file: crud-cassandra/build.sh

# new file: crud-cassandra/src/cassandra.yaml

# new file: crud-cassandra/src/datastax.repo

# new file: crud-cassandra/src/epel7.repo

# new file: crud-cassandra/src/main.go

# new file: crud-cassandra/src/start.sh

# new file: crud-cassandra/src/supervisord.conf

# new file: crud/.gitkeep

Add User Config

git config --global user.email "deepak.aagarwal@gmail.com"

git config --global user.name "Deepak Agarwal"

Commit the files

$ git commit -m "Added Docker project for CRUD service with Golang"

[develop 9aaaedb] Added Docker project for CRUD service with Golang

11 files changed, 1421 insertions(+)

create mode 100644 docker/cassandra/.gitkeep

create mode 100644 docker/crud-cassandra/Dockerfile

create mode 100644 docker/crud-cassandra/ReadMe

create mode 100755 docker/crud-cassandra/build.sh

create mode 100755 docker/crud-cassandra/src/cassandra.yaml

create mode 100755 docker/crud-cassandra/src/datastax.repo

create mode 100755 docker/crud-cassandra/src/epel7.repo

create mode 100755 docker/crud-cassandra/src/main.go

create mode 100755 docker/crud-cassandra/src/start.sh

create mode 100755 docker/crud-cassandra/src/supervisord.conf

create mode 100644 docker/crud/.gitkeep

Push to the develop branch

git push origin develop

Username for 'https://github.com': deepagargit

Password for 'https://deepagargit@github.com':

Counting objects: 14, done.

Delta compression using up to 16 threads.

Compressing objects: 100% (13/13), done.

Writing objects: 100% (13/13), 16.14 KiB | 0 bytes/s, done.

Total 13 (delta 1), reused 0 (delta 0)

To https://github.com/deepagargit/crud-service

87c3096..9aaaedb develop -> develop

Build the container image

./crud-cassandra/build.sh

Spawn the container

docker run -it -p 8080:8080 --name p4 deepagar/crud-cassandra

sudo docker exec -i -t p4 bash

Validate cqlsh in container

$ cqlsh

Connected to Test Cluster at 127.0.0.1:9042.

[cqlsh 5.0.1 | Cassandra 2.2.4 | CQL spec 3.3.1 | Native protocol v4]

Use HELP for help.

cqlsh> use demo;

cqlsh:demo> select * from policy;

class | name | blob

-------+------+------

$ exit

Push the container image to docker repository

docker login

docker push deepagar/crud-cassandra

Deploy (Manually) the micro-service over AWS VM

Create the VM

Create the VM in EC2

Install docker

sudo curl -sSL https://get.docker.com/ | sh

sudo usermod -aG docker centos

List docker container

$ sudo docker ps -a

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

Start the docker contain using docker image from repository

docker run -it -p 8080:8080 --name p4 deepagar/crud-cassandra

Unable to find image 'deepagar/crud-cassandra:latest' locally

latest: Pulling from deepagar/crud-cassandra

fa5be2806d4c: Pull complete

98783644cfab: Pull complete

4609e0ebb5d4: Pull complete

c812b6264db1: Pull complete

ea1036165fa4: Pull complete

7a21760c41c7: Pull complete

542acddd307e: Pull complete

4ade23629f70: Pull complete

e94a69157b27: Pull complete

7e6c33318a29: Pull complete

96a5ae6fbdba: Pull complete

6500fd2abf12: Pull complete

c318cbc3e970: Pull complete

7a375ec1e6fb: Pull complete

b6495ac2f1b9: Pull complete

474f92c6f273: Pull complete

1f9b54b17fa3: Pull complete

dcec5e78b34d: Pull complete

39792fa55bc7: Pull complete

e5deca7a2da3: Pull complete

f19b231d0906: Pull complete

4003b883c0eb: Pull complete

4664ac8d5be6: Pull complete

cfe4c7737d0d: Pull complete

f8f5bc5925a5: Pull complete

1c8afedc0bc9: Pull complete

a6efcfa258d0: Pull complete

77fdb57d5964: Pull complete

ad9e0a7dcbb2: Pull complete

97e7e6dc8cdb: Pull complete

8e83abb7513b: Pull complete

Digest: sha256:0a76c40c82d269c717ecf8361a3077a8c398d9b3626efe1f9dd478ab1d30b0d8

Status: Downloaded newer image for deepagar/crud-cassandra:latest

Starting Cassandra

/usr/lib/python2.7/site-packages/supervisor/options.py:296: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.

'Supervisord is running as root and it is searching '

2015-12-20 11:29:38,777 CRIT Supervisor running as root (no user in config file)

2015-12-20 11:29:38,781 INFO supervisord started with pid 6

2015-12-20 11:29:39,787 INFO spawned: 'sshd' with pid 9

2015-12-20 11:29:39,789 INFO spawned: 'crud-service' with pid 10

2015-12-20 11:29:39,790 INFO spawned: 'cassandra-crud' with pid 11

2015-12-20 11:29:39,791 INFO spawned: 'cassandra' with pid 12

2015-12-20 11:29:40,798 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

2015-12-20 11:29:40,798 INFO success: crud-service entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

2015-12-20 11:29:40,798 INFO success: cassandra-crud entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

2015-12-20 11:29:40,798 INFO success: cassandra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

Validate accessing the crud interface

READ

curl -i http://52.35.62.110:8080/api/v1/policies/common

HTTP/1.1 404 Not Found

Content-Type: application/json; charset=utf-8

Date: Sun, 20 Dec 2015 11:31:57 GMT

Content-Length: 40

{"error":"no policy(s) into the table"}

Create

curl -i -X POST -H "Content-Type: application/json" -d "{ \"class\": \"common\", \"name\": \"email-1\", \"blob\": \"rule1\" }" http://52.35.62.110:8080/api/v1/policies

HTTP/1.1 201 Created

Content-Type: application/json; charset=utf-8

Date: Sun, 20 Dec 2015 11:34:24 GMT

Content-Length: 51

{"class":"common","name":"email-1","blob":"rule1"}

[centos@ip-172-31-14-230 ~]$

[centos@ip-172-31-14-230 ~]$ curl -i -X POST -H "Content-Type: application/json" -d "{ \"class\": \"common\", \"name\": \"email-2\", \"blob\": \"rule2\" }" http://52.35.62.110:8080/api/v1/policies

HTTP/1.1 201 Created

Content-Type: application/json; charset=utf-8

Date: Sun, 20 Dec 2015 11:34:32 GMT

Content-Length: 51

{"class":"common","name":"email-2","blob":"rule2"}

curl -i http://52.35.62.110:8080/api/v1/policies/common

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Date: Sun, 20 Dec 2015 11:35:01 GMT

Content-Length: 104

[{"class":"common","name":"email-1","blob":"rule1"},{"class":"common","name":"email-2","blob":"rule2"}]

Update

curl -i -X PUT -H "Content-Type: application/json" -d "{ \"blob\": \"{rule-x}\" }" http://52.35.62.110:8080/api/v1/policies/common/email-1

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Date: Sun, 20 Dec 2015 11:35:21 GMT

Content-Length: 54

{"class":"common","name":"email-1","blob":"{rule-x}"}

[centos@ip-172-31-14-230 ~]$

[centos@ip-172-31-14-230 ~]$ curl -i http://52.35.62.110:8080/api/v1/policies/common

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Date: Sun, 20 Dec 2015 11:35:24 GMT

Content-Length: 107

[{"class":"common","name":"email-1","blob":"{rule-x}"},{"class":"common","name":"email-2","blob":"rule2"}]

curl -i http://52.35.62.110:8080/api/v1/policies/common/email-1

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Date: Sun, 20 Dec 2015 11:36:15 GMT

Content-Length: 54

{"class":"common","name":"email-1","blob":"{rule-x}"}

Delete

curl -i -X DELETE http://52.35.62.110:8080/api/v1/policies/common/email-1

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Date: Sun, 20 Dec 2015 11:36:35 GMT

Content-Length: 54

{"class":"common","name":"email-1","blob":"{rule-x}"}

curl -i http://52.35.62.110:8080/api/v1/policies/common

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Date: Sun, 20 Dec 2015 11:36:41 GMT

Content-Length: 53

[{"class":"common","name":"email-2","blob":"rule2"}]

curl -i http://52.35.62.110:8080/api/v1/policies/common/email-2

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Date: Sun, 20 Dec 2015 11:36:50 GMT

Content-Length: 51

{"class":"common","name":"email-2","blob":"rule2"}

curl -i http://52.35.62.110:8080/api/v1/policies/common/email-1

HTTP/1.1 404 Not Found

Content-Type: application/json; charset=utf-8

Date: Sun, 20 Dec 2015 11:36:53 GMT

Content-Length: 31

{"error":"no policy(s) found"}

Integrate AWS Deployment with a domain using ELB and Region53

Create EC2 ELB and associate instance

The AWS EC2 ELB got created following the Getting Started with ELB.

The ELB could be accessed from EC2 -> EC2 Dashboards -> Load Balancers from AWS console.

EC2 ELB with DNS deepagar-ELB-1344047162.us-west-2.elb.amazonaws.com gets created

Access the CRUD service using AWS ELB

curl -i http://deepagar-ELB-1344047162.us-west-2.elb.amazonaws.com:8080/api/v1/policies/common

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Date: Sun, 20 Dec 2015 15:02:09 GMT

Content-Length: 53

Connection: keep-alive

[{"class":"common","name":"email-2","blob":"rule2"}]

Integrating with AWS Route 53

The first step is to register a new domain or transfer an existing domain. I did with new domain over AWS.

Getting Started with Route 53

Once domain is successfully registered (takes ~60 minutes), it is visible at route53 home

Create resource record sets to tell Amazon Route 53 how you want to route traffic for the domain. For more information, see Adding Resource Record Sets for a New Domain.

If you already have a hosted zone for your domain, go to the hosted zones page from route53 home

On the Hosted Zones page, choose the name of the hosted zone in which you want to create resource record sets. Click Create Record Set.

Follow Using domain-names with elb to create record-set.

Click Create Record Set. Under Create Record Set, do the following:

Leave the default name, which is the name of your domain.

From Type, select A — IPv4 address.

For Alias, click Yes. An alias enables Amazon Route 53 to associate your domain name with an AWS resource, such as a load balancer.

Click Alias Target. Select your load balancer from the list. The console adds the dualstack prefix. Note that the value of Alias Hosted Zone ID is based on the load balancer that you selected.

From Routing Policy, select Simple.

Leave Evaluate Target Health set to No.

Click Create.

Validate CRUD service using AWS Route53 domain name

curl -i http://deepagar.net:8080/api/v1/policies/common

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Date: Sun, 20 Dec 2015 15:41:34 GMT

Content-Length: 53

Connection: keep-alive

[{"class":"common","name":"email-2","blob":"rule2"}]

curl -i -X POST -H "Content-Type: application/json" -d "{ \"class\": \"common\", \"name\": \"email-3\", \"blob\": \"rule3\" }" http://deepagar.net:8080/api/v1/policies

HTTP/1.1 201 Created

Content-Type: application/json; charset=utf-8

Date: Sun, 20 Dec 2015 15:51:41 GMT

Content-Length: 51

Connection: keep-alive

{"class":"common","name":"email-3","blob":"rule3"}

curl -i http://deepagar.net:8080/api/v1/policies/common

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Date: Sun, 20 Dec 2015 15:51:47 GMT

Content-Length: 104

Connection: keep-alive