Elastic Container Service - Lexicon from CLI

Prerequisites

Firstly install and configure both the AWSCLI and ECSCLI.

These are the steps to use the CLI to generate the Lexicon infrastructure with a Fargate Task. All .json configuration files are available on github.

Cluster

1
2
3
4
5
6
7
8
--- Create ECS cluster using config and profile setup from `Prerequisites`
ecs-cli up --cluster-config carl_configuration_name --ecs-profile carl_ecs_cli_profile

--- Get default security group ID for the VPC. Use the VPC ID from the previous output
aws ec2 describe-security-groups --filters Name=vpc-id,Values=vpc-00000000000000000 --region ap-southeast-2

--- Security group rule to allow inbound access on port 80
aws ec2 authorize-security-group-ingress --group-id sg-0000000000000000 --protocol tcp --port 80 --cidr 0.0.0.0/0 --region ap-southeast-2

Compose File

Here you will need subnet ID 1, subnet ID 2 and security group ID values which would have been displayed when you ran ecs-cli up. If you cleared the console you can use aws ec2 describe-vpcs and look at the Tags to make an educated guess.

References

Elastic Container Registry

Amazon Elastic Container Registry (ECR) is a fully-managed Docker container registry that makes it easy for developers to store, manage, and deploy Docker container images.

Registry

Create ECR Repository from the AWS Console, example: lexicon-webmvc

1
2
000000000000.dkr.ecr.ap-southeast-2.amazonaws.com/lexicon-webmvc
This is the same as a docker hub image: microsoft/mssql-server-linux:2017-CU13

Push image from Local

Use the CLI to generate your login command

1
2
3
4
5
6
7
aws ecr get-login --no-include-email --region ap-southeast-2

docker login -u AWS -p [MASSIVE ENCRIPTED BLOB] https://000000000000.dkr.ecr.ap-southeast-2.amazonaws.com

docker tag docker-compose-lexicon_lexicon-webmvc 000000000000.dkr.ecr.ap-southeast-2.amazonaws.com/lexicon-webmvc

docker push 000000000000.dkr.ecr.ap-southeast-2.amazonaws.com/lexicon-webmvc

Now the image will be available at the ECR.

References

Elastic Container Service Simple Demo

This runs on top of Amazon Elastic Compute Cloud (Amazon EC2) and the steps below follow the AWS GUI (graphical user interface). I learnt most of the content below by following Arthur Ulfeldt‘s tutorial Deploying Docker to AWS.

This demo will simply write the current datetime to a volume using the busybox image, the container is called data-source. Another container running nginx will then display this data, this container is called data-server.

Simple Demo Infrastructure Overview

Calling data-server on its public IP will then display as follows:

nginx data-server

Setup

Create your account at https://aws.amazon.com/ login and under AWS Management Console type or look for ECS, this will take you to the ECS Dashboard

Cluster

Create cluster type of launch type FARGATE, it may be called something like Networking only and mention Powered by AWS Fagate - things in IT change daily :)

  • Cluster name: simple-demo2
  • Check Create VPC
  • Tag: Description|simple-demo2 (this is the key|value)
  • Check Enable container insights for CloudWatch
  • From the CLI you can view all clusters:

Task Definition

Select Task Definitions -> Create new Task Definition

  • Select the FARGATE template
  • Task Definition Name: task-definition-data-server
  • Requires Compatibilities: FARGATE
  • Task Role: escTaskExecutionRole
  • Network Mode: awsvpc
  • Task execution role: ecsTaskExecutionRole
  • Task memory (GB): 0.5GB
  • Task CPU (vCPU): 0.25 vCPU

Volumes:

  • create one called shared-data

CONTAINER 1

  • Container name: data-source
  • Image: busybox
  • Memory Limits (MiB): 128
  • Entry point: sh, -c, while true; do echo $(date) > /shared-data/index.html; sleep 5; done
  • Mount points: select source volume, and set the path /shared-data to match the shell script
  • Log configuration: check Auto-configure CloudWatch Logs

CONTAINER 2

  • Container name: data-server
  • Image: nginx
  • Memory Limits (MiB): 128
  • Port mappings: 0 -> 80
  • Mount points: select source volume, and set the path /usr/share/nginx/html

Run Task

Now manually run the above task definition from Tasks tab, Run new Task.

Once it starts up select the running task, copy its Public IP into a browsers and you should see the current time update every 5 seconds. Per the image above this IP was 3.104.47.134

References

AWS Install & Configure CLI

Prerequisites

To configure the below you will need the required access key id and secret access key which you can get from the AWS AMI Console (Identity and Access Management) you will need to be logged in. Any commands that point to a .json configuration file are available on github.

AWSCLI

  1. Download and install using the MSI installer for windows.

This will live in C:\Program Files\Amazon\AWSCLI and should then work from any terminal.

  1. Check version
1
aws --version
  1. Configure with the keys you got from the IAM in the amazon console.
1
2
3
4
5
C:\> aws configure
AWS Access Key ID [None]: HOEHOEHOEHOHEOHEOHE
AWS Secret Access Key [None]: HO/hehOehoHEOHEhohEOHeohEOH+EohOEe
Default region name [None]: ap-southeast-2
Default output format [None]:

This creates these files which you can edit with any text editor.

1
2
3
4
5
6
7
8
~ C:\Users\[USERNAME]\.aws\credentials
[default]
aws_access_key_id = HOEHOEHOEHOHEOHEOHE
aws_secret_access_key = HO/hehOehoHEOHEhohEOHeohEOH+EohOEe

~ C:\Users\[USERNAME]\.aws\config
[default]
region = ap-southeast-2
  1. Then you can test it works
1
aws iam list-roles
  1. Create the task execution IAM role
1
2
3
4
5
6
--- Create the task execution role
C:\dev\aws\ami>
aws iam --region ap-southeast-2 create-role --role-name ecsTaskExecutionRole --assume-role-policy-document file://task-execution-assume-role.json

--- Attach the task execution role policy
aws iam --region ap-southeast-2 attach-role-policy --role-name ecsTaskExecutionRole --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy

ECSCLI

This will live in C:\Program Files\Amazon\ECSCLI

  1. Run windows powersell as administrator
1
2
3
4
5
--- create folder
New-Item C:\Program Files\Amazon\ECSCLI -type directory

--- install
Invoke-WebRequest -OutFile ‘C:\Program Files\Amazon\ECSCLI\ecs-cli.exe’ https://amazon-ecs-cli.s3.amazonaws.com/ecs-cli-windows-amd64-latest.exe
  1. Edit the environment variables and add C:\Program Files\Amazon\ECSCLI to the PATH variable field

  2. Restart powersell and check version

1
ecs-cli --version
  1. Configure with the same keys used above

I used the profile names carl_ecs_cli_profile and carl_configuration_name below when setting up ECS for the Lexicon via the AWS CLI.

1
2
3
ecs-cli configure profile --profile-name carl_ecs_cli_profile --access-key HOEHOEHOEHOHEOHEOHE --secret-key HO/hehOehoHEOHEhohEOHeohEOH+EohOEe

ecs-cli configure --cluster ClusterName1 --default-launch-type FARGATE --region ap-southeast-2 --config-name carl_configuration_name

Note that --cluster needs to satisfy regular expression pattern: [a-zA-Z][-a-zA-Z0-9]*

This creates these files which you can edit with any text editor.

1
2
3
4
5
~ C:\Users\[USERNAME]\AppData\Local\ecs\config
: carl_configuration_name

~ C:\Users\[USERNAME]\AppData\Local\ecs\credentials
: carl_ecs_cli_profile

References

Elastic Container Service Stack Overview

VPC - Virtual Private Cloud

A virtual private cloud (VPC) is a virtual network dedicated to your AWS account. It is logically isolated from other virtual networks in the AWS Cloud. You can launch your AWS resources, such as Amazon EC2 instances, into your VPC. A VPC spans all the Availability Zones in the region.

When you create a VPC, you must specify a range of IPv4 addresses for the VPC in the form of a Classless Inter-Domain Routing (CIDR) block; for example, 10.0.0.0/16. This is the primary CIDR block for your VPC.

Amazon Elastic Container Service Stack Overview

Amazon just wraps all this up under “Web Servers” in their official diagram

Amazon Web Servers

Subnets

After creating a VPC, you can add one or more subnets in each Availability Zone. When you create a subnet, you specify the CIDR block for the subnet, which is a subset of the VPC CIDR block.

EC2 - Elastic Compute Cloud

Amazon Elastic Compute Cloud (EC2) gives you a platform for your infrastructure to live in. The below is a high level diagram showing the structure and relationships. For access you will need to create an account at https://aws.amazon.com/

ECS - Elastic Container Service

This is the AWS container orchestration service that supports Docker.

Cluster

Service

Start from task definition and keep it running.

Task

You can manually run a task from a task definition but its best to use a Service to keep it running. A Task is a running collection of docker containers.

Container

Docker container spun up from its image.