Cloud Manager

Objective

Provide a central management API and administrative UI that automates the lifecycle of cloud infrastructure elements.

Success metrics

Assumptions

Milestones

Glossary

Cloud Foundation

A set of global services that manage the OpenMethods' cloud infrastructure.  These services include:

• Cloud Manager
• Docker Registry
• AWS Cloudwatch
• Redundant OAuth Gateways

Shard

An isolated and independent copy of the OpenMethods cloud infrastructure.  A Shard contains three types of infrastructure elements:

• Core Components - Application Manager, Redundant Cloud Gateways
• Shared Versioned Components - Web MediaBar, Config Server, etc
• Dedicated Versioned Components - HIS, QueueAdapter, etc

Deployment Ring

A logical grouping of Shared and Dedicated components related to a single major.minor version of a product.  A Deployment Ring may contain components located around the world or within a single AWS region depending on the scope and nature of its parent Shard.

Cloud Management VPC

An AWS network structure that is intended to contain all services of the Cloud Foundation.  There is a single Cloud Management VPC located in US_EAST_1 (Northern Virginia).  All Shard VPCs will have connections and routes to this central Cloud Management VPC.

Shard Management VPC

An AWS network structure that is intended to contain all Core and Shared Versioned Components for a Shard.  Each Shard has a single Shard Management VPC located in the home region for the Shard.  For instance, the GDPR Shard's Management VPC is located in EU_WEST_3 (Paris).

Shard Service VPC

An AWS network structure that is intended to contain a set of Dedicated Versioned Components for a Shard.  A Shard will have any number of Service VPCs that are possibly located around the world.  The General Population Shard is meant to service customers around the world and as such, it will have many Service VPCs.  A Service VPC will also provide secure access to customer resources through an IPSEC infrastructure provided by Aviatrix.

Requirements

Architecture

Cloud Foundation

Shard Management Infrastructure

Deployment Ring Shared Infrastructure

Putting it all Together

AWS Network Design

There are several constraints that must be kept in mind when designing a world wide AWS cloud platform:

• Limited number of AWS resources (some can be increased, some can’t)
• Avoid networking conflicts between AWS regions as well as customer networks
• Consistent numbering for our various cloud shards (general population, GDPR, GovCloud, etc)
• Ease automated assignment of networks
• Support for two availability zones per region

There are three types of VPCs we will be utilizing: Cloud Management VPC, Shard Management VPCs and Shard Service VPCs.  A single Cloud Management VPC holds the centralized cloud management services for the OpenMethods Cloud solution.  Each cloud shard will have a single Shard Management VPC that hosts the common, centralized components that are shared between customers that reside in the shard.  These Shard Management VPCs will share the 10.255.0.0/16 address space.  The first (left to right) three bits of the third byte will be used to indicate which of 8 possible shards the Shard Management VPC belongs to.  The forth bit is used to subnet between the two availability zones used for redundancy.  The remaining address space is used for instances within the VPC and availability zone subnets.  Here’s the table of shard prefixes in binary:

The benefit of this approach allows you to tell that the server is in a Shard Management VPC and which shard it’s a member of just based on the server’s IP address.  An address that starts with 10.255.x.x will always be a Shard Management VPC and the range of the third number will tell you which shard the Shard Management VPC belongs to, i.e 10.255.0.0 – 10.255.31.255 belongs to shard 1.

The second type of VPC we will use is a Shard Service VPC.  Shard Service VPCs will host the services related to specific customers and the IPSec connectivity needed to communicate with any on premise CTI or data endpoints.  Each shard will have any number of service VPCs distributed around the globe.  Although AWS often provides many data centers (called regions) globally, we will be creating a logical grouping of these AWS regions into larger geographical areas:

• North America
• South America
• EMEA
• APAC
• Oceana

For example, we consider Canada, US, and Mexico to be a single area called North America, while AWS has multiple regions within this area. To support the multi-dimensional deployments of Service VPCs, we will use a slightly different network addressing scheme.  Each geographical region will have one (or more as we expand) /16 address space used across the service vpcs in that area:

As with Shard Management VPCs, Shard Service VPCs belong to a single cloud shard and are not shared.  The ownership of the Service VPC is designated by the first (left to right) three bits of the third byte.  Unlike Core VPCs, the address space needs to be divided up further to allow multiple Service VPCs to be created and to limit the size of the VPC subnets to eliminate customer overlap where possible.  This is handled by using the 4^th through 7^th bits of the third octet to identify the Service VPC specifically.  The eighth bit is used to segment off the availability zone subnet space.  Here’s the breakdown of the Shard 1, North American network address space:

The vast majority of the time there will only be 1 or 2 Service VPCs in a given geographical area associated with a shard.  As with Core VPCs, you’ll be able to tell which geographical region based on the second octet.  The associated shard can be determined by looking at the range of the third octet, 10.254.0.0 – 10.254.31.255 indicates shard 1.  The Service VPC is determined by the sub range, 10.254.0.0 – 10.254.1.255 for Service VPC 1.  The availability zone can be determined by evens or odds of the third octet, even is AZ A which odd is AZ B.

User interaction and design

Open Questions

Out of Scope