Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Page Properties


Target release
Epic
Document status
Status
titleDRAFT
Document owner

Trip Gilman

DesignerTech leadTechnical writersQA

Objective

Provide a central management API and administrative UI that automates the lifecycle of cloud infrastructure elements.

Success metrics

...

Assumptions

Milestones

...

Glossary

Cloud Foundation

A set of global services that manage the OpenMethods' cloud infrastructure.  These services include:

  • Cloud Manager
  • Docker Registry
  • AWS Cloudwatch
  • Redundant OAuth Gateways

Shard

An isolated and independent copy of the OpenMethods cloud infrastructure.  A Shard contains three types of infrastructure elements:

  • Core Components - Application Manager, Redundant Cloud Gateways
  • Shared Versioned Components - Web MediaBar, Config Server, etc
  • Dedicated Versioned Components - HIS, QueueAdapter, etc

Deployment Ring

A logical grouping of Shared and Dedicated components related to a single major.minor version of a product.  A Deployment Ring may contain components located around the world or within a single AWS region depending on the scope and nature of its parent Shard.

Cloud Management VPC

An AWS network structure that is intended to contain all services of the Cloud Foundation.  There is a single Cloud Management VPC located in US_EAST_1 (Northern Virginia).  All Shard VPCs will have connections and routes to this central Cloud Management VPC.

Shard Management VPC

An AWS network structure that is intended to contain all Core and Shared Versioned Components for a Shard.  Each Shard has a single Shard Management VPC located in the home region for the Shard.  For instance, the GDPR Shard's Management VPC is located in EU_WEST_3 (Paris).

Shard Service VPC

An AWS network structure that is intended to contain a set of Dedicated Versioned Components for a Shard.  A Shard will have any number of Service VPCs that are possibly located around the world.  The General Population Shard is meant to service customers around the world and as such, it will have many Service VPCs.  A Service VPC will also provide secure access to customer resources through an IPSEC infrastructure provided by Aviatrix.

Requirements

...

Status
colourRed
titleHIGH

...

Architecture

Cloud Foundation

Image Removed

Shard Management Infrastructure

Image Removed

Deployment Ring Shared Infrastructure

Image Removed

Putting it all Together

Image Removed

AWS Network Design

There are several constraints that must be kept in mind when designing a world wide AWS cloud platform:

...

Document owner
Designer
Tech lead
Technical writers
QA


Objective

Provide a central management API and administrative UI that automates the lifecycle of cloud infrastructure elements.

Success metrics

GoalMetric


Assumptions

Milestones

Roadmap Planner
maplinks
timelinetrue
pagelinks
source %7B%22title%22%3A%22Roadmap%20Planner%22%2C%22timeline%22%3A%7B%22startDate%22%3A%222018-07-16%2000%3A00%3A00%22%2C%22endDate%22%3A%222019-06-16%2000%3A00%3A00%22%2C%22displayOption%22%3A%22MONTH%22%7D%2C%22lanes%22%3A%5B%7B%22title%22%3A%22Dashboard%22%2C%22color%22%3A%7B%22lane%22%3A%22%23f6c342%22%2C%22bar%22%3A%22%23fadb8e%22%2C%22text%22%3A%22%23594300%22%2C%22count%22%3A1%7D%2C%22bars%22%3A%5B%7B%22title%22%3A%22Feature%201%22%2C%22description%22%3A%22This%20is%20the%20first%20bar.%22%2C%22startDate%22%3A%222018-07-01%2000%3A00%3A00%22%2C%22duration%22%3A2%2C%22rowIndex%22%3A0%2C%22id%22%3A%22eeb2d902-e723-4ff0-a10d-43a5793089a4%22%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22title%22%3A%22Feature%202%22%2C%22description%22%3A%22This%20is%20the%20second%20bar.%22%2C%22startDate%22%3A%222018-09-02%2011%3A38%3A36%22%2C%22duration%22%3A2.297029702970297%2C%22rowIndex%22%3A1%2C%22id%22%3A%220e8f437c-39f1-4d70-b85b-3ca89fd57a00%22%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A0%2C%22startDate%22%3A%222018-11-15%2006%3A10%3A41%22%2C%22id%22%3A%2290251c8c-3b74-4d2b-b850-684ffd40514b%22%2C%22title%22%3A%22Feature%203%22%2C%22description%22%3A%22%22%2C%22duration%22%3A2.00990099009901%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A1%2C%22startDate%22%3A%222019-01-18%2019%3A14%3A51%22%2C%22id%22%3A%22d0529fe8-6c8c-45e8-8078-d75c2f063dd9%22%2C%22title%22%3A%22Feature%204%22%2C%22description%22%3A%22%22%2C%22duration%22%3A2.01980198019802%2C%22pageLink%22%3A%7B%7D%7D%5D%7D%2C%7B%22title%22%3A%22Notification%22%2C%22color%22%3A%7B%22lane%22%3A%22%233b7fc4%22%2C%22bar%22%3A%22%236c9fd3%22%2C%22text%22%3A%22%23ffffff%22%2C%22count%22%3A1%7D%2C%22bars%22%3A%5B%7B%22title%22%3A%22iOS%20App%22%2C%22description%22%3A%22This%20is%20the%20third%20bar.%22%2C%22startDate%22%3A%222018-07-13%2021%3A23%3A10%22%2C%22duration%22%3A2.5%2C%22rowIndex%22%3A0%2C%22id%22%3A%22dd93afcd-2704-4253-adb8-809a336cc5ba%22%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A0%2C%22startDate%22%3A%222018-10-10%2004%3A59%3A24%22%2C%22id%22%3A%228dc8bc2d-b7cb-44a8-b526-7d1ede51f690%22%2C%22title%22%3A%22Android%22%2C%22description%22%3A%22%22%2C%22duration%22%3A2.4752475247524752%2C%22pageLink%22%3A%7B%7D%7D%5D%7D%5D%2C%22markers%22%3A%5B%7B%22title%22%3A%22Milestone%201%22%2C%22markerDate%22%3A%222018-07-31%2016%3A38%3A01%22%7D%2C%7B%22markerDate%22%3A%222018-10-08%2001%3A25%3A32%22%2C%22title%22%3A%22Go%2FNo%20go%22%7D%2C%7B%22markerDate%22%3A%222018-12-31%2001%3A54%3A03%22%2C%22title%22%3A%22Milestone%202%22%7D%5D%7D
titleRoadmap%20Planner
hashc013a8cf46e3276583f2ddb4893243e4

Glossary

Cloud Foundation

A set of global services that manage the OpenMethods' cloud infrastructure.  These services include:

  • Cloud Manager
  • Docker Registry
  • AWS Cloudwatch
  • Redundant OAuth Gateways

Shard

An isolated and independent copy of the OpenMethods cloud infrastructure.  A Shard contains three types of infrastructure elements:

  • Core Components - Application Manager, Redundant Cloud Gateways
  • Shared Versioned Components - Web MediaBar, Config Server, etc
  • Dedicated Versioned Components - HIS, QueueAdapter, etc

Deployment Ring

A logical grouping of Shared and Dedicated components related to a single major.minor version of a product.  A Deployment Ring may contain components located around the world or within a single AWS region depending on the scope and nature of its parent Shard.

Cloud Management VPC

An AWS network structure that is intended to contain all services of the Cloud Foundation.  There is a single Cloud Management VPC located in US_EAST_1 (Northern Virginia).  All Shard VPCs will have connections and routes to this central Cloud Management VPC.

Shard Management VPC

An AWS network structure that is intended to contain all Core and Shared Versioned Components for a Shard.  Each Shard has a single Shard Management VPC located in the home region for the Shard.  For instance, the GDPR Shard's Management VPC is located in EU_WEST_3 (Paris).

Shard Service VPC

An AWS network structure that is intended to contain a set of Dedicated Versioned Components for a Shard.  A Shard will have any number of Service VPCs that are possibly located around the world.  The General Population Shard is meant to service customers around the world and as such, it will have many Service VPCs.  A Service VPC will also provide secure access to customer resources through an IPSEC infrastructure provided by Aviatrix.

Requirements

#RequirementUser StoryImportanceJira IssueNotes
1
Status
colourRed
titleHIGH


2




Architecture

Cloud Foundation

Image Added

Shard Management Infrastructure

Image Added

Deployment Ring Shared Infrastructure

Image Added

Putting it all Together

Image Added

AWS Network Design

There are several constraints that must be kept in mind when designing a world wide AWS cloud platform:

  • Limited number of AWS resources (some can be increased, some can’t)
  • Avoid networking conflicts between AWS regions as well as customer networks
  • Consistent numbering for our various cloud shards (general population, GDPR, GovCloud, etc)
  • Ease automated assignment of networks
  • Support for two availability zones per region


There are three types of VPCs we will be utilizing: Cloud Management VPC, Shard Management VPCs and Shard Service VPCs.  A single Cloud Management VPC holds the centralized cloud management services for the OpenMethods Cloud solution.  The Cloud Management VPC will occupy the 10.255.0.0/16 address space.  Each cloud shard will have a single Shard Management VPC that hosts the common, centralized components that are shared between customers that reside in the shard.  An initial Class B network is dedicated to each shard.  The Shard Management VPC occupies the lower 8 Class C networks, which are split between two availability zones for redundancy.  The remaining address space is used for Shard Service VPCs to contain instances.  Additional Class B networks can be assigned to a Shard if more address space is needed.


Shard Networking

Shard Id

Base CIDR

Availability Zone A Subnet

Availability Zone B Subnet

1 – Development

10.254.0.0/16

10.254.0.0/22 (1019 hosts)

10.254.4.0/22

2 – Gen Pop

10.253.0.0/16

10.253.0.0/22

10.253.4.0/22

3 – GDPR

10.252.0.0/16

10.252.0.0/22

10.252.4.0/22

4 - GovCloud?

10.251.0.0/16

10.251.0.0/22

10.251.4.0/22


The benefit of this approach allows you to tell that the server is in a Shard Management VPC and which shard it’s a member of just based on the server’s IP address.  An address that falls within 10.254.0.0-10.254.7.255 will always be a Shard Management VPC and the range of the third number will tell you which Shard Service VPC it belongs to, i.e 10.254.8.0 belongs to Shard 1 North Virginia.


The second type of VPC we will use is a Shard Service VPC.  Shard Service VPCs will host the services related to specific customers and the IPSec connectivity needed to communicate with any on premise CTI or data endpoints.  Each shard will have any number of service VPCs distributed around the globe.  As with Shard Management VPCs, Shard Service VPCs belong to a single cloud shard and are not shared.  The full service VPC address space is laid out below with grey rows not being implemented today but may be in the future:


Geography

AWS Region

Service VPC Id

Service VPC CIDR

Availability Zone A Subnet

Availability Zone B Subnet

1 – Development

N. Virginia (us-east-1)

1

10.254.8.0/21

10.254.8.0/22 (1019 hosts)

10.254.12.0/22


Mumbai (ap-south-1)

2

10.254.16.0/21

10.254.16.0/22

10.254.20.0/22

2 - General PopulationN. Virginia (us-east-1)110.253.8.0/2110.253.8.0/22 (1019 hosts)10.253.12.0/22

Ohio (us-east-2)210.253.16.0/2110.253.16.0/2210.253.20.0/22

N California (us-west-1)310.253.24.0/2110.253.24.0/2210.253.28.0/22
3 - GDPRParis (eu-west-3)110.252.8.0/2110.252.8.0/22 (1019 hosts)10.252.12.0/22
4 - Gov CloudN. Virginia (us-east-1)110.251.8.0/2110.251.8.0/22 (1019 hosts)10.251.12.0/22


The vast majority of the time there will only be 1 Service VPC in a given geographical area associated with a shard.  As with Core VPCs, you’ll be able to tell which geographical region based on the second octet.  The associated shard can be determined by looking at the range of the third octet, 10.254.0.0 – 10.254.31.255 indicates shard 1.  The Service VPC is determined by the sub range, 10.254.0.0 – 10.254.15.255 for Service VPC 1.  The availability zone can be determined by high or low values of the third octet within the VPC range.

User interaction and design

Create Shard

  1. An adminstrator logs in 

Open Questions

QuestionAnswerDate Answered

Out of Scope


Deprecated


Info

The cloud networking has undergone a significant rework.  The following is the original direction that has been abandoned


There are three types of VPCs we will be utilizing: Cloud Management VPC, Shard Management VPCs and Shard Service VPCs.  A single Cloud Management VPC holds the centralized cloud management services for the OpenMethods Cloud solution.  Each cloud shard will have a single Shard Management VPC that hosts the common, centralized components that are shared between customers that reside in the shard.  These Shard Management VPCs will share the 10.255.0.0/16 address space.  The first (left to right) three bits of the third byte will be used to indicate which of 8 possible shards the Shard Management VPC belongs to.  The forth bit is used to subnet between the two availability zones used for redundancy.  The remaining address space is used for instances within the VPC and availability zone subnets.  Here’s the table of shard prefixes in binary:


Shard Id

Bit Prefix

Core VPC CIDR

Availability Zone A Subnet

Availability Zone B Subnet

1 – Development

000

10.255.0.0/19

10.255.0.0/20 (4091 hosts)

10.255.16.0/20

2 – Gen Pop

001

10.255.32.0/19

10.255.32.0/20

10.255.48.0/20

3 – GDPR

010

10.255.64.0/19

10.255.64.0/20

10.255.80.0/20

4 - GovCloud?

011

10.255.96.0/19

10.255.96.0/20

10.255.112.0/20

5

100

10.255.128.0/19

10.255.128.0/20

10.255.144.0/20

6

101

10.255.160.0/19

10.255.160.0/20

10.255.176.0/20

7

110

10.255.192.0/19

10.255.192.0/20

10.255.208.0/20

8

111

10.255.224.0/19

10.255.224.0/20

10.255.240.0/20


The benefit of this approach allows you to tell that the server is in a Shard Management VPC and which shard it’s a member of just based on the server’s IP address.  An address that starts with 10.255.x.x will always be a Shard Management VPC and the range of the third number will tell you which shard the Shard Management VPC belongs to, i.e 10.255.0.0 – 10.255.31.255 belongs to shard 1.

...

For example, we consider Canada, US, and Mexico to be a single area called North America, while AWS has multiple regions within this area. To support the multi-dimensional deployments of Service VPCs, we will use a slightly different network addressing scheme.  Each geographical region will have one (or more as we expand) /16 address space used across the service vpcs in that area:


Geographical Area

Base CIDR

Expansion CIDR

North America

10.254.0.0/16

10.249.0.0/16

EMEA

10.253.0.0/16

10.248.0.0/16

APAC

10.252.0.0/16

10.247.0.0/16

Oceana

10.251.0.0/16

10.246.0.0/16

South America

10.250.0.0/16

10.245.0.0/16


As with Shard Management VPCs, Shard Service VPCs belong to a single cloud shard and are not shared.  The ownership of the Service VPC is designated by the first (left to right) three bits of the third byte.  Unlike Core VPCs, the address space needs to be divided up further to allow multiple Service VPCs to be created and to limit the size of the VPC subnets to eliminate customer overlap where possible.  This is handled by using the 4th bit of the third octet to identify the Service VPC specifically.  The fifth bit is used to segment off the availability zone subnet space.  The full service VPC address space is laid out below with grey rows not being implemented today but may be in the future:


Geography

Shard Id

Shard Prefix

Service VPC Id

Service VPC Prefix

Service VPC CIDR

Availability Zone A Subnet

Availability Zone B Subnet

North America

1 – Development

000

1

0

10.254.0.0/20

10.254.0.0/21 (2043 hosts)

10.254.8.0/21




2

1

10.254.16.0/20

10.254.16.0/21

10.254.24.0/21


2 - General Population

001

1

0

10.254.32.0/20

10.254.32.0/21

10.254.40.0/21




2

1

10.254.48.0/20

10.254.48.0/21

10.254.56.0/21


3 - GDPR

010

1

0

10.254.64.0/20

10.254.64.0/21

10.254.72.0/21




2

1

10.254.80.0/20

10.254.80.0/21

10.254.88.0/21


4 - Gov Cloud

011

1

0

10.254.96.0/20

10.254.96.0/21

10.254.104.0/21




2

1

10.254.112.0/20

10.254.112.0/21

10.254.120.0/21

EMEA1 – Development0001010.253.0.0/2010.253.0.0/21 (2043 hosts)10.253.8.0/21



2110.253.16.0/2010.253.16.0/2110.253.24.0/21

2 - General Population0011010.253.32.0/2010.253.32.0/2110.253.40.0/21



2110.253.48.0/2010.253.48.0/2110.253.56.0/21

3 - GDPR0101010.253.64.0/2010.253.64.0/2110.253.72.0/21



2110.253.80.0/2010.253.80.0/2110.253.88.0/21

4 - Gov Cloud0111010.253.96.0/2010.253.96.0/2110.253.104.0/21



2110.253.112.0/2010.253.112.0/2110.253.120.0/21
APAC1 – Development0001010.252.0.0/2010.252.0.0/21 (2043 hosts)10.252.8.0/21



2110.252.16.0/2010.252.16.0/2110.252.24.0/21

2 - General Population0011010.252.32.0/2010.252.32.0/2110.252.40.0/21



2110.252.48.0/2010.252.48.0/2110.252.56.0/21

3 - GDPR0101010.252.64.0/2010.252.64.0/2110.252.72.0/21



2110.252.80.0/2010.252.80.0/2110.252.88.0/21

4 - Gov Cloud0111010.252.96.0/2010.252.96.0/2110.252.104.0/21



2110.252.112.0/2010.252.112.0/2110.252.120.0/21
Oceana1 – Development00010





21



2 - General Population00110





21



3 - GDPR01010





21



4 - Gov Cloud01110





21


South America1 – Development00010





21



2 - General Population00110





21



3 - GDPR01010





21



4 - Gov Cloud01110





21



The vast majority of the time there will only be 1 Service VPC in a given geographical area associated with a shard.  As with Core VPCs, you’ll be able to tell which geographical region based on the second octet.  The associated shard can be determined by looking at the range of the third octet, 10.254.0.0 – 10.254.31.255 indicates shard 1.  The Service VPC is determined by the sub range, 10.254.0.0 – 10.254.15.255 for Service VPC 1.  The availability zone can be determined by high or low values of the third octet within the VPC range.

User interaction and design

Create Shard

  1. An adminstrator logs in 

Open Questions

...