ArraY Alliance Application Modernization
Key Facts:
Array Crew is an platform, with a mission to support professionals in the film and television industry from underrepresented populations. Array Crew provides a community for below-the-line talent from diverse backgrounds and the opportunity to be connected with hiring managers from every major production studio and streaming service.
Challenge:
As Array Crew business expanded along with increase in number of customers the need for computing resources increased, and company faced scaling, availability, agile, reliability and cost challenges. The company found that increasing the number of servers to support business growth delivered diminishing returns on compute performance but limited ability to scale efficiently along with cost-effectively. Increasing the number of servers also caused problem on monitoring all the servers.
Array crew engaged Continuum Innovations to help design, architect, and manage its entire IT infrastructure and set about helping the company achieve regulatory compliance, stabilizing their IT infrastructure, and optimizing security to support and protect the company’s SaaS platform.
Continuum Innovations did this while providing day-to-day IT support and management of all of their cloud architecture, as well as providing expert leadership on performance enhancements. Through Amazon Web Services, Continuum Innovations was able to provision and operate Array Crew infrastructure.
The Continuum Innovations Solution:
AWS Services Used:
AWS CloudFormation
Amazon Virtual Private Cloud(VPC)
Amazon CloudWatch
Amazon SNS
Amazon Elastic Kubernetes Service(EKS)
Amazon EC2
Amazon Open Search
AWS Lambda
AWS Code Pipeline
AWS Code Build
AWS Elastic Container Repository(ECR)
Third Party Services:
Datadog
Slack
PagerDuty
Third party applications or services used:
Datadog:
Datadog is an observability service for cloud-scale applications, providing monitoring of servers, databases, tools, and services, through a SaaS-based data analytics platform. It is a monitoring and analytics tool for information technology (IT) and DevOps teams that can be used to determine performance metrics as well as event monitoring for infrastructure and cloud services.
PagerDuty:
PagerDuty is an alarm aggregation and dispatching service for system administrators and support teams. It collects alerts from your monitoring tools, gives you an overall view of all of your monitoring alarms, and alerts an on duty engineer if there’s a problem.
Slack:
Slack is a messaging app that connects people to the information that they need. Slack makes it easy to contact your colleagues – you can message anyone inside or outside your organisation and collaborate just as you would in person. People can work in dedicated spaces called channels, which bring the right people and information together.
Architecture Diagrams of the specific customer deployment:
AWS Services used as part of solution:
Amazon Virtual Private Cloud(VPC):
To launch instances on a private, isolated network, Continuum Innovations leveraged AWS VPC. This supported multiple subnets, route tables, Internet and NAT gateways, including NACLs. In addition this incorporates the benefits of AWS’ scalable infrastructure and automatic failover from the provisioned virtual private gateway. This service also provides network packet logging with VPC flow logs.
Amazon CloudWatch:
Amazon CloudWatch is a monitoring and observability service built for DevOps engineers, developers, site reliability engineers (SREs), and IT managers. Continuum Innovations used this service to get gain complete visibility of AWS resources, applications, and services running on AWS by collecting and monitoring operational data in the form of logs, metric and events.
Amazon SNS:
Amazon SNS is a fully managed service that is used to send notifications to a subscription. The content of the notification is defined by the SNS topic in AWS. Continuum used to send a notification to a central channel in slack along with Project Manager when there is a change in code pipeline states.
Amazon Elastic Kubernetes Service(EKS):
Amazon EKS is a managed service, which allows to run Kubernetes on AWS without having to maintain its own Kubernetes control plane. Continuum used EKS in multiple AWS Availability Zones which makes the application with high availability and EKS can automatically detect and replace unhealthy nodes and provide zero-downtime upgrades and the latest security patches. We can also use on-demand computing resources flexibly and further reduce computing costs with Amazon EC2 Instances.
Amazon EC2:
Amazon Elastic Compute Cloud (Amazon EC2) provides scalable computing capacity in the Amazon Web Services (AWS) Cloud. Continuum Innovations used EC2 along with autoscaling for EKS nodes group so no of nodes can scale automatically according to the demand.
Amazon Open Search:
OpenSearch enables to easily ingest, secure, search, aggregate, view, and analyze data. These capabilities are popular for use cases such as application search, log analytics, and more. With OpenSearch, we benefit from having an open source product you can use, modify, extend, monetize, and resell how we want. At the same time, we will continue to provide a secure, high-quality search and analytics suite with a rich roadmap of new and innovative functionality.
Amazon Lambda:
AWS Lambda is an event-driven, serverless computing platform provided by Amazon as a part of Amazon Web Services. It is a computing service that runs code in response to events and automatically manages the computing resources required by that code. Continuum used lambda function to send notifications to slack about code pipeline state.
Amazon Code Pipeline:
AWS CodePipeline is a fully managed continuous delivery service that helps to automate the release pipelines for fast and reliable application and infrastructure updates. Continuum created multiple code pipelines for multiple repository based on the branches.
Amazon Elastic Container Registry(ECR):
Amazon Elastic Container Registry (Amazon ECR) is a fully-managed Docker container registry that makes it easy for developers to store, manage, and deploy Docker container images. Continuum created multiple repositories to store the container images build by the code build. These images are used to deploy application in to Kubernetes nodes.
AWS CloudFormation:
AWS CloudFormation is a service that helps to model and set up AWS resources so that we can spend less time managing those resources and more time focusing on applications that run in AWS. We create a template that describes all the AWS resources that we want and CloudFormation takes care of provisioning and configuring those resources for you. We don’t need to individually create and configure AWS resources and figure out what’s dependent on what; CloudFormation handles that. Continuum mostly used cloudformation to provision aws services for infrastructure.
EKS:
Continuum has used CloudFormation nested stack template format to provision EKS cluster and nodes for EKS. The template provision EC2 along with auto scaling group as nodes for EKS. The template also creates attach role with required permission for EC2 instances to join EKS cluster.
VPC:
Continuum has used cloudformation template to provision VPC which includes subnets, Internet Gateway and NAT Gateway. We have total of 6 subnets which includes 4 private subnets and 2 public subnets.
Amazon Open Search:
OpenSearch is a distributed, open-source search and analytics suite used for a broad set of use cases like real-time application monitoring, log analytics, and website search. Continuum provisioned AWS Open Search for Search feature in our application. We store user data in open search so we can use search feature in our application to search for users based on particular fields.
Amazon Code Pipeline:
Continuum has provisioned code pipelines using cloudformation which includes 3 stages source, build and deploy. Github is the source, which will be triggered when there is a commit. The code build will get details from source artifacts and builds the container image and sends to ECR. The artifacts are stored in S3 bucket. Code deploy will deploy the containers in EKS using the container image in ECR.
AWS Lambda:
Continuum has used lambda function for sending custom notifications about code pipeline stage to slack. We have written lambda function code in python which will triggered whenever a stage in code pipeline is started.
Result:
Continuum Innovations solution helps Array Alliance meet increasing users demands proactively and cost effectively based on the latest AWS technologies. With the Amazon EKS and CI/CD, changes can be implemented in about 30 minutes, compared to the multiple hours or even days applications require to review the code, get approval, and deploy the fix. With the ongoing support and performance optimization from Continuum Innovations is also able to address Array Alliance evolving, complex cost optimization, reliability, and scalability needs.
