How CoinList Achieved 87.5% Cost Savings with a Scalable and Resilient Platform on AWS

🏢 ABOUT THE COMPANY

CoinList, one of the most popular launching platforms where early adopters invest in and trade the best digital assets through the purchase and sale of tokens, hired me via Toptal to help them containerize and deploy six Ruby on Rails (RoR) applications that were running on EC2 servers.

By leveraging Docker, Infrastructure as Code (IaC) using Terraform and Terraform Cloud, CI/CD pipelines on GitHub Actions integrated with AWS CodeDeploy, which allows adopting a GitOps approach, AWS ECS Fargate, and many other AWS services, a new, modern, and resilient platform for effortlessly executing its applications was delivered for CoinList.

⛰ CHALLENGES

CoinList faced unique challenges in modernizing its Infrastructure and applications. The previous setup on EC2 servers caused issues during token sales due to traffic spikes that consumed all computing resources, leading to a poor user experience for investors.

Additionally, their old CI/CD structure needed to support their growth and cause slowness during test and application build steps. Because of that condition, CoinList wanted to replace the old solution (Jenkins) with a new, more modern one (GitHub Actions).

✅ SOLUTION

The partnership with CoinList started in July 2023. We kicked off the project and decided to break it into milestones representing each application that would be modernized and migrated to the new platform. CoinList chose Amazon Elastic Container Service (ECS) with AWS Fargate to run containers without managing servers or clusters of Amazon EC2 instances. As they no longer had to provision, configure, or scale clusters of virtual machines, they could focus on other areas that helped the business growth.

The modernization and migration were ordered by difficulty level, where we migrated the most straightforward (with fewer dependencies) applications before the hardest ones.

I performed the following tasks to tackle this project:

• Developed a new Dockerfile for each RoR application to build the application image securely, such as running the application with a minimal-privileged user.

• Improved each application's existing Docker Compose structure so that Developers could reproduce the application and its backing services locally for development and testing purposes.

• Developed GitHub Actions CI/CD pipelines to automate software test, build, and deployment using the Blue/Green deployment model controlled by AWS CodeDeploy and on top of ECS Fargate.

• Developed Terraform modules from scratch (e.g., to autoscale ECS services) and from an existing codebase. These modules were stored in HCP Terraform (formerly known as Terraform Cloud) and then consumed to create and manage the Infrastructure.

• Implemented the new application fleets on ECS (API, Workers, Web) using Terraform. Most were integrated with application load balancers, which received the external traffic and forwarded it to the corresponding backend service.

Additionally, we used the GitOps approach to manage Infrastructure, where every change had to be approved through a peer review and maintained in GitHub repositories. Manual changes weren't allowed, which helped us use Git as the only source of truth.

I also communicated regularly with the client through Slack, constantly updating the project status (progress, problems, and plans). We tracked the project's progress through Linear integrated with GitHub.

🏆 OUTCOMES

• Massive savings on infrastructure costs (approximately 87.5% decrease) have been realized since the containerized approach allowed them to use more minor container instances (compared to the previously used EC2 instances) able to grow and shrink (autoscale) when needed (e.g., during a token sale event when their traffic increases). Ultimately, their main application cost has dropped from about $4 an hour to approximately $0.50.

• Scaling the main application (Compliance) in the old architecture took approximately 45 minutes and was error-prone. The modernization decreased this time to approximately 5 minutes and increased the confidence level in the deployment process.

• Implementing CI/CD pipelines has significantly standardized and accelerated CoinList's software delivery process, enhancing the efficiency of its Engineering team and streamlining its operations, making it more agile and responsive.

• The new Infrastructure was also faster than the previous one (EC2)—CoinList could process more requests using fewer hardware resources, contributing to massive savings.