Fintastic
ECS to EKS Migration, FinOps
AWS: FinOps, EKS, ArgoCD, KEDA, Karpenter, Costs, CI/CD
About
Fintastic is an AI-driven Financial Planning and Analysis (FP&A) platform designed to optimize planning and analysis processes for businesses. By leveraging advanced AI capabilities, Fintastic provides insights and automates financial planning, making it a crucial tool for organizations aiming to streamline their financial operations.
Challenge
Fintastic faced several challenges during their modernization process:
- Issues migrating from ECS to EKS.
- A need for enhanced observability of their environments.
- Better control and visibility over cloud costs, including identifying cost spikes and reducing unnecessary expenses.
Solution
Dcode.tech provided a comprehensive solution to address Fintastic's challenges:
- Infrastructure Management: Utilized Terraform to manage all infrastructure, ensuring consistency and repeatability.
- Autoscaling and Service Management: Configured Karpenter for autoscaling nodes in the EKS cluster and managed services using Helm charts with ArgoCD deployments. We also added KEDA for autoscaling pods based on custom metrics and scaling to zero.
- Monitoring and Observability: Implemented Logz.io for log management, and Prometheus and Grafana for metrics monitoring. Configured OpenTelemetry to send data to Logz.io.
- Secrets Management: Used AWS Secrets Manager to manage application secrets securely.
- CICD Deployments: Modified CICD pipelines to work with EKS.
- Cost Control and Visibility: Tagged all AWS resources to track and manage costs effectively. Built custom Cost Explorer dashboards to monitor cost spikes and identified cost drivers like ECR, Inspector, S3, CloudWatch logs, and ALBs.
- Cost Reduction Measures:
Created ECR Lifecycle policies to remove unneeded images, reducing costs associated with ECR and Inspector.
Configured Ingress groups to consolidate multiple Ingresses to a single ALB, and a single ALB with multiple target groups for legacy ECS-based services.
Established a VPC Endpoint for CloudWatch to reduce traffic costs.
Removed unneeded log groups.
Add budget alarms to monitor spending.
Results
The migration and optimization project led to significant improvements for Fintastic:
- Cost Reduction: Implemented measures resulted in lowered cloud costs, including reduced ECR, Inspector, and ALB expenses.
- Enhanced Control and Visibility: Custom Cost Explorer dashboards and budget alarms provided better visibility and control over cloud spending.
- Improved Observability: The use of Logz.io, Prometheus, and Grafana improved monitoring and troubleshooting capabilities.
- Streamlined Infrastructure Management: Terraform and ArgoCD deployments streamlined infrastructure management and deployment processes.
- Kubernetes Adoption: Fintastic successfully migrated to Kubernetes for all environments, benefiting from its scalability and management features.