AWS Quick Tip: Optimizing Cloud Costs with Conditional Infrastructure

As developers, having a development environment that is as close as possible to the production environment is key for faster iteration. It gives us assurance that if something “works on my machine”, it will also work in production as well as all other environments.

With serverless applications that use infrastructure as code, we may easily find ourselves slowed down by constant redeployments of the whole stack. This can be a huge impact in terms of cost, time, and energy. A simple way to reduce this impact is by conditionally deploying resources. While conditional deployment is not a novel concept and is supported by AWS itself, this post aims to share an additional, practical approach to implementing conditional deployment.

Consider this common scenario: in a recent project, my team faced the challenge of deploying entire AWS stacks daily. This process, while necessary, was expensive. A prime example was the deployment of the OpenSearch Domain. This is a service which can be slow to deploy and costly. Each deployment consumed over 20 minutes, adding to our daily operational costs. To address this, we turned to a solution using the AWS Cloud Development Kit (AWS CDK).

import { Domain } from "aws-cdk-lib/aws-opensearchservice";
import { StringParameter } from "aws-cdk-lib/aws-ssm";
import { Construct } from "constructs";
import { buildResourceName, getStage } from "helpers";

export class OpenSearchDomain extends Construct {
  public domainEndpoint: string;
  public domainArn: string;

  constructor(scope: Construct, id: string) {
    super(scope, id);

    if (!isOperationalStage()) {
			// Reuse the pre-existing OpenSearch instance for developer stacks
			this.domainEndpoint = StringParameter.fromStringParameterAttributes(
        this,
        `dev-${OPEN_SEARCH_DOMAIN_ENDPOINT_PARAMETER}`,
        { parameterName: `dev-${OPEN_SEARCH_DOMAIN_ENDPOINT_PARAMETER}` }
      ).stringValue;

      this.domainArn = StringParameter.fromStringParameterAttributes(
        this,
        `dev-${OPEN_SEARCH_DOMAIN_ARN_PARAMETER}`,
        { parameterName: `dev-${OPEN_SEARCH_DOMAIN_ARN_PARAMETER}` }
      ).stringValue;
    } else {
			// Deploy a new OpenSearch domain for key stages (production, staging, etc.)
      const domain = new Domain(this, buildResourceName("domain"), 
			  // Domain configuration details
      });

      this.domainEndpoint = domain.domainEndpoint;
      this.domainArn = domain.domainArn;
      });
    }
  }
}

// Determine if the current stage being deployed is the reusable stage
export const isOperationalStage = (): boolean => {
  const stages = new Set(["dev", "staging", "production"]);
  const deployingStage = getStage();
  return stages.has(deployingStage);
};

In this design, we can still maintain the deployment of other pieces of our stack that might reference the domain. However, we strategically optimize resource allocation by conditionally reusing a pre-existing OpenSearch Domain instance in non-key environments, such as developer-specific stacks. For production, staging, and other critical environments, a new instance of OpenSearch Domain is deployed. This approach significantly reduces the daily deployment times and operational costs. Adopting this conditional deployment method has become a standard practice in our application development, ensuring efficient use of resources and minimizing unnecessary expenditures.