Author: Matt Campbell

HashiCorp has moved the AWS Cloud Control (AWSCC) provider to general availability. The AWSCC provider is automatically generated based on the Cloud Control API published by AWS implying that new AWS features can be supported in Terraform upon their release. Originally released in 2021 as a tech preview, the move to version 1.0 includes several new features including sample configurations and improved schema-level documentation.

AWSCC is built on top of the AWS Cloud Control API. The Cloud Control API provides CRUDL (create, read, update, delete, and list) operations to use with AWS cloud resources. Any resource type published to the CloudFormation Public Registry has a standard JSON schema that can be used with this API.

As part of this release, there are now over 270 resources with sample configurations. For example, awscc_ec2_key_pair allows for specifying a key pair to use with an EC2 instance. An existing key pair can be specified in the PublicKeyMaterial property; omitting that property will generate a new key pair.

resource "awscc_ec2_key_pair" "example" {
  key_name            = "example"
  public_key_material = ""

  tags = [{
    key   = "Modified By"
    value = "AWSCC"
  }]
}

In addition, more than 75 resources now have improved attribute-level documentation. The resources have detailed descriptions of how to use the attributes within the resource-accepted values. This includes context about the attribute, how it’s used, and the expected values for each attribute.

The AWSCC is not meant as a replacement for the standard AWS provider. As noted by Aurora Chun, Product Marketing Manager at HashiCorp, “using the AWSCC and AWS providers together equips developers with a large catalog of resources across established and new AWS services.” The providers can be used in conjunction to provision resources:

# Use the AWS provider to provision an S3 bucket
resource "aws_s3_bucket" "example" {
  bucket_prefix = "example"
}
 
# Use the AWSCC provider to provision an Amazon Personalize dataset
resource "awscc_personalize_dataset" "interactions" {
  ...
 
  dataset_import_job = {
    data_source = {
      data_location = aws_s3_bucket.interactions_import.bucket
    }
  }
}

The AWSCC provider is generated from the latest CloudFormation schemas and releases weekly with all new services added to the Cloud Control API. There are some resources in the CloudFormation schema that are not compatible with the AWSCC provider. A full list of these can be found on GitHub.

Within Azure, the AzAPI Provider enables similar support for the Azure ARM (Azure Resource Management) REST APIs. While there isn’t a Terraform provider available for it, CloudGraph provides a similar API experience to AWS Cloud Control. CloudGraph has support for AWS, Azure, GCP, and Kubernetes.

The Terraform AWS Cloud Control provider is available for download now from the Terraform Registry. The AWSCC provider requires Terraform CLI version 1.0.7 or higher. The source code for the provider is available on GitHub and is licensed under the MPL-2.0 license. Additional information can be found within the provider document and the tutorial.

AWS has announced improved launch times for Windows containers running on AWS Fargate. Launch times were improved by pre-baking the AMIs, leveraging EC2 fast launch, and eliminating the network proxy. The team has also provided recommendations for users to further enhance their container launch times.

AWS states they “have reduced the infrastructure ready time by up to 42% for Windows Server 2022 Core”. They note that Fargate Windows launch times can be divided into three high-level buckets: infrastructure ready time, container image pull time, and task startup time.

Infrastructure ready time encompasses the time needed to provision the underlying compute and progress to pulling the container image. Container image pull time involves containerd pulling and extracting the container image. Finally, task startup time covers the Fargate agent working with containerd to start the task containers with the appropriate configuration.

The AWS team pre-baked more efficient Windows Server AMIs specifically optimized for Fargate. This reduces the set-up required for each individual container thereby reducing potential latency and errors in the deployment process. Mark Nash, Senior Consultant with Contino, describes an image bakery as

A codified, automated and repeatable process where you create custom images used for creating VMs, Containers or Disks. They can help speed up the provisioning time for a VM by baking the required configuration of the systems into the VM image rather than configuring it after the instance is built.

The team also enabled EC2 fast launch for Fargate Windows task launches. EC2 fast launch leverages pre-provisioned snapshots to minimize the need for reboots and complicated initialization steps during the Windows OS launch process.

Behind the scenes, EC2 fast launch creates a set of temporary t3 instances based on the provided settings. Snapshots of those instances are taken once they complete the standard launch steps. The original instances are terminated and the snapshots are used as needed to produce the requested instances. AWS has committed that “sufficient Fast Launch snapshots are available at all times for catering to the demand”.

The team replaced a side-car-based networking proxy with an improved process that runs Fargate worker processes from within the task network namespace. This removes the need for the network proxy entirely. Finally, Fargate is now starting the instance bootstrap immediately after the Windows Server OS boot process is completed versus waiting for the EC2 Launch Agent to begin the process.

The team has some recommendations to further improve the launch performance of Windows Server tasks running on Fargate. To make full use of the Fargate Windows cache, the containers need to be built on a recent Windows Server base image. New base container images are released by Microsoft on the second Tuesday of each month. The Fargate Windows cache only stores the latest and the previous month’s Windows Server core base images.

The team also recommends using Windows Server 2022 Core images as they have a smaller footprint and therefore faster boot times. Part of that footprint reduction is achieved by not providing a GUI or Windows desktop features.

Google Cloud’s closest equivalent service to AWS Fargate is Cloud Run. However, at the time of writing, Cloud Run does not support the Windows operating system for its containers. Azure’s equivalent service is Azure Container Intances which has support for both Linux and Windows containers.

Windows is only available with AWS Fargate when running on Amazon ECS. There is no Windows support available when running Fargate on Amazone EKS. The improved launch times are available in all regions that AWS Fargate is available in.

HashiCorp has released version 1.8 of Terraform, their infrastructure-as-code language. The release introduces provider-defined functions. This enables the creation of custom functions within a given provider that handle computational-style tasks. Several providers, including AWS, GCP, and Kubernetes, have introduced new provider-defined functions alongside this release. Version 1.8 also introduces improvements to refactoring across resource types.

Provider-defined functions can be used in any Terraform expression with the following calling syntax: provider::provider_name::function_name(). These functions can perform several tasks, including data transformation, parsing data, assembling data, and simplifying validations and assertions.

Coinciding with the release of Terraform 1.8, several Terraform providers have been updated to include provider-defined functions. The 5.40 release of the Terraform AWS provider now has provider-defined functions to parse and build ARNs (Amazon Resource Names). For example, arn_parse can be used to retrieve the account ID for a given resource:

# create an ECR repository
resource "aws_ecr_repository" "hashicups" {
  name = "hashicups"
  
  image_scanning_configuration {
    scan_on_push = true
  }
}
 
# output the account ID of the ECR repository
output "hashicups_ecr_repository_account_id" {
  value = provider::aws::arn_parse(aws_ecr_repository.hashicups.arn).account_id
}

Included in the 5.23 release of the Terraform Google Cloud provider is a function to parse regions, zones, names, and projects from resource IDs that are not managed within the Terraform configuration.

resource "google_cloud_run_service_iam_member" "example_run_invoker_jane" {
  member   = "user:jane@example.com"
  role     = "run.invoker"
  service  = provider::google::name_from_id(var.example_cloud_run_service_id)
  location = provider::google::location_from_id(var.example_cloud_run_service_id)
  project  = provider::google::project_from_id(var.example_cloud_run_service_id)
}

The 2.28 release of the Terraform Kubernetes provider includes a provider-defined function for encoding and decoding Kubernetes manifests into Terraform.

resource "kubernetes_manifest" "example" {
  manifest = provider::kubernetes::manifest_decode(file("${path.module}/manifest.yaml"))
}

Version 2.30.0 of the HashiCorp Terraform extension for Visual Studio Code includes syntax highlighting and auto-completion support for provider-defined functions.

OpenTofu, the recent fork of Terraform, has indicated that they will be adding support for provider-defined functions. User janosdebugs posted in the OpenTofu GitHub repo that “provider-implemented functions have been presented to the TSC [Technical Steering Committee] and is being planned for OpenTofu 1.8.”. At the time of writing, OpenTofu is on version 1.6.2.

The release also introduces new functionality to move supported resources between resource types in a faster and less error-prone manner. This enhances the moved block behavior to support moving between resources of different types if the target resource type declares it can be converted from the source resource type. Providers can add this support to handle various use cases such as renaming a provider or splitting a resource.

Terraform 1.8 is available now from GitHub or within Terraform Cloud. More details about the release can be found on the HashiCorp blog, the upgrade guide, and in the changelog.