使用 Terraform 部署 MWAA

MWAA Terraform 子產品

MWAA Terraform 子產品的下載下傳位址

​​​​​​​git clone https://github.com/aws-ia/terraform-aws-mwaa.git

在示例/基本檔案夾中，有一個簡單的 MWAA 堆棧，可以使用它來測試一切是否正常。

├── README.md
├── dags
│   └── hello_world_dag.py
├── main.tf
├── mwaa
│   └── requirements.txt
├── outputs.tf
├── providers.tf
└── variables.tf
           

MWAA 和 DAGS 檔案夾

variables.tf

此檔案包含更改 MWAA 環境的配置選項 - 環境名稱、AWS 區域和預設标簽。

variable "name" {
  description = "Name of MWAA Environment"
  default     = "terraform-dzone-mwaa"
  type        = string
}

variable "region" {
  description = "region"
  type        = string
  default     = "eu-central-1"
}

variable "tags" {
  description = "Default tags"
  default     = {"env": "dzone", "dept": "AWS Developer Relations"}
  type        = map(string)
}

variable "vpc_cidr" {
  description = "VPC CIDR for MWAA"
  type        = string
  default     = "10.1.0.0/16"
}
           

main.tf

main.tf 包含主要的 Terraform 配置檔案，它将使用 variables.tf 中包含的值部署資源。 

locals {

  azs         = slice(data.aws_availability_zones.available.names, 0, 2)

  bucket_name = format("%s-%s", "aws-ia-mwaa", data.aws_caller_identity.current.account_id)

}

接下來可以建立和上傳示例 DAG 和

requirements.txt

#-----------------------------------------------------------
# Create an S3 bucket and upload sample DAG
#-----------------------------------------------------------
#tfsec:ignore:AWS017 tfsec:ignore:AWS002 tfsec:ignore:AWS077
resource "aws_s3_bucket" "this" {
  bucket = local.bucket_name
  tags   = var.tags
}

resource "aws_s3_bucket_acl" "this" {
  bucket = aws_s3_bucket.this.id
  acl    = "private"
}

resource "aws_s3_bucket_versioning" "this" {
  bucket = aws_s3_bucket.this.id
  versioning_configuration {
    status = "Enabled"
  }
}
resource "aws_s3_bucket_server_side_encryption_configuration" "this" {
  bucket = aws_s3_bucket.this.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

resource "aws_s3_bucket_public_access_block" "this" {
  bucket                  = aws_s3_bucket.this.id
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

# Upload DAGS
resource "aws_s3_object" "object1" {
  for_each = fileset("dags/", "*")
  bucket   = aws_s3_bucket.this.id
  key      = "dags/${each.value}"
  source   = "dags/${each.value}"
  etag     = filemd5("dags/${each.value}")
}

# Upload plugins/requirements.txt
resource "aws_s3_object" "reqs" {
  for_each = fileset("mwaa/", "*")
  bucket   = aws_s3_bucket.this.id
  key      = each.value
  source   = "mwaa/${each.value}"
  etag     = filemd5("mwaa/${each.value}")
}
           

這裡定義了 MWAA 環境的名稱、Apache Airflow 的版本（1.12、2.0.2 或 2.2.2 是 MWAA 服務目前支援的 Apache Airflow 版本），以及MWAA Worker 節點的大小（mw1.small、mw1.medium 或 mw1.large）。然後定義 dags 檔案夾的名稱，Apache Airflow 将使用它作為“dags 檔案夾”來搜尋要運作的 DAG。最後，可以選擇設定 plugins.zip 和 requirements.zip 檔案和位置，但預設情況下不設定這些。