Notes for Week 5 of 2021

I was mostly poking into AWS with an infrastructure project.

Terraform, AWS and Aurora

Amazon Aurora evolved and now seems to be offering a good trade-off between the comfort and reliability of relational databases with the scalability of distributed clusters. Terraform is my weapon of choice in dealing with AWS. Random learnings and problems:

  • You can choose either having a global database (replicated clusters in multiple regions) or a multi-master cluster (multiple writeable leaders)

    • Multi-master is MySQL only and unsupported for PostgreSQL

  • Amazon boasts sub-second replication between regions, which would be very cool

    • Can be monitored using AuroraGlobalDBReplicationLag

  • The global database have larger instance requirements than its local counterparts; currently, the smallest required is db.r5.large

  • There is a limit on only one secondary region for global databases

    • for MySQL, the alternative is cross-region cluster replication with a limit of five clusters

    • for PostgreSQL, you can set up read replicas, but you have to use bare RDS and not Aurora

  • I ended up using aws_rds_global_cluster over terraform-aws-modules/rds-aurora/aws. Either way, they do use different approach and naming, use either and do not mix & match

  • Global clusters auto-created in all availability zones in the given region, even when not asked to

    • If all AZs are not specified for the cluster in the terraform file, this will cause terraform to try to recreate the cluster on every run

    • To avoid that, you have to explicitly set up subnets for all AZs in the given region and assign them to a region-specific DB subnet. Otherwise, you’ll get random errors

    • You need to do that probably because cluster volumes are always replicated across three AZs

  • Clusters within the same region can have encrypted storage using an automatically-managed encryption key, but that’s not true for multi-region clusters. Those have to have encryption keys managed in KMS and explicitly assigned to the cluster

    • Note you need to decide upfront since encryption of unencrypted Aurora instances is not supported

  • The global database is a bit of a misnomer for PostgreSQL. For MySQL, Aurora supports enabling write forwarding, meaning that all cluster nodes act as writeable for the application. Write forwarding is not supported for PostgreSQL, meaning you have to manage your rite own logic for which connections talk to replicas and which talk to the primary write node. Doing so is close enough to handle sharding manually and means your application has to designed for using clusters upfront

  • Disaster recovery across regions (or manual failover) is a manual process is done by manually creating a new global database from a disconnected cluster, which needs to be coordinated with the application that has to stop writing to the primary. In other words, do not expect smooth failover without an outage

  • That said, in-region failover is done by updating CNAMEs, hence should be smooth

HTTPie gardening

I am joining Jakub in pruning and gardening HTTPie issues. In a good habit, we’re going from the oldest one and discussing what’s valid, what is still blocked in an underlying library and what needs deeper thinking on the rest of the HTTPie. It’s fun.

Readings from this week

Published in Weekly Notes and tagged • •

All texts written by . I'd love to hear your feedback. If you've liked this, you may want to subscribe for my monthly newsletter, RSS , or Mastodon. You can always return to home page or read about the site and its privacy handling.