Implementing a Cloud Foundation in months from scratch – using AWS Landing Zone Accelerator (LZA)

When an organisation is planning to move their workloads from on-premises to the cloud, there are usually a lot of questions around security and cost efficiency. The latter is usually the driver for cloudification, while the former is a requirement for the organisation to even consider moving to the cloud. 

For bigger organisations, one of the first questions that they will face is: How will we manage all of our workloads and monitor the cloud efficiently?

In the past the answer has usually varied depending on who you asked, but almost all solutions contain AWS Organizations and AWS Control Tower. 

These AWS services help you manage your organisation in the cloud, and set up a basic Landing Zone (a well-architected, multi-account AWS environment that is scalable and secure) quickly and with ease, but they do not give you a complete solution. Instead you would need to build another layer of infrastructure on top of it (or order it from your AWS Partner).

AWS has released their own solution to this problem, which as you probably guessed, uses AWS Organizations and AWS Control Tower, called Landing Zone Accelerator (LZA). This open source project was first released nearly two years ago (May 23rd, 2022), and has since then received a lot of updates and improvements. 

Under the hood of LZA is AWS CDK, which in turn uses Cloudformation, but the solution abstracts this layer from the user. Meaning that as a user of LZA, you do not need to know or understand CDK. But as I will touch on later, knowledge of Cloudformation is (most likely) needed. Since the solution uses CDK it does mean that everything is deployed to the cloud as Cloudformation stacks.

What is the Landing Zone Accelerator then? 

In AWS own words:

The Landing Zone Accelerator on AWS solution deploys a foundational set of capabilities that is designed to align with AWS best practices and multiple global compliance frameworks. With this AWS Solution, you can better manage and govern your multi-account environment that has highly-regulated workloads and complex compliance requirements. When used in coordination with other AWS services, it provides a comprehensive, low-code solution across more than 35 AWS services.

Note: This solution will not, by itself, make you compliant. It provides the foundational infrastructure from which additional complementary solutions can be integrated.

What does this mean for you? 

In short this means that it is a solution that provides you with the tools to create a Cloud Foundation for your organisation, using a low-code solution that follows AWS best practices. What low-code means in this context is the users still need to understand how AWS works, but almost all things can be configured in YAML templates. 

I say almost all, because even though the LZA team is constantly pushing new updates and features, you still might have to create your own custom configurations. AWS has thought of this, instead of catering to everyone and every use case, they have added the possibility to add custom Cloudformation templates as part of the solution, which will enable you to have all of your code in one repository and have LZA manage the custom templates.

This blog is not a technical deep dive into LZA. Instead I want to go through the experience working with the solution, and bring up the good things and the things that are good to know / need to be improved. If you’re interested in learning more in-depth how LZA works, these are good links to start with:

• Landing Zone Accelerator on AWS
• Deploy a cloud foundation to support highly-regulated workloads and complex compliance requirements
• AWS labs - Landing zone accelerator on AWS

The good things about LZA

The best thing about LZA is that the heavy lifting involved in creating a Cloud Foundation is taken care of by AWS. If you’re an experienced AWS user, it’s very easy to get started and play around with the solution. While the solution is limited to what you can do, as mentioned earlier, the things it does, it does very well.

Once you’ve set up your AWS Organization and Control Tower, all you need to do is deploy one Cloudformation template in your Management account (or a delegated LZA account), the rest is taken care of by LZA. Voilá, you have your first version of the foundation ready.

If you don’t want to start off with a basic version of LZA, AWS has developed multiple different baselines for different industries. For example if you’re working within the education sector, there are sample configurations you can choose to deploy and it will set up a Cloud Foundation that has a lot more resources deployed. You can find all of these in the Github linked above.

It’s also good to mention that even though LZA is an open source solution it’s still fully supported by AWS. You can open a support ticket with AWS Support (under Control Tower, for some reason), and get help with issues you have with LZA. From my experience, this has been very helpful and if the Support cannot help you directly they will escalate it to the team responsible. You also always have the option to open an issue on Github directly, which is nice.

The “to-be-improved” things about LZA

Despite all the praise I’ve given AWS and LZA, there are still some things that could be improved on. Some things might be improved in the future, some might not.

There are two big things I want to bring up: Documentation and Developer experience.

Documentation

There is a lot of documentation on LZA, on the Github pages you can find Typedocs that will help you out. But in our experience the lack of documentation, or in some cases wrong documentation, was slowing us down a lot. There are currently some resources that you will not know how it works from looking at the documentation without trying them out yourself. 

For example, you cannot change the retention time for Cloudwatch (this will break the pipeline). Hopefully by the time you read this blog, this has been improved.

Developer experience

Secondly, about developer experience. On one hand, once you get the hang of how LZA works, it is very easy to make changes and run the built in pipelines (which are great), but each pipeline run takes about one (1) hour to run to completion. In the early stages of development, this will slow you down, as in many cases you need to deploy resources in a specific order. 

Later on, in the maintenance phase, this is not that big of a deal. On the contrary, all the checks done in the pipeline are a very welcomed feature. LZA does provide you with the possibility to run certain stages of the pipeline separately through the Command Line Interface (CLI), but according to AWS this is not recommended.

A case-study & hands-on experience

To bring this all together, and give some concrete examples of the positives and negatives of LZA, I will go through a case where LZA was used to implement a Cloud Foundation. The customer is Legal Register Centre (Oikeusrekisterikeskus), and they chose to use LZA since they wanted to (among other reasons):

• Develop the Cloud Foundation themselves
• Not rely on third parties

Comment from Mika Mäkelä, the Lead Architect with the Legal Register Centre:

“It was a no-brainer to use AWS supported solution (LZA) as our configuration tool to accomplish our cloud compliance and security goals.”

Learning how to use LZA is both easy and very difficult at the same time. In theory, there are only a handful of configuration files that you need to modify, and everything is in YAML format (except for policies, which is JSON). But in practice, there are many things that can go wrong and you need to learn “how LZA thinks”. Once you get the hang of it, things start to move quicker.

We started the project back in December of 2023, and the first thing was to read up on the basics of LZA. How do you make changes, what things are part of the pipelines etc. When we got started on developing, the first thing we noticed was that development slowed down due to the pipeline running for one hour each change (as mentioned earlier), and the second thing was the lack of documentation. 

To help us out, we had access to AWS Solutions Architects to help us understand how certain things work under the hood and access to AWS Support to help troubleshoot the actual implementation.

During development, here are some of the odd things we found that was not documented, that we used a lot of time on debugging:

• Pipeline might fail due to Lambda concurrency issue, with no clear error code indicating the issue
• Codebuild default Service Quota for concurrent jobs is two (2), and LZA uses three (3) concurrent jobs in the pipeline
• Resources in the security config are only meant to be delegated to the Audit account. We got stuck trying to delegate to a dedicated security account, which meant some manual cleanup and help from AWS support.

One big feature I miss from Cloudformation (and other IaC languages) is a “Depends on” parameter. Instead, you have to know where in the pipeline a certain resource is deployed and run the pipeline multiple times.

 We ran into this issue multiple times, where we tried deploying multiple things at once but the pipelines failed since they were deployed at different stages than we thought. This caused a deadlock a couple of times, which was only resolved by deleting Cloudformation stacks and re-running the pipeline. 

At the time of writing this blog, we are slowly moving to the maintenance stage of the Cloud Foundation. There are still things to do, small tweaks here and there, but all the main features have been implemented. Getting from the start of the project to this stage, took about 5 months. 

This might not sound very quick, but reflecting on the amount of features that have been implemented it is very impressive. If I were to start over again, with the knowledge that I have now, this could easily be implemented in a couple of months.

The choice by the Legal Register Centre to go with LZA was a good one. Despite the drawbacks I have mentioned in this blog, the results speak for themselves. Instead of relying on third parties maintaining their foundation, it is on the shoulders of AWS. As more and more customers start using LZA (hopefully), there is an incentive for AWS to maintain this project and develop it further.