Setting up an AWS Landing Zone in China

A couple of months ago, I received a new task – setting up an AWS landing zone. That’s kind of a routine assignment, isn’t it? Well, there was something different this time. I needed to set up a landing zone in China.

You may have heard that AWS in China is somewhat different to AWS in other regions. And if you haven’t, you can check it yourself: log in to AWS Console and open the regions list. You won’t see any Chinese regions there. Why? Because it’s different.

When I got the task of setting up a landing zone in AWS China, I was referred to this blog post by my Nordcloud colleague Mariusz Preiss: Tutorial: Setting up AWS in China. It’s a very good article but is almost 3 years old now, so some things have changed. So enter this blog post, which gives you the latest.

First things first

You can’t simply open an account in AWS China. You need a legal entity registered in China or must reside in China yourself. And if you want to externally open ports other than 22 (SSH) and 3389 (RDP), you need to have an internet content provider (ICP) licence, commercial or non-commercial, depending on the services you want to provide.

Once you have that in place, the following information will help (and please do refer to Mariusz’s tutorial post for additional detail).

So far, AWS China consists of 2 AWS regions operated by third parties: in Beijing, operated by Sinnet, and in Ningxia, operated by NWCD. Third parties are needed because AWS needs to comply with China’s regulatory and legal requirements, and they couldn’t do it on their own.

AWS China has a different domain: www.amazonaws.cn. This means it has a different AWS Console: https://console.amazonaws.cn/. Yes, that’s how separate it is from the rest of AWS.

AWS China also has its own support: a separate ticketing system and separate personnel. If you ask AWS support a question about AWS China, you’ll get an answer based on the documentation available at https://docs.amazonaws.cn/. I’ve heard stories of people having to communicate with support using Google Translate (because they were getting answers in Chinese), but that wasn’t the case for me: I had a couple of interactions with AWS support in China and got answers in English.

Sometimes (but, not always) AWS China documentation is completely different from the global versions; sometimes it’s been copy-pasted from the global, including unsupported parameters and options mentioned. For example, the Amazon CloudTrail documentation says you can set up an organisation trail. Well, you can’t – an organisation trail isn’t supported. Unfortunately, AWS support may be reading that very same documentation and quoting it in its replies to you (something I experienced). 

Accounts in AWS China are completely separate from accounts in AWS global. This means, for example, that you can’t set up VPC peering or S3 replication. And you can’t have a user in IAM global assume a role in the AWS China region.

But wait, there’s more!

The things mentioned above aren’t the only differences. I’ve noticed a few other things, but before I jump to them, let me make a little disclaimer. 

I was working on setting up a landing zone in AWS China, so my exposure was limited both in time and in terms of the services I was using.

I was working with the following:

• Logical entities structure for billing and permissions
• Basic networking
• Security log management
• Access control
• Compliance rules and dashboards
• Landing zone (accounts, security) automation

So, if I don’t mention anything about those areas, it either means it works the same way as in AWS global or I didn’t go deep enough to notice the difference.

I didn’t check many things:

• Application resources (instances, applications, databases, etc)
• Application monitoring
• Application-specific users and roles
• On-premises connectivity and monitoring

With all that in mind, let’s jump in.

ARNs

The ARNs are slightly different, e.g. it's arn:aws-cn:iam instead of arn:aws:iam. So, if your Cloudformation templates are explicitly using ARNs (for example, arn:aws:iam::aws:policy/job-function/ViewOnlyAccess), you’re in trouble. Luckily, there is a Partition pseudo parameter for CloudFormation. Use it at all times.

amazonaws.com.cn instead of amazonaws.com

In AWS China, endpoints are using URL suffix amazonaws.com.cn, e.g. s3.cn-northwest-1.amazonaws.com.cn. See lists of endpoints at official documentation: for Beijing and Ningxia. Of course, you can get endpoints from SSM like you always do.

Like in the previous case, there is URLSuffix pseudo parameter for CloudFormation.

Now to the best part. Most of the service principals do not need that amazonaws.com.cn URL suffix – they're using the same suffix amazonaws.com as global ones. Always check if the AWS::URLSuffix is needed for your service principal or if you should just hardcode amazonaws.com. Unfortunately, there’s no official list of service principals for China regions (try searching the documentation per service).

Mind the Great Firewall

Wikipedia describes the Great Firewall as the combination of legislative actions and technologies used to regulate the internet domestically in China, and it affects how you work with AWS there.

First of all, the Great Firewall is one of the main reasons why you may want to use AWS China. If you want to be serving users in China, you need to be in the Chinese region. Even if you place your services in the AWS Singapore region, your traffic will need to go through the Firewall, with all effects such as dropped connections, limited bandwidth and so on.

Next, because of the Firewall, when using AWS China from outside China, expect low performance (both in terms of speed and latency) and high packet loss (depending on AWS region, up to 40-50%; lowest in Singapore, ca 15-25%). The usual latency is from 120ms (Singapore) to 200-250 (Ohio/Frankfurt). Bandwidth isn’t great either, so for example, when downloading some files from repositories in the US/EU, you should expect transfers from 10 to 100KB/s. The fastest tested region (at this time) was Singapore, with up to 6MB/s connection speed using S3 copy. The situation is a bit better during China’s night hours, which in Central Europe means after 4 PM.

And, finally, VPN. Having a VPN from China to the outside world is generally not something that the Firewall likes. In AWS China, VPN support is quite limited – you can only set up a Virtual Private Gateway. As an alternative, you can use an open-source VPN software or carrier VPN service. The 3 local carriers are China Mobile, China Unicom and China Telecom, all of which have a VPN licence from the government. VPNs between China and other (global) regions could be used as long as it is for internal company use only, not third parties. Bear in mind that the Great Firewall detects VPN traffic on the gateway quite well (even if only HTTPS/SSL) and blacklists/blocks it if there are violations, so you don't receive any traffic.

Limited support for AWS Organisations

Amazon Organisations are now available in the Chinese regions, but with quite limited features. Policies (including service control policies) aren’t supported. I couldn’t find any information on whether they’ll be supported any time soon.

Also, you can’t enable CloudTrail for all accounts in your organisation, even though the documentation says you can.

AMIs are separate from global ones

You can't copy AMIs from a global to a Chinese AWS Account via API. You can only achieve that by working with disk dumps, which is very time-consuming. Import/export should work as well, but I’ve not tested it myself.

Also, you can't simply export/import images from the Marketplace in Global regions and import them into China. An explanation from AWS says: "The product code for the billing is included in the AMI, so the product would not be billed correctly. The correct path for this kind of request is to contact the marketplace software vendor and ask them the possibility of deploying it in China and how."

And while talking about AMIs, I’d like to also mention that some EC2 instance types aren’t available in China, for example c5a, c6i, etc.

S3 separation

Since AWS China is completely separate from global AWS, you can use the same bucket names again. The AWS documentation says the S3 namespace is global, and it is. But China has its own namespace.

As I’ve mentioned before, no cross-region replication (CRR) is available between global and China regions. But you can reach out to Global S3 buckets from within EC2 in AWS China using global endpoints, and vice-versa. You can reach S3 buckets in China using endpoints as well, but you need to have an internet content provider (ICP) licence to have access – because you need a licence to have web ports open! Of course, you can always reach S3 buckets via API.

Some AWS Config rules are not (yet?) supported

Some examples are:

• DYNAMODB_TABLE_ENCRYPTION_ENABLED
• CLOUDWATCH_LOG_GROUP_ENCRYPTED
• DAX_ENCRYPTION_ENABLED
• ELASTICSEARCH_ENCRYPTED_AT_REST
• ELASTICSEARCH_NODE_TO_NODE_ENCRYPTION_CHECK

You can see the pattern – those are encryption rules. But not all encryption config rules are unavailable; for example, CLOUD_TRAIL_ENCRYPTION_ENABLED is available.

Some other unavailable services

At the time of writing, Simple Email Service (SES) isn’t available. You can ask AWS Technical Account Manager (TAM) about future availability, but it doesn’t look like it’s going to happen any time soon. A third-party email solution is needed, e.g., setting up a server with Sendmail. 

Some services are available only in one of the regions: Cognito is only available in the Beijing region, and Workspaces is only available in the Ningxia region.

To make your life harder, you might see service features in the IAM Policy Generator that aren't available in AWS China.

I’m pretty sure there are other services or service features that aren’t (yet) available in AWS China, but I’ve never tried to check everything. 

A few final observations

So, what did I learn during my interactions with AWS in China? A few things that are very specific to these AWS regions:

• Always check the service you need is (a) there and (b) provides the functionality you need. The services are not 100% the same as in other countries, so always try those services to make sure they have the features you need
• Don’t rely on blog articles and presentations from the internet if they say a service isn’t available or it’s missing functionality. The situation is constantly changing and information gets outdated. For example, now there’s some VPN support
• Check the documentation and ask support, but don’t fully trust it – sometimes it’s been just blindly copy-pasted from the global (I had this experience)

I hope my observations help, and that if you start working with AWS China, you will have as much fun as I had (seriously, it was fun despite the challenges).