Day 2 at Re:Invent – Builders & Musicians Come Together

When Werner Vogels makes bold statements, expectations are set high. So when Vogel's tweeted 15 minutes before the start of re:Invent's day 2 keynote, we had to wonder what was coming. And how right we were. The close to 3 hours spent in the Venetian hotel in Las Vegas was an experience in itself. Andy Jassy opened the keynote with a long list of customers and partners, alongside the latest business figures. AWS are currently running at an 18 billion run rate with an incredible 42% YoY growth. With millions of active customers - defined as accounts that have used AWS in the last 30 days - the platform is by far the most used on the planet. As per Gartner's 2016 Worldwide Market Segment Share analysis, the company (successfully led by Jassy), has achieved a 44.1% market share in 2016, up from 39% in 2015, more than everyone else combined. This became easily noticeable when AWS displayed an entire catalogue of new services throughout the keynote. The general stance Jassy took this year was that AWS are trying to serve their customers exactly what they asked for in terms of new products. The mission of AWS is nothing short of fixing the IT industry in favour of the end-users and customers. The first on stage was a live 'house' band, performing a segment of 'Everything is Everything' by Lauryn Hill, the chorus rhyming with 'after winter must come spring'. Presumably, AWS was referring to the world of IT still being in a kind of eternal 'winter'. The concept we also heard here was that AWS would not stop building their portfolio and that they want to offer all the tools their 'builders' and customers need. AWS used Jassy's keynote for some big announcements (of course, set to music), with themes across the following areas:

• Compute
• Database
• Data Analytics
• Machine Learning and
• IoT

The Compute Revolution Goes On

Starting in the compute services area, an overview of the vast number of compute instance types and families were shown, with special emphasis given to the Elastic GPU options. There were a few announcements also made on the Tuesday night, including Bare Metal Instances, Streamlined Access to Spot Capacity & Hibernation, making it easier for you to get up to 90% of savings on normal pricing. There was also M5 instances which offer better-priced performance than their predecessors, and H1 instances offering fast and dense storage for Big Data applications. However, with the arrival of Kubernetes in the industry, it was the release of the Elastic Kubernetes that was the most eagerly anticipated. Not only have AWS recognised that their customers wanted Kubernetes on AWS, but they also realise that there's a lot of manual labour involved in maintaining and managing the servers that run ECS & EKS. To solve this particular problem, AWS announced AWS Fargate, a fully managed service for both ECS & EKS meaning no more server management and therefore increasing the ROI in running containers on the platform. This is available for ECS now and will be available for EKS in early 2018. Having started with servers and containers, Jassy then moved on to the next logical evolution of infrastructure services: Serverless. With a 300% usage growth, it's fair to say that if you're not running something on Lambda yet, you will be soon. Jassy reiterated that AWS are building services that integrate with the rest of the AWS platform to ensure that builders don't have to compromise. They want to make progress and get things done fast. Ultimately, this is what AWS compute will mean to the world: faster results. Look out for a dedicated EKS blog post coming soon!

Database Freedom

The next section of the keynote must have had some of AWS's lawyers on the edge of their seats, and also the founder of a certain database vendor... AWS seem to have a clear goal to put an end to the historically painful 'lock-in' some customers experience, referring frequently to 'database freedom'. There's a lot of cool things happening with databases at the moment, and many of the great services and solutions shown at re:Invent are built using AWS database services. Out of all of these, Aurora is by far growing the fastest, and actually is the fastest growing service in the entire history of AWS. People love Aurora because it can scale out for millions of reads per second. It can also autoscale new read replicas and offers seamless recovery from reading replica failures. People want to be able to do this faster, which is why AWS launched a new Aurora features, Auto Multi-Master. This allows for zero application downtime due to any write node failure (previously, AWS suggested this took around 30 seconds), and zero downtime due to an availability zone failure. During 2018 AWS will also introduce the ability to have multi-region masters - this will allow customers to easily scale their applications across regions have a single, consistent data source. Lastly, and certainly not least, was the announcement of Aurora Serverless. which is an on-demand, auto-scaling, Serverless version of Aurora. The users pay by the second - an unbelievably powerful feature for many use cases. Finally, Jassy turned its focus point to DynamoDB service, which scaled to ~12.9 million requests per second at its peak during the last Amazon Prime Day. Just let that sink in for a moment! The DynamoDB service is used by a huge number of major global companies, powering mission-critical workloads of all kinds. The reason for this is, from our perspective, is the fact that it's very easy to access and use as a service. What was announced today was the new feature DynamoDB Global Tables. This enables users to build high performance, globally distributed applications. The final database feature released for DynamoDB was managed back-up & restore, allowing for on-demand backups, point-in-time recovery (in the past 35 days), allowing backups for data archival or regulatory requirements to be taken of hundreds of TB with no interruption. Jassy wrapped up the database section of his keynote by announcing Amazon Neptune, a fully managed graph database which will make it easy to build and run applications that work with highly connected data sets.

Analytics

Next Jassy turned to Analytics, commenting that people want to be using S3 as their data lake. Athena allows for easy querying of structured data within S3, however, most analytics jobs involve processing only a subset of the data stored within S3 objects and Athena requires the whole object to the processed. To ease the pain, AWS released S3 Select - allowing for applications, (including Athena) to retrieve a subset of data from an S3 object using simple SQL expressions - AWS claim drastic performance increases - possibly up to 400% performance. Many of our customers are required by regulation to store logs for up to 7 years and as such ship them to Glacier to reduce the cost of storage. This becomes problematic if you need to query this data though. How great would it be if this could become part of your data lake? Jassy asked, before announcing Glacier Select. Glacier Select allows for queries to be run directly on data stored in Glacier, extending your data lake into Glacier while reducing your storage costs.

Machine Learning

The house band introduced Machine Learning with 'Let it Rain' from Eric Clapton. Dr Matt Woods made an appearance and highlighted how important machine learning is to Amazon itself. The company uses a lot of it, from personal recommendations on Amazon.com to Fulfillment automation & inventory in its warehouses. Jassy highlighted that AWS only invests in building technology that its customers need, (and, remember Amazon.com is a customer!) not because it is cool, or it is funky. Jassy described three tiers of Machine Learning: Frameworks and Interfaces, Platform Services & Application Services. At the Frameworks and Interfaces tier emphasis was placed on the broad range of frameworks that could be used on AWS, recognising that one shoe does not fit every foot and the best results come when using the correct tool for the job. Moving to the Platform Services tier, Jassy highlighted that most companies do not have to expect machine learning practitioners (yet) - it is after all a complex beast. To make this easy for developers, Amazon SageMaker was announced - a fully-managed service that enables data scientists and developers to quickly and easily build, train, and deploy machine learning models at any scale. Also at the platform tier, AWS launched DeepLens, a deep learning enabled wireless video camera designed to help developers grow their machine learning skills. This integrates directly with SageMaker giving developers an end-to-end solution to learn, develop and test machine learning applications. DeepLens will ship in early 2018, available on Amazon.com for $249. The machine learning announcements did not stop there! As Jassy moved into the Application Services tier AWS launched:

• Amazon Rekognition Video - real-time, deep learning powered video analysis,
• Amazon Kinesis Video Streams- Making it easy to securely ingest and store video/audio into AWS so that it can be processed with Rekognition Video
• Amazon Transcribe - Automatic Speech Recognition for audio files stored in S3,
• Amazon Translate - Automatically Translate Text between languages, and,
• Amazon Comprehend - Fully managed natural language processing, that uses machine learning to analyze your text.

IoT Finally, Jassy turned to IoT - identifying five 'frontiers' each with its own release, either available now, or in early 2018:

1. Getting into the game - IoT One Click (in Preview) will make it easy for simple devices to trigger AWS Lambda functions that execute a specific action.
2. Device Management - AWS IoT Device Management will provide fleet management of connected devices, including the onboarding, organisation, monitor and remote management through a devices lifetime.
3. IoT Security - AWS IoT Device Defender (early 2018) will provide security management to your fleet of IoT devices, including auditing to ensure your fleet meets best practice.
4. IoT Analytics - AWS IoT Analytics, making it easy to cleanse, process, enrich, store, and analyze IoT data at scale.
5. Smaller Devices - Amazon FreeRTOS, an operating system for microcontrollers.

Over the next weeks and days, the Nordcloud team will be diving deeper into these new announcements, (including our first thoughts after getting our hands on the new releases) We'll also publish our thoughts and how they can benefit you. It should be noted that, compared to previous years, AWS are announcing more outside the keynotes, in sessions and on their Twitch Channel and so there are many new releases which are not gaining the attention they might deserve. Examples include T2 Unlimited, Inter-Region VPC Peering and Launch Templates for EC2 - as always the best place to keep up-to-date is the AWS 'whats new' page. If you would like to discuss how any of today's announcements could benefit your business, please get in touch.