Open source news and updates #133

{"value":"## **October 28th, 2022 - Instalment #133**\n\n### **Welcome**\n\n\nWelcome to the Amazon Web Services open source newsletter, edition #133. This week has been all about KubeCon, and there were some important announcements from Amazon Web Services. If you missed these, I have tried to capture the important ones here, but I want to start off with probably my favourite which was the announcement during Nate Taber keynote on some of the investments we are providing to CNCF and OpenSSF. Amazon Web Services is providing \$3 million in credits to CNCF, as well as providing \$10 million over 3 years to the Open SSF and providing fixes and test suites to open source projects.\n\n\nThis week we feature the following new open source projects, \"Amazon Web Services-logs-comptroller\" keep on top of your Amazon Web Services CloudWatch log groups, \"cloudcanvas\" a Mac based app for interacting with some of your Amazon Web Services resources, \"atlearn\" a tool to help you with transfer learning, \"Amazon Web Services-code-habits\" bootstrap your dev setup with some good defaults, \"cargo-check-external-types\" a tool for Rust developers, \"OQpy\", \"spin-wasm-multi-compute\" a science project on wasm, and plenty more.\n\nWe also cover open source topics including MySQL, MariaDB, cdk8s, Apache Kafka, Apache CloudStack, Red Hat Enterprise Linux, OpenFold, Composer, Flask, OQpy, Kotlin, Amazon Web Services CDK, eventcatalog, traefik, Redis, CNCF, OpenSSF, Ploomber, Soopervisor, and more! Before you go, make sure you check out this weeks videos and we have the usual round up of open source events that are coming up.\n\n\n### **Feedback**\n\nPlease let me know how we can improve this newsletter as well as how Amazon Web Services can better work with open source projects and technologies by completing ++[this very short survey](https://eventbox.dev/survey/NUSZ91Z)++ that will take you probably less than 30 seconds to complete. Thank you so much!\n\n\n### **Celebrating open source contributors**\n\nThe articles and projects shared in this newsletter are only possible thanks to the many contributors in open source. I would like to shout out and thank those folks who really do power open source and enable us all to learn and build on top of what they have created.\n\nSo thank you to the following open source heroes: Sai Vennam, Toshal Dudhwala, Imaya Kumar Jagannathan, Massimo Re Ferrè, Matt Morgan, Shimon Tolts, Philip Reinhold, Li Chen, Jean-Christophe Jaskula, Peter Karalekas, Stephanie Teo, Bandish Shah, Olivier Cruchant, Joseph Keating, Matt Hedges, Steve Roberts, Angel Pizarro, Jason Rupard, Mariana Ramos Franco, Ran Isenberg, Matt Martz, Abhishek Gupta, John Mille, and Elena van Engelen-Maslova.\n\n### **Latest open source projects**\n\n*The great thing about open source projects is that you can review the source code. If you like the look of these projects, make sure you that take a look at the code, and if it is useful to you, get in touch with the maintainer to provide feedback, suggestions or even submit a contribution.*\n\n### Tools\n\n### Amazon Web Services-logs-comptroller\n\n++[Amazon Web Services-logs-comptroller](https://aws-oss.beachgeek.co.uk/275)++ is a neat project from Matt Morgan that I found via Corey's ++[newsletter](https://aws-oss.beachgeek.co.uk/276)++ that allows you to keep on top of your Amazon Web Services CloudWatch log groups. Amazon Web Services Logs Comptroller is a CDK Construct that can be used to set CloudWatch log retention as well as prune orphaned LogGroups. Amazon Web Services Logs Comptroller is written entirely using Step Functions and ASL. As I often have to manually do this for the demos I run, this is going to come in super handy.\n\n### **cloudcanvas**\n\n++[cloudcanvas](https://aws-oss.beachgeek.co.uk/27d)++ is an interesting project to keep your eyes on that provides an interactive canvas to help you manage your cloud resources. It is a downloadable app for your Mac (sorry, Mac only from the looks of it at the moment) that allows you to have a more interactive experience when working with Amazon Web Services. Currently as this is in early stages, it offers a limited set of Amazon Web Services services. I have downloaded and installed it, but just need to sort out my test SSO account before I can get stuck in. If I get time, I might show this on the next episode of Build on Open Source.\n\n### **atlearn**\n\n++[atlearn](https://aws-oss.beachgeek.co.uk/27e)++ is a Transfer Learning toolkit that supports easy model building on top of the pre-trained models. It provides a set of APIs that enable users to load pre-trained models, build and train their customised transfer learning models, and export the models for inference and deployment. Read the documentation to find out more about why this tool was created and some of the benefits, and then check out the demos provided.\n\n\n\n### **Amazon Web Services-code-habits**\n\n++[Amazon Web Services-code-habits](https://aws-oss.beachgeek.co.uk/27f)++ is a library with Make targets, Ansible playbooks, Jinja templates (and more) designed to boost common software development tasks and enhance governance. If you are looking for a way to automate and bootstrap your new projects, why not check this out.\n\nCheck out the repo for a demo screenshot of this running.\n\n### **cargo-check-external-types**\n\n++[cargo-check-external-types](https://aws-oss.beachgeek.co.uk/27g)++ is a static analysis tool for Rust library authors to set and verify which types from other libraries are allowed to be are exposed in their public API. This is useful for ensuring that a breaking change to a dependency doesn't force a breaking change in the library that's using it.\n\nThe tool has two output formats to cover different use-cases:\n\n\n- errors (the default): Output error messages for each type that is exposed in the public API and exit with status 1 if there is at least one error. This is useful for continuous integration.\n- markdown-table: Output the places types are exposed as a Markdown table. This is intended as a discovery tool for established projects.\n\nMake sure you check out the docs for more details on this handy tool, as well as some important prereq's that you need before you can use this too.\n\n### **OQpy**\n\n++[OQpy](https://aws-oss.beachgeek.co.uk/26s)++ The goal of oqpy (\"ock-pie\") is to make it easy to generate OpenQASM 3 + OpenPulse in Python. The oqpy library builds off of the openqasm3 and openpulse packages, which serve as Python reference implementations of the abstract syntax tree (AST) for the OpenQASM 3 and OpenPulse grammars. What are OpenQASM 3 and OpenPulse? Well I had the same questions, and thankfully these are answered in the project readme. We also have this post, ++[Amazon Web Services open-sources OQpy to make it easier to write quantum programs in OpenQASM 3](https://aws-oss.beachgeek.co.uk/26t)++ where Philip Reinhold, Li Chen, Jean-Christophe Jaskula, Peter Karalekas, and Stephanie Teo collaborate to share an example of what is possible using OQpy and OpenQASM 3.\n\n\n\n## **Demos, Samples, Solutions and Workshops**\n\n### **spin-wasm-multi-compute**\n\n++[spin-wasm-multi-compute](https://aws-oss.beachgeek.co.uk/277)++ is a new \"science project\" from all round good egg Massimo Re Ferrè where he provides some code samples as he explores the wonderful world of wasm. This demo repo is a rough proof of concept for running the same WASM artefact on Amazon EC2, [Amazon ECS](https://aws.amazon.com/cn/ecs/?trk=cndc-detail)/Fargate and Amazon Web Services Lambda. I also love how Massimo has used another open source project (cdk-dia) to visualise the resulting stack that is deployed via Amazon Web Services CDK.\n\n\n\n### **detect-and-handling-custom-pii-with-Amazon Web Services-glue-studio-and-Amazon Web Services-glue-databrew**\n\n++[detect-and-handling-custom-pii-with-Amazon Web Services-glue-studio-and-Amazon Web Services-glue-databrew](https://aws-oss.beachgeek.co.uk/278)++ aside from winning this weeks open source repo with the longest name, provides an Amazon Web Services CloudFormation (CFn) template that builds sample data pipeline, with the functionality of identifying personal and sensitive data using the Sensitive Data Detection feature of Amazon Web Services Glue Studio and apply hashing algorithm to protect columns identified with the use of Amazon Web Services Glue DataBrew, through an event-driven and serverless architecture.\n\n\n\n### **apprunner-hotel-app**\n\n++[apprunner-hotel-app](https://aws-oss.beachgeek.co.uk/279)++ this project provisions the base layer infrastructure to demonstrate how AppRunner leverages a VPC Connector to interact with a DB in a private subnet.\n\n### **Amazon Web Services-dynamodb-parallel-queries**\n\n++[Amazon Web Services-dynamodb-parallel-queries](https://aws-oss.beachgeek.co.uk/27a)++ this repo provides a sample application and queries for you to see how these work and then configure a load generator which you can tune, to see how they perform. Code is in Java.\n\n\n### **selective-search-with-mutual-information-cotraining**\n\n++[selective-search-with-mutual-information-cotraining](https://aws-oss.beachgeek.co.uk/27b)++ this repo provide code samples for the research paper, ++[MICO: Selective Search with Mutual Information Co-training](https://aws-oss.beachgeek.co.uk/27c)++. What is this research paper on? From the paper:\n\nIn contrast to traditional exhaustive search, selective search first clusters documents into several groups before all the documents are searched exhaustively by a query, to limit the search executed within one group or only a few groups. Selective search is designed to reduce the latency and computation in modern large-scale search systems.\n\n\n## **Amazon Web Services and Community blog posts**\n\n### **Amazon Web Services CDK**\n\nAmazon Web Services Community Builder Ran Isenberg has put together this post, ++[Deploy to Amazon Web Services with GitHub Actions and Amazon Web Services CDK](https://aws-oss.beachgeek.co.uk/273)++ that provides a hands on guide to show you how you can use Amazon Web Services CDK to deploy your CDK stacks via GitHub Actions. Need to give this one a try at some point. [hands on]\n\n### **eventcatalog**\n\nAmazon Web Services Serverless Developer Advocate all round open source good guy, David Boyne built ++[EventCatalog](https://aws-oss.beachgeek.co.uk/18d)++ as a POC and experiment to see if it could help document event driven architectures. Earlier this week he shared a post on how some of the community have been using it and putting together \"epic guides\" (we need way more epic guides in my opinion). In ++[Using Amazon Web Services CDK to Deploy EventCatalog](https://aws-oss.beachgeek.co.uk/274)++ Matt Martz shows you how you can deploy EventCatalog to Amazon Web Services using Amazon Web Services CDK.\n\n\n\n### **Redis**\n\nFellow Developer Advocate Abhishek Gupta has put together something for Redis fans in his post, ++[Using Redis on Cloud? Here are ten things you should know](https://aws-oss.beachgeek.co.uk/272)++ where he covers a range of Redis-related best practices, tips and tricks including cluster scalability, client side configuration, integration, and metrics.\n\n\n\n### **traefik**\n\nTraefik is a modern HTTP reverse proxy and load balancer that makes deploying micro services easy. Amazon Web Services Community Builder John Mille has put together this post, ++[Traefik Proxy in Amazon Web Services with Amazon Web Services ECS - Part 2](https://aws-oss.beachgeek.co.uk/271)++, showing you how you can use this in conjunction with Amazon Web Services ECS Anywhere to manage your ECS traffic where ever it is.\n\n\n\n### **Kotlin**\n\nElena van Engelen-Maslova dives deep into understanding the performance characteristics of Kotlin on Amazon Web Services Lambda in her post, ++[This Week in Kotlin on Amazon Web Services Lambda: To ARM64 or Not To ARM64?](https://aws-oss.beachgeek.co.uk/270)++. Exploring JVM and GraalVM native runtimes and how they compare are just some of the insights that await you dear reader.\n\n\n\n### **Ploomber and Soopervisor**\n\nPloomber and Soopervisor and two open source tools that you can use to build data pipelines in Python (Ploomber) and export those data pipelines to a number of outputs (Argo, Apache Airflow, Amazon Web Services Batch and others). In the second part of a three part blog post, ++[Deploying a Data Science Platform on Amazon Web Services: Running containerized experiments (Part II)](https://aws-oss.beachgeek.co.uk/26z)++ Eduardo Blancas provides a detailed walk through of how you can use these on Amazon Web Services. Watch out for part three coming soon.\n\n\n### **Flask**\n\nIn the post, ++[Automate Python Flask Deployment to the Amazon Web Services Cloud](https://aws-oss.beachgeek.co.uk/26u)++, Joseph Keating and Matt Hedges demonstrate how to use Amazon Web Services services and open source tools to automate the deployment of a Python Flask application to the Amazon Web Services Cloud.\n\n\n\n### **Composer**\n\n++[Composer](https://aws-oss.beachgeek.co.uk/26r)++ is a PyTorch library that enables you to train neural networks faster, at lower cost, and to higher accuracy. In the post, ++[Reduce deep learning training time and cost with MosaicML Composer on Amazon Web Services](https://aws-oss.beachgeek.co.uk/26q)++ Bandish Shah and Olivier Cruchant highlight challenges commonly reported specifically in Deep Learning (DL) training, and how the open-source library MosaicML Composer helps solve them.\n\n\n\n### **Other posts and quick reads**\n\n- ++[Implementing Pod Security Standards in Amazon EKS](https://aws-oss.beachgeek.co.uk/26n)++ provides a hands on guide on how to apply various Pod security standards in [Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail) [hands on]\n\n\n\n- ++[Run inference at scale for OpenFold, a PyTorch-based protein folding ML model, using Amazon EKS](https://aws-oss.beachgeek.co.uk/26o)++ shows how you can deploy OpenFold models on [Amazon Elastic Kubernetes Service](https://aws.amazon.com/cn/eks/?trk=cndc-detail) ([Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail)) and how to scale the EKS clusters to drastically reduce MSA computation and protein structure inference times [hands on]\n\n\n\n- ++[Build real-time multi-user experiences using GraphQL on Amazon Web Services Amplify](https://aws-oss.beachgeek.co.uk/26p)++ is a tutorial where you will build a share-able todo list app using GraphQL and Amazon Web Services Amplify [hands on]\n\n\n### **Kubernetes**\n\nAs KubeCon NA wraps up, there were a few announcements (covered below in the Quick Updates section) as well as some new blog posts that dive deeper into these.\n\nSteve Roberts took a look at ++[Amazon Web Services Batch for Amazon Elastic Kubernetes Service](https://aws.amazon.com/blogs/aws/aws-batch-for-amazon-elastic-kubernetes-service/)++ introducing what it is, and how it works.\n\n\n\nWe also have this post, ++[How Amazon Web Services Batch developed support for Amazon Elastic Kubernetes Service](https://aws-oss.beachgeek.co.uk/26w)++\nwhere Angel Pizarro and Jason Rupard look behind the scenes and talk about the initial motivation and design choices when developing the service, as well as some of the challenges that the team overcame.\n\nWhilst not related to KubeCon, Mariana Ramos Franco put together this post, ++[Speed up Highly Available Deployments on Kubernetes](https://dev.to/aws/!%5Bhttps://aws-oss.beachgeek.co.uk/26x%5D(https://aws-oss.beachgeek.co.uk/26x))++. The [Amazon Managed Service for Prometheus](https://aws.amazon.com/cn/prometheus/?trk=cndc-detail) provides a Prometheus-compatible monitoring service that is operated on Kubernetes. In the post, she dives deep into the internals of our Prometheus service and how we speed up our deployment times without impacting availability.\n\n\n\n### **Quick updates**\n\n### **Apache Kafka**\n\n[Amazon Managed Streaming for Apache Kafka](https://aws.amazon.com/cn/msk/?trk=cndc-detail) (Amazon MSK) now supports Apache Kafka version 3.3.1 for new and existing clusters. Apache Kafka 3.3.1 includes several bug fixes and new features that improve performance. Some of the key features include enhancements to metrics and partitioner. Amazon MSK will continue to use and manage Zookeeper for quorum management in this release for stability. For a complete list of improvements and bug fixes, see the Apache Kafka release notes for 3.3.1.\n\n\n### **cdk8s**\n\nCDK For Kubernetes Plus (CDK8s+) is a multi-language class library for defining Kubernetes applications using high level intent based constructs. We featured a great overview of this in the ++[last episode of Build on Open Source](https://www.twitch.tv/videos/1630135246)++ if you want a primer.\n\nCDK8s+ aims to lower the entry barrier and improve maintainability of Kubernetes manifests by offering a hand crafted construct for each core Kubernetes object, exposing a richer API with reduced complexity. With this launch, CDK8s+ is now generally available and stable for use. This means that the API will remain unchanged and fully supported (no breaking changes), at least until the next major version. CDK8s+ is vended as a separate library for each Kubernetes spec version, all those libraries are now generally available and stable to use.\n\nCustomers also want to validate their manifest by applying either community or organisational policies. CDK8s now supports integration with third-party tools that facilitate this, and can perform validation as part of the synthesis process. This supports manifests produced by CDK8s adhering to the necessary policies.\n\nIf you want to read more about this, check out this post, ++[Announcing general availability of cdk8s+ and support for manifest validation](https://aws-oss.beachgeek.co.uk/26m)++ from Shimon Tolts, CEO and Co-Founder of Datree. The post shows you a few examples of how cdk8s+ can simplify the authoring of Kubernetes manifests, and how cdk8s+ can integrate with third-party policy enforcement tools (such as Datree), to provide a guardrail that protects your manifests against misconfigurations, and prevent them from reaching your cluster.\n\n### **MySQL, MariaDB, and PostgreSQL**\n\nAmazon Relational Database Service (RDS) for MySQL, MariaDB and PostgreSQL now support 15 read replicas per instance, including up to 5 cross region read replicas, delivering up to 3X the previous read capacity.\n\n[Amazon RDS](https://aws.amazon.com/cn/rds/?trk=cndc-detail) Read Replicas provide enhanced performance and durability for [Amazon RDS](https://aws.amazon.com/cn/rds/?trk=cndc-detail) database (DB) instances. They make it easy to elastically scale out beyond the capacity constraints of a single DB instance for read-heavy database workloads. You can create one or more replicas of a given source DB instance and serve high-volume application read traffic from multiple copies of your data, thereby increasing aggregate read throughput. For disaster recovery, read replicas can be promoted when needed to become standalone DB instances.\n\n### **MySQL**\n\nAnnounced this week was up to 10x faster exports to [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail) for snapshot exports for [Amazon Aurora](https://aws.amazon.com/cn/rds/aurora/?trk=cndc-detail) MySQL-Compatible Edition for MySQL 5.7 and 8.0. The performance improvement is automatically applied to all types of database snapshot exports, including manual snapshots, automated system snapshots, and snapshots created by the Amazon Web Services Backup service.\n\nWith this capability, customers can export data up to 10x faster from an [Amazon Aurora](https://aws.amazon.com/cn/rds/aurora/?trk=cndc-detail) database and store it in an [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail) bucket using Amazon Web Services CLI or Amazon Web Services Management Console. After the data is exported, they can analyze the exported data using [Amazon Athena](https: //aws.amazon.com/cn/athena/?trk=cndc-detail) and/or other tools and services. The export process runs in the background and doesn't affect the performance of active database instances.\n\n\n### **Apache CloudStack**\n\nApache CloudStack, a project of the Apache Software Foundation, is open source software designed to deploy and manage large networks of virtual machines. [Amazon Elastic Kubernetes Service](https://aws.amazon.com/cn/eks/?trk=cndc-detail) ([Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail)) Anywhere on Apache CloudStack was announced last week, and this expands the choice of infrastructure options for customers running Kubernetes on-premises. Apache CloudStack enhances the list of deployment options for [Amazon EKS Anywhere](https://aws.amazon.com/cn/eks/eks-anywhere/?trk=cndc-detail) customers, which already includes bare metal servers and VMware vSphere.\n\nWhile developing this new feature, we collaborated with the Apache CloudStack community to build the Cluster API provider for Apache CloudStack (CAPC), which has now joined the Kubernetes SIG. [Amazon EKS Anywhere](https://aws.amazon.com/cn/eks/eks-anywhere/?trk=cndc-detail) leverages CAPC to provide customers declarative, Kubernetes-style APIs for cluster creation, configuration, and management.\n\n### **Red Hat Enterprise Linux**\n\nIn addition to Bottlerocket and Ubuntu, you can now use Red Hat Enterprise Linux (RHEL) and have broader choice of operating systems to create and operate [Amazon EKS Anywhere](https://aws.amazon.com/cn/eks/eks-anywhere/?trk=cndc-detail) clusters in your on-premises data centres. RHEL support is available for [Amazon EKS Anywhere](https://aws.amazon.com/cn/eks/eks-anywhere/?trk=cndc-detail) clusters running on VMware vSphere, on Apache CloudStack, or directly on bare metal servers.\n\nRHEL is a popular operating system for on-premises workloads, and customers often customize their RHEL images with security hardening and other organisational best practices. Now, you can start from your custom base images, and use [Amazon EKS Anywhere](https://aws.amazon.com/cn/eks/eks-anywhere/?trk=cndc-detail) tooling to build RHEL images for running Kubernetes clusters.\n\n## **Videos of the week**\n\n### **Kubernetes**\n\nCloud Native Rejekts is the b-side conference giving a second chance to the many wonderful, but rejected, talks leading into KubeCon + CloudNativeCon. You will need to fast forward to 6:52:28 to hear Mariana Ramos Franco do a talk of her blog post that I have highlight above.\n\n\n### **Crossplane**\n\nWhat is Crossplane? In this video, Sai Vennam from Containers from the couch shows you how Crossplane enables platform teams to manage their infrastructure and managed services on cloud providers like Amazon Web Services. He outlines how you can use Crossplane to deploy and manage a production-ready [Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail) cluster.\n\n\n<video src=\\"https://dev-media.amazoncloud.cn/1e4d2b0819394dbe9a438ea70db94c78_Crossplane%20on%20Kubernetes%20Explained.mp4\\" class=\\"manvaVedio\\" controls=\\"controls\\" style=\\"width:160px;height:160px\\"></video>\n\n### **Observability**\n\nJoin Toshal Dudhwala and Imaya Kumar Jagannathan as they share how you can leverage Amazon Web Services managed open-source observability solutions to monitor your containerised or serverless workloads at scale.\n\n<video src=\\"https://dev-media.amazoncloud.cn/f3488cb05e8240fd8839a6226a483af6_Accelerating%20Adoption%20of%20AWS%20Open-Source%20Observability%20Services%20-%20AWS%20Online%20Tech%20Talks.mp4\\" class=\\"manvaVedio\\" controls=\\"controls\\" style=\\"width:160px;height:160px\\"></video>\n\n\n### **Build on Open Source**\n\nFor those unfamiliar with this show, Build on Open Source is where we go over this newsletter and then invite special guests to dive deep into their open source project. Expect plenty of code, demos and hopefully laughs.\n\nWe have put together a playlist so that you can easily access all the other episodes of the Build on Open Source show. ++[Build on Open Source playlist](https://aws-oss.beachgeek.co.uk/24u)++\n\n## **Events for your diary**\n\n### **All Things Open**\n### **October 30th - 2nd November, Raleigh, NC**\n\nI am excited about attending and speaking at All Things Open next week (check out my session, ++[Automating Data Pipelines with Apache Airflow](https://aws-oss.beachgeek.co.uk/26y)++) Matt Auerbach has put together this post, ++[Amazon Web Services at ATO 2022](https://2022.allthingsopen.org/aws-at-ato-2022/)++ and there will be a bunch of Amazon Web Services open source folk at the Amazon Web Services booth so hope to see some of you. I will have some very nice Amazon Web Services Build On Open Source challenge coins, so come find me and share a story or swap some equally cool swag to bag one of these exclusive items.\n\n### **Build on Amazon Web Services Open Source**\n### **November 4th, 9am BST**\n\nJoin us for the sixth episode of the Build on Amazon Web Services series, featuring a live round up of the latest projects and news as well as a special guest speaker. We have another special guest lined up, and I am very excited to welcome Javier Ramirez from QuestDB who will be coming on the show and sharing more about this open source times series database that is blazingly fast. Follow the show on @buildonopen for more details. Check it out on ++[https://twitch.tv/aws](https://twitch.tv/aws)++\n\n\n### **Amazon Web Services Elastic Kubernetes Service (EKS) Workshop**\n### **November 10th, London 5pm**\n\nJoin us for an interactive workshop on containers, Docker, Fargate and [Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail), hosted by ClearScale and Amazon Web Services. This live, virtual workshop includes three hours of interactive presentation and hands-on lab work. You will take part in the setup and deployment of containers using EKS. Follow along and work directly with Amazon Web Services professionals and ClearScale (an Amazon Web Services Premier Tier Services Partner) in this Level 200 training session.\n\nYou can find out more about this event by ++[checking out the event page and signing up.](https://aws-oss.beachgeek.co.uk/22y)++\n\n### **re:Invent**\n### **November 28th - December 3rd, Las Vegas**\n\nre:Invent is only a few weeks away so I want to share a few things that will hopefully be of interest.\n\nFirst up, we will be running the Build On Live stream throughout re:Invent and we would love to feature you! If either yourself, or perhaps you know a community member going to re:Invent and think they will absolutely love to attend the livestream, we want to hear from you. Please nominate a community member you want to hear from during Build On Live ++[using this survey](https://eventbox.dev/survey/6B0ED1J)++.\n\nSecond, check out this handy way to look at all the amazing open source sessions, then check out this ++[dashboard](https://aws-oss.beachgeek.co.uk/252)++ [sign up required]. I would love to hear which ones you are excited about so please let me know in the comments or via Twitter. If you want to hear what my top three, must watch sessions, then this is what I would attend (sadly, as an Amazon Web Services employee I am not allowed to attend sessions)\n\n\n\n1. OPN306 Amazon Web Services Lambda Powertools: Lessons from the road to 10 million downloads - Heitor Lessa is going to deliver an amazing session on the journey from idea to one of the most loved and used open source tools for Amazon Web Services Lambda users\n2. BOA204 When security, safety, and urgency all matter: Handling Log4Shell - Cannot wait for this session from Abbey Fuller who will walk us through how we managed this incident\n3. OPN202 Maintaining the Amazon Web Services Amplify Framework in the open - Matt Auerbach and Ashish Nanda are going to share details on how Amplify engineering managers work with the OSS community to build open-source software\n\nThere are many other great open source sessions, and hopefully I will try and put together a more comprehensive list as approach re:Invent.\n\n### **OpenSearch**\n### **Every other Tuesday, 3pm GMT**\n\nThis regular meet-up is for anyone interested in OpenSearch & Open Distro. All skill levels are welcome and they cover and welcome talks on topics including: search, logging, log analytics, and data visualisation.\n\nSign up to the next session, ++[OpenSearch Community Meeting](https://aws-oss.beachgeek.co.uk/1az)++\n\n## **Stay in touch with open source at Amazon Web Services**\n\nI hope this summary has been useful. Remember to check out the ++[Open Source homepage](https://aws.amazon.com/opensource/?opensource-all.sort-by=item.additionalFields.startDate&opensource-all.sort-order=asc)++ to keep up to date with all our activity in open source by following us on ++[@AWSOpen](https://twitter.com/AWSOpen)++\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","render":"<h2><a id=\\"October_28th_2022__Instalment_133_0\\"></a><strong>October 28th, 2022 - Instalment #133</strong></h2>\\n<h3><a id=\\"Welcome_2\\"></a><strong>Welcome</strong></h3>\\n<p>Welcome to the Amazon Web Services open source newsletter, edition #133. This week has been all about KubeCon, and there were some important announcements from Amazon Web Services. If you missed these, I have tried to capture the important ones here, but I want to start off with probably my favourite which was the announcement during Nate Taber keynote on some of the investments we are providing to CNCF and OpenSSF. Amazon Web Services is providing \$3 million in credits to CNCF, as well as providing \$10 million over 3 years to the Open SSF and providing fixes and test suites to open source projects.</p>\n<p>This week we feature the following new open source projects, “Amazon Web Services-logs-comptroller” keep on top of your Amazon Web Services CloudWatch log groups, “cloudcanvas” a Mac based app for interacting with some of your Amazon Web Services resources, “atlearn” a tool to help you with transfer learning, “Amazon Web Services-code-habits” bootstrap your dev setup with some good defaults, “cargo-check-external-types” a tool for Rust developers, “OQpy”, “spin-wasm-multi-compute” a science project on wasm, and plenty more.</p>\n<p>We also cover open source topics including MySQL, MariaDB, cdk8s, Apache Kafka, Apache CloudStack, Red Hat Enterprise Linux, OpenFold, Composer, Flask, OQpy, Kotlin, Amazon Web Services CDK, eventcatalog, traefik, Redis, CNCF, OpenSSF, Ploomber, Soopervisor, and more! Before you go, make sure you check out this weeks videos and we have the usual round up of open source events that are coming up.</p>\n<h3><a id=\\"Feedback_13\\"></a><strong>Feedback</strong></h3>\\n<p>Please let me know how we can improve this newsletter as well as how Amazon Web Services can better work with open source projects and technologies by completing <ins><a href=\\"https://eventbox.dev/survey/NUSZ91Z\\" target=\\"_blank\\">this very short survey</a></ins> that will take you probably less than 30 seconds to complete. Thank you so much!</p>\n<h3><a id=\\"Celebrating_open_source_contributors_18\\"></a><strong>Celebrating open source contributors</strong></h3>\\n<p>The articles and projects shared in this newsletter are only possible thanks to the many contributors in open source. I would like to shout out and thank those folks who really do power open source and enable us all to learn and build on top of what they have created.</p>\n<p>So thank you to the following open source heroes: Sai Vennam, Toshal Dudhwala, Imaya Kumar Jagannathan, Massimo Re Ferrè, Matt Morgan, Shimon Tolts, Philip Reinhold, Li Chen, Jean-Christophe Jaskula, Peter Karalekas, Stephanie Teo, Bandish Shah, Olivier Cruchant, Joseph Keating, Matt Hedges, Steve Roberts, Angel Pizarro, Jason Rupard, Mariana Ramos Franco, Ran Isenberg, Matt Martz, Abhishek Gupta, John Mille, and Elena van Engelen-Maslova.</p>\n<h3><a id=\\"Latest_open_source_projects_24\\"></a><strong>Latest open source projects</strong></h3>\\n<p><em>The great thing about open source projects is that you can review the source code. If you like the look of these projects, make sure you that take a look at the code, and if it is useful to you, get in touch with the maintainer to provide feedback, suggestions or even submit a contribution.</em></p>\\n<h3><a id=\\"Tools_28\\"></a>Tools</h3>\\n<h3><a id=\\"Amazon_Web_Serviceslogscomptroller_30\\"></a>Amazon Web Services-logs-comptroller</h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/275\\" target=\\"_blank\\">Amazon Web Services-logs-comptroller</a></ins> is a neat project from Matt Morgan that I found via Corey’s <ins><a href=\\"https://aws-oss.beachgeek.co.uk/276\\" target=\\"_blank\\">newsletter</a></ins> that allows you to keep on top of your Amazon Web Services CloudWatch log groups. Amazon Web Services Logs Comptroller is a CDK Construct that can be used to set CloudWatch log retention as well as prune orphaned LogGroups. Amazon Web Services Logs Comptroller is written entirely using Step Functions and ASL. As I often have to manually do this for the demos I run, this is going to come in super handy.</p>\n<h3><a id=\\"cloudcanvas_34\\"></a><strong>cloudcanvas</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/27d\\" target=\\"_blank\\">cloudcanvas</a></ins> is an interesting project to keep your eyes on that provides an interactive canvas to help you manage your cloud resources. It is a downloadable app for your Mac (sorry, Mac only from the looks of it at the moment) that allows you to have a more interactive experience when working with Amazon Web Services. Currently as this is in early stages, it offers a limited set of Amazon Web Services services. I have downloaded and installed it, but just need to sort out my test SSO account before I can get stuck in. If I get time, I might show this on the next episode of Build on Open Source.</p>\n<h3><a id=\\"atlearn_38\\"></a><strong>atlearn</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/27e\\" target=\\"_blank\\">atlearn</a></ins> is a Transfer Learning toolkit that supports easy model building on top of the pre-trained models. It provides a set of APIs that enable users to load pre-trained models, build and train their customised transfer learning models, and export the models for inference and deployment. Read the documentation to find out more about why this tool was created and some of the benefits, and then check out the demos provided.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/0712f05028994b8e80e285ceb737bed1_example.jpg\\" alt=\\"example.jpg\\" /></p>\n<h3><a id=\\"Amazon_Web_Servicescodehabits_44\\"></a><strong>Amazon Web Services-code-habits</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/27f\\" target=\\"_blank\\">Amazon Web Services-code-habits</a></ins> is a library with Make targets, Ansible playbooks, Jinja templates (and more) designed to boost common software development tasks and enhance governance. If you are looking for a way to automate and bootstrap your new projects, why not check this out.</p>\n<p>Check out the repo for a demo screenshot of this running.</p>\n<h3><a id=\\"cargocheckexternaltypes_50\\"></a><strong>cargo-check-external-types</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/27g\\" target=\\"_blank\\">cargo-check-external-types</a></ins> is a static analysis tool for Rust library authors to set and verify which types from other libraries are allowed to be are exposed in their public API. This is useful for ensuring that a breaking change to a dependency doesn’t force a breaking change in the library that’s using it.</p>\n<p>The tool has two output formats to cover different use-cases:</p>\n<ul>\\n<li>errors (the default): Output error messages for each type that is exposed in the public API and exit with status 1 if there is at least one error. This is useful for continuous integration.</li>\n<li>markdown-table: Output the places types are exposed as a Markdown table. This is intended as a discovery tool for established projects.</li>\n</ul>\\n<p>Make sure you check out the docs for more details on this handy tool, as well as some important prereq’s that you need before you can use this too.</p>\n<h3><a id=\\"OQpy_62\\"></a><strong>OQpy</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/26s\\" target=\\"_blank\\">OQpy</a></ins> The goal of oqpy (“ock-pie”) is to make it easy to generate OpenQASM 3 + OpenPulse in Python. The oqpy library builds off of the openqasm3 and openpulse packages, which serve as Python reference implementations of the abstract syntax tree (AST) for the OpenQASM 3 and OpenPulse grammars. What are OpenQASM 3 and OpenPulse? Well I had the same questions, and thankfully these are answered in the project readme. We also have this post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/26t\\" target=\\"_blank\\">Amazon Web Services open-sources OQpy to make it easier to write quantum programs in OpenQASM 3</a></ins> where Philip Reinhold, Li Chen, Jean-Christophe Jaskula, Peter Karalekas, and Stephanie Teo collaborate to share an example of what is possible using OQpy and OpenQASM 3.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/0e5c357ba26d4d6791858b1d7cadd44b_fig3-3-2.png\\" alt=\\"fig332.png\\" /></p>\n<h2><a id=\\"Demos_Samples_Solutions_and_Workshops_68\\"></a><strong>Demos, Samples, Solutions and Workshops</strong></h2>\\n<h3><a id=\\"spinwasmmulticompute_70\\"></a><strong>spin-wasm-multi-compute</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/277\\" target=\\"_blank\\">spin-wasm-multi-compute</a></ins> is a new “science project” from all round good egg Massimo Re Ferrè where he provides some code samples as he explores the wonderful world of wasm. This demo repo is a rough proof of concept for running the same WASM artefact on Amazon EC2, Amazon ECS/Fargate and Amazon Web Services Lambda. I also love how Massimo has used another open source project (cdk-dia) to visualise the resulting stack that is deployed via Amazon Web Services CDK.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/d8f3a906d9674317a73c876b1f6f3482_architecture.png\\" alt=\\"architecture.png\\" /></p>\n<h3><a id=\\"detectandhandlingcustompiiwithAmazon_Web_ServicesgluestudioandAmazon_Web_Servicesgluedatabrew_76\\"></a><strong>detect-and-handling-custom-pii-with-Amazon Web Services-glue-studio-and-Amazon Web Services-glue-databrew</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/278\\" target=\\"_blank\\">detect-and-handling-custom-pii-with-Amazon Web Services-glue-studio-and-Amazon Web Services-glue-databrew</a></ins> aside from winning this weeks open source repo with the longest name, provides an Amazon Web Services CloudFormation (CFn) template that builds sample data pipeline, with the functionality of identifying personal and sensitive data using the Sensitive Data Detection feature of Amazon Web Services Glue Studio and apply hashing algorithm to protect columns identified with the use of Amazon Web Services Glue DataBrew, through an event-driven and serverless architecture.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/0a5d35e4bddb40698f8889e98cfe55a7_sensitive-data-detection-architecture.png\\" alt=\\"sensitivedatadetectionarchitecture.png\\" /></p>\n<h3><a id=\\"apprunnerhotelapp_82\\"></a><strong>apprunner-hotel-app</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/279\\" target=\\"_blank\\">apprunner-hotel-app</a></ins> this project provisions the base layer infrastructure to demonstrate how AppRunner leverages a VPC Connector to interact with a DB in a private subnet.</p>\n<h3><a id=\\"Amazon_Web_Servicesdynamodbparallelqueries_86\\"></a><strong>Amazon Web Services-dynamodb-parallel-queries</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/27a\\" target=\\"_blank\\">Amazon Web Services-dynamodb-parallel-queries</a></ins> this repo provides a sample application and queries for you to see how these work and then configure a load generator which you can tune, to see how they perform. Code is in Java.</p>\n<h3><a id=\\"selectivesearchwithmutualinformationcotraining_91\\"></a><strong>selective-search-with-mutual-information-cotraining</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/27b\\" target=\\"_blank\\">selective-search-with-mutual-information-cotraining</a></ins> this repo provide code samples for the research paper, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/27c\\" target=\\"_blank\\">MICO: Selective Search with Mutual Information Co-training</a></ins>. What is this research paper on? From the paper:</p>\n<p>In contrast to traditional exhaustive search, selective search first clusters documents into several groups before all the documents are searched exhaustively by a query, to limit the search executed within one group or only a few groups. Selective search is designed to reduce the latency and computation in modern large-scale search systems.</p>\n<h2><a id=\\"Amazon_Web_Services_and_Community_blog_posts_98\\"></a><strong>Amazon Web Services and Community blog posts</strong></h2>\\n<h3><a id=\\"Amazon_Web_Services_CDK_100\\"></a><strong>Amazon Web Services CDK</strong></h3>\\n<p>Amazon Web Services Community Builder Ran Isenberg has put together this post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/273\\" target=\\"_blank\\">Deploy to Amazon Web Services with GitHub Actions and Amazon Web Services CDK</a></ins> that provides a hands on guide to show you how you can use Amazon Web Services CDK to deploy your CDK stacks via GitHub Actions. Need to give this one a try at some point. [hands on]</p>\n<h3><a id=\\"eventcatalog_104\\"></a><strong>eventcatalog</strong></h3>\\n<p>Amazon Web Services Serverless Developer Advocate all round open source good guy, David Boyne built <ins><a href=\\"https://aws-oss.beachgeek.co.uk/18d\\" target=\\"_blank\\">EventCatalog</a></ins> as a POC and experiment to see if it could help document event driven architectures. Earlier this week he shared a post on how some of the community have been using it and putting together “epic guides” (we need way more epic guides in my opinion). In <ins><a href=\\"https://aws-oss.beachgeek.co.uk/274\\" target=\\"_blank\\">Using Amazon Web Services CDK to Deploy EventCatalog</a></ins> Matt Martz shows you how you can deploy EventCatalog to Amazon Web Services using Amazon Web Services CDK.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/8593f55667a24077bff7ee5fb7820d36_jTQ0lrEnP.jpg\\" alt=\\"jTQ0lrEnP.jpg\\" /></p>\n<h3><a id=\\"Redis_110\\"></a><strong>Redis</strong></h3>\\n<p>Fellow Developer Advocate Abhishek Gupta has put together something for Redis fans in his post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/272\\" target=\\"_blank\\">Using Redis on Cloud? Here are ten things you should know</a></ins> where he covers a range of Redis-related best practices, tips and tricks including cluster scalability, client side configuration, integration, and metrics.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/442fb55903b94e9780b9ec9223d1d287_8rxqHosu3Fwv27XFiPWV3weoNTYh-UilKyoM4vnx3wbvDD3TBAAjfbv-bpLMvQCFoFL_L2HUQF4USJ2UVMFf4wO-zeJymPcv5PhzcsIcGd0zr_aqFYtZrYVt0SRmzxifE_bwbKVHqqhHYSSshrDFpju4cF5VShPn7q5H8vMwS7Gj--RKVyhxQRv9tw.png\\" alt=\\"8rxqHosu3Fwv27XFiPWV3weoNTYhUilKyoM4vnx3wbvDD3TBAAjfbvbpLMvQCFoFL_L2HUQF4USJ2UVMFf4wOzeJymPcv5PhzcsIcGd0zr_aqFYtZrYVt0SRmzxifE_bwbKVHqqhHYSSshrDFpju4cF5VShPn7q5H8vMwS7GjRKVyhxQRv9tw.png\\" /></p>\n<h3><a id=\\"traefik_116\\"></a><strong>traefik</strong></h3>\\n<p>Traefik is a modern HTTP reverse proxy and load balancer that makes deploying micro services easy. Amazon Web Services Community Builder John Mille has put together this post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/271\\" target=\\"_blank\\">Traefik Proxy in Amazon Web Services with Amazon Web Services ECS - Part 2</a></ins>, showing you how you can use this in conjunction with Amazon Web Services ECS Anywhere to manage your ECS traffic where ever it is.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/d80d869beee04138896d16be5e541cdd_ecs-anywhere.drawio.png\\" alt=\\"ecsanywhere.drawio.png\\" /></p>\n<h3><a id=\\"Kotlin_122\\"></a><strong>Kotlin</strong></h3>\\n<p>Elena van Engelen-Maslova dives deep into understanding the performance characteristics of Kotlin on Amazon Web Services Lambda in her post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/270\\" target=\\"_blank\\">This Week in Kotlin on Amazon Web Services Lambda: To ARM64 or Not To ARM64?</a></ins>. Exploring JVM and GraalVM native runtimes and how they compare are just some of the insights that await you dear reader.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/96789c1fc6614e788b61ac19e62d8c67_1%20jcWkvVJpVf5syBkBl25j0A.png\\" alt=\\"1 jcWkvVJpVf5syBkBl25j0A.png\\" /></p>\n<h3><a id=\\"Ploomber_and_Soopervisor_128\\"></a><strong>Ploomber and Soopervisor</strong></h3>\\n<p>Ploomber and Soopervisor and two open source tools that you can use to build data pipelines in Python (Ploomber) and export those data pipelines to a number of outputs (Argo, Apache Airflow, Amazon Web Services Batch and others). In the second part of a three part blog post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/26z\\" target=\\"_blank\\">Deploying a Data Science Platform on Amazon Web Services: Running containerized experiments (Part II)</a></ins> Eduardo Blancas provides a detailed walk through of how you can use these on Amazon Web Services. Watch out for part three coming soon.</p>\n<h3><a id=\\"Flask_133\\"></a><strong>Flask</strong></h3>\\n<p>In the post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/26u\\" target=\\"_blank\\">Automate Python Flask Deployment to the Amazon Web Services Cloud</a></ins>, Joseph Keating and Matt Hedges demonstrate how to use Amazon Web Services services and open source tools to automate the deployment of a Python Flask application to the Amazon Web Services Cloud.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/e1b91ba673744784a64b4f4fc8432fdb_Application-architecture-diagram.png\\" alt=\\"Applicationarchitecturediagram.png\\" /></p>\n<h3><a id=\\"Composer_139\\"></a><strong>Composer</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/26r\\" target=\\"_blank\\">Composer</a></ins> is a PyTorch library that enables you to train neural networks faster, at lower cost, and to higher accuracy. In the post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/26q\\" target=\\"_blank\\">Reduce deep learning training time and cost with MosaicML Composer on Amazon Web Services</a></ins> Bandish Shah and Olivier Cruchant highlight challenges commonly reported specifically in Deep Learning (DL) training, and how the open-source library MosaicML Composer helps solve them.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/22dce7b9c449413294b13cb42bd7b8a6_image001-3.jpg\\" alt=\\"image0013.jpg\\" /></p>\n<h3><a id=\\"Other_posts_and_quick_reads_145\\"></a><strong>Other posts and quick reads</strong></h3>\\n<ul>\\n<li><ins><a href=\\"https://aws-oss.beachgeek.co.uk/26n\\" target=\\"_blank\\">Implementing Pod Security Standards in Amazon EKS</a></ins> provides a hands on guide on how to apply various Pod security standards in Amazon EKS [hands on]</li>\n</ul>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/bbc5fb26b0c14ea5bd075748ae755b1b_pod-2.jpg\\" alt=\\"pod2.jpg\\" /></p>\n<ul>\\n<li><ins><a href=\\"https://aws-oss.beachgeek.co.uk/26o\\" target=\\"_blank\\">Run inference at scale for OpenFold, a PyTorch-based protein folding ML model, using Amazon EKS</a></ins> shows how you can deploy OpenFold models on Amazon Elastic Kubernetes Service (Amazon EKS) and how to scale the EKS clusters to drastically reduce MSA computation and protein structure inference times [hands on]</li>\n</ul>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/38cbc20a48e44650b283ccb2227eeb8c_ML-11441-Cameo_Inference_Time.png\\" alt=\\"ML11441Cameo_Inference_Time.png\\" /></p>\n<ul>\\n<li><ins><a href=\\"https://aws-oss.beachgeek.co.uk/26p\\" target=\\"_blank\\">Build real-time multi-user experiences using GraphQL on Amazon Web Services Amplify</a></ins> is a tutorial where you will build a share-able todo list app using GraphQL and Amazon Web Services Amplify [hands on]</li>\n</ul>\\n<h3><a id=\\"Kubernetes_158\\"></a><strong>Kubernetes</strong></h3>\\n<p>As KubeCon NA wraps up, there were a few announcements (covered below in the Quick Updates section) as well as some new blog posts that dive deeper into these.</p>\n<p>Steve Roberts took a look at <ins><a href=\\"https://aws.amazon.com/blogs/aws/aws-batch-for-amazon-elastic-kubernetes-service/\\" target=\\"_blank\\">Amazon Web Services Batch for Amazon Elastic Kubernetes Service</a></ins> introducing what it is, and how it works.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/4f4b18a56a954009bb754fc5f4d71b5b_batch_for_eks-animation.gif\\" alt=\\"batch_for_eksanimation.gif\\" /></p>\n<p>We also have this post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/26w\\" target=\\"_blank\\">How Amazon Web Services Batch developed support for Amazon Elastic Kubernetes Service</a></ins><br />\\nwhere Angel Pizarro and Jason Rupard look behind the scenes and talk about the initial motivation and design choices when developing the service, as well as some of the challenges that the team overcame.</p>\n<p>Whilst not related to KubeCon, Mariana Ramos Franco put together this post, <ins><a href=\\"https://dev.to/aws/!%5Bhttps://aws-oss.beachgeek.co.uk/26x%5D(https://aws-oss.beachgeek.co.uk/26x)\\" target=\\"_blank\\">Speed up Highly Available Deployments on Kubernetes</a></ins>. The Amazon Managed Service for Prometheus provides a Prometheus-compatible monitoring service that is operated on Kubernetes. In the post, she dives deep into the internals of our Prometheus service and how we speed up our deployment times without impacting availability.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/e805bc5c29ac48668b115ae644dd5fa1_zone-disruption-budget-controller.png\\" alt=\\"zonedisruptionbudgetcontroller.png\\" /></p>\n<h3><a id=\\"Quick_updates_173\\"></a><strong>Quick updates</strong></h3>\\n<h3><a id=\\"Apache_Kafka_175\\"></a><strong>Apache Kafka</strong></h3>\\n<p>Amazon Managed Streaming for Apache Kafka (Amazon MSK) now supports Apache Kafka version 3.3.1 for new and existing clusters. Apache Kafka 3.3.1 includes several bug fixes and new features that improve performance. Some of the key features include enhancements to metrics and partitioner. Amazon MSK will continue to use and manage Zookeeper for quorum management in this release for stability. For a complete list of improvements and bug fixes, see the Apache Kafka release notes for 3.3.1.</p>\n<h3><a id=\\"cdk8s_180\\"></a><strong>cdk8s</strong></h3>\\n<p>CDK For Kubernetes Plus (CDK8s+) is a multi-language class library for defining Kubernetes applications using high level intent based constructs. We featured a great overview of this in the <ins><a href=\\"https://www.twitch.tv/videos/1630135246\\" target=\\"_blank\\">last episode of Build on Open Source</a></ins> if you want a primer.</p>\n<p>CDK8s+ aims to lower the entry barrier and improve maintainability of Kubernetes manifests by offering a hand crafted construct for each core Kubernetes object, exposing a richer API with reduced complexity. With this launch, CDK8s+ is now generally available and stable for use. This means that the API will remain unchanged and fully supported (no breaking changes), at least until the next major version. CDK8s+ is vended as a separate library for each Kubernetes spec version, all those libraries are now generally available and stable to use.</p>\n<p>Customers also want to validate their manifest by applying either community or organisational policies. CDK8s now supports integration with third-party tools that facilitate this, and can perform validation as part of the synthesis process. This supports manifests produced by CDK8s adhering to the necessary policies.</p>\n<p>If you want to read more about this, check out this post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/26m\\" target=\\"_blank\\">Announcing general availability of cdk8s+ and support for manifest validation</a></ins> from Shimon Tolts, CEO and Co-Founder of Datree. The post shows you a few examples of how cdk8s+ can simplify the authoring of Kubernetes manifests, and how cdk8s+ can integrate with third-party policy enforcement tools (such as Datree), to provide a guardrail that protects your manifests against misconfigurations, and prevent them from reaching your cluster.</p>\n<h3><a id=\\"MySQL_MariaDB_and_PostgreSQL_190\\"></a><strong>MySQL, MariaDB, and PostgreSQL</strong></h3>\\n<p>Amazon Relational Database Service (RDS) for MySQL, MariaDB and PostgreSQL now support 15 read replicas per instance, including up to 5 cross region read replicas, delivering up to 3X the previous read capacity.</p>\n<p>Amazon RDS Read Replicas provide enhanced performance and durability for Amazon RDS database (DB) instances. They make it easy to elastically scale out beyond the capacity constraints of a single DB instance for read-heavy database workloads. You can create one or more replicas of a given source DB instance and serve high-volume application read traffic from multiple copies of your data, thereby increasing aggregate read throughput. For disaster recovery, read replicas can be promoted when needed to become standalone DB instances.</p>\n<h3><a id=\\"MySQL_196\\"></a><strong>MySQL</strong></h3>\\n<p>Announced this week was up to 10x faster exports to Amazon S3 for snapshot exports for Amazon Aurora MySQL-Compatible Edition for MySQL 5.7 and 8.0. The performance improvement is automatically applied to all types of database snapshot exports, including manual snapshots, automated system snapshots, and snapshots created by the Amazon Web Services Backup service.</p>\n<p>With this capability, customers can export data up to 10x faster from an Amazon Aurora database and store it in an Amazon S3 bucket using Amazon Web Services CLI or Amazon Web Services Management Console. After the data is exported, they can analyze the exported data using Amazon Athena and/or other tools and services. The export process runs in the background and doesn’t affect the performance of active database instances.</p>\n<h3><a id=\\"Apache_CloudStack_203\\"></a><strong>Apache CloudStack</strong></h3>\\n<p>Apache CloudStack, a project of the Apache Software Foundation, is open source software designed to deploy and manage large networks of virtual machines. Amazon Elastic Kubernetes Service (Amazon EKS) Anywhere on Apache CloudStack was announced last week, and this expands the choice of infrastructure options for customers running Kubernetes on-premises. Apache CloudStack enhances the list of deployment options for Amazon EKS Anywhere customers, which already includes bare metal servers and VMware vSphere.</p>\n<p>While developing this new feature, we collaborated with the Apache CloudStack community to build the Cluster API provider for Apache CloudStack (CAPC), which has now joined the Kubernetes SIG. Amazon EKS Anywhere leverages CAPC to provide customers declarative, Kubernetes-style APIs for cluster creation, configuration, and management.</p>\n<h3><a id=\\"Red_Hat_Enterprise_Linux_209\\"></a><strong>Red Hat Enterprise Linux</strong></h3>\\n<p>In addition to Bottlerocket and Ubuntu, you can now use Red Hat Enterprise Linux (RHEL) and have broader choice of operating systems to create and operate Amazon EKS Anywhere clusters in your on-premises data centres. RHEL support is available for Amazon EKS Anywhere clusters running on VMware vSphere, on Apache CloudStack, or directly on bare metal servers.</p>\n<p>RHEL is a popular operating system for on-premises workloads, and customers often customize their RHEL images with security hardening and other organisational best practices. Now, you can start from your custom base images, and use Amazon EKS Anywhere tooling to build RHEL images for running Kubernetes clusters.</p>\n<h2><a id=\\"Videos_of_the_week_215\\"></a><strong>Videos of the week</strong></h2>\\n<h3><a id=\\"Kubernetes_217\\"></a><strong>Kubernetes</strong></h3>\\n<p>Cloud Native Rejekts is the b-side conference giving a second chance to the many wonderful, but rejected, talks leading into KubeCon + CloudNativeCon. You will need to fast forward to 6:52:28 to hear Mariana Ramos Franco do a talk of her blog post that I have highlight above.</p>\n<h3><a id=\\"Crossplane_222\\"></a><strong>Crossplane</strong></h3>\\n<p>What is Crossplane? In this video, Sai Vennam from Containers from the couch shows you how Crossplane enables platform teams to manage their infrastructure and managed services on cloud providers like Amazon Web Services. He outlines how you can use Crossplane to deploy and manage a production-ready Amazon EKS cluster.</p>\n<p><video src=\\"https://dev-media.amazoncloud.cn/1e4d2b0819394dbe9a438ea70db94c78_Crossplane%20on%20Kubernetes%20Explained.mp4\\" controls=\\"controls\\"></video></p>\\n<h3><a id=\\"Observability_229\\"></a><strong>Observability</strong></h3>\\n<p>Join Toshal Dudhwala and Imaya Kumar Jagannathan as they share how you can leverage Amazon Web Services managed open-source observability solutions to monitor your containerised or serverless workloads at scale.</p>\n<p><video src=\\"https://dev-media.amazoncloud.cn/f3488cb05e8240fd8839a6226a483af6_Accelerating%20Adoption%20of%20AWS%20Open-Source%20Observability%20Services%20-%20AWS%20Online%20Tech%20Talks.mp4\\" controls=\\"controls\\"></video></p>\\n<h3><a id=\\"Build_on_Open_Source_236\\"></a><strong>Build on Open Source</strong></h3>\\n<p>For those unfamiliar with this show, Build on Open Source is where we go over this newsletter and then invite special guests to dive deep into their open source project. Expect plenty of code, demos and hopefully laughs.</p>\n<p>We have put together a playlist so that you can easily access all the other episodes of the Build on Open Source show. <ins><a href=\\"https://aws-oss.beachgeek.co.uk/24u\\" target=\\"_blank\\">Build on Open Source playlist</a></ins></p>\n<h2><a id=\\"Events_for_your_diary_242\\"></a><strong>Events for your diary</strong></h2>\\n<h3><a id=\\"All_Things_Open_244\\"></a><strong>All Things Open</strong></h3>\\n<h3><a id=\\"October_30th__2nd_November_Raleigh_NC_245\\"></a><strong>October 30th - 2nd November, Raleigh, NC</strong></h3>\\n<p>I am excited about attending and speaking at All Things Open next week (check out my session, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/26y\\" target=\\"_blank\\">Automating Data Pipelines with Apache Airflow</a></ins>) Matt Auerbach has put together this post, <ins><a href=\\"https://2022.allthingsopen.org/aws-at-ato-2022/\\" target=\\"_blank\\">Amazon Web Services at ATO 2022</a></ins> and there will be a bunch of Amazon Web Services open source folk at the Amazon Web Services booth so hope to see some of you. I will have some very nice Amazon Web Services Build On Open Source challenge coins, so come find me and share a story or swap some equally cool swag to bag one of these exclusive items.</p>\n<h3><a id=\\"Build_on_Amazon_Web_Services_Open_Source_249\\"></a><strong>Build on Amazon Web Services Open Source</strong></h3>\\n<h3><a id=\\"November_4th_9am_BST_250\\"></a><strong>November 4th, 9am BST</strong></h3>\\n<p>Join us for the sixth episode of the Build on Amazon Web Services series, featuring a live round up of the latest projects and news as well as a special guest speaker. We have another special guest lined up, and I am very excited to welcome Javier Ramirez from QuestDB who will be coming on the show and sharing more about this open source times series database that is blazingly fast. Follow the show on @buildonopen for more details. Check it out on <ins><a href=\\"https://twitch.tv/aws\\" target=\\"_blank\\">https://twitch.tv/aws</a></ins></p>\n<h3><a id=\\"Amazon_Web_Services_Elastic_Kubernetes_Service_EKS_Workshop_255\\"></a><strong>Amazon Web Services Elastic Kubernetes Service (EKS) Workshop</strong></h3>\\n<h3><a id=\\"November_10th_London_5pm_256\\"></a><strong>November 10th, London 5pm</strong></h3>\\n<p>Join us for an interactive workshop on containers, Docker, Fargate and Amazon EKS, hosted by ClearScale and Amazon Web Services. This live, virtual workshop includes three hours of interactive presentation and hands-on lab work. You will take part in the setup and deployment of containers using EKS. Follow along and work directly with Amazon Web Services professionals and ClearScale (an Amazon Web Services Premier Tier Services Partner) in this Level 200 training session.</p>\n<p>You can find out more about this event by <ins><a href=\\"https://aws-oss.beachgeek.co.uk/22y\\" target=\\"_blank\\">checking out the event page and signing up.</a></ins></p>\n<h3><a id=\\"reInvent_262\\"></a><strong>re:Invent</strong></h3>\\n<h3><a id=\\"November_28th__December_3rd_Las_Vegas_263\\"></a><strong>November 28th - December 3rd, Las Vegas</strong></h3>\\n<p>re:Invent is only a few weeks away so I want to share a few things that will hopefully be of interest.</p>\n<p>First up, we will be running the Build On Live stream throughout re:Invent and we would love to feature you! If either yourself, or perhaps you know a community member going to re:Invent and think they will absolutely love to attend the livestream, we want to hear from you. Please nominate a community member you want to hear from during Build On Live <ins><a href=\\"https://eventbox.dev/survey/6B0ED1J\\" target=\\"_blank\\">using this survey</a></ins>.</p>\n<p>Second, check out this handy way to look at all the amazing open source sessions, then check out this <ins><a href=\\"https://aws-oss.beachgeek.co.uk/252\\" target=\\"_blank\\">dashboard</a></ins> [sign up required]. I would love to hear which ones you are excited about so please let me know in the comments or via Twitter. If you want to hear what my top three, must watch sessions, then this is what I would attend (sadly, as an Amazon Web Services employee I am not allowed to attend sessions)</p>\n<ol>\\n<li>OPN306 Amazon Web Services Lambda Powertools: Lessons from the road to 10 million downloads - Heitor Lessa is going to deliver an amazing session on the journey from idea to one of the most loved and used open source tools for Amazon Web Services Lambda users</li>\n<li>BOA204 When security, safety, and urgency all matter: Handling Log4Shell - Cannot wait for this session from Abbey Fuller who will walk us through how we managed this incident</li>\n<li>OPN202 Maintaining the Amazon Web Services Amplify Framework in the open - Matt Auerbach and Ashish Nanda are going to share details on how Amplify engineering managers work with the OSS community to build open-source software</li>\n</ol>\\n<p>There are many other great open source sessions, and hopefully I will try and put together a more comprehensive list as approach re:Invent.</p>\n<h3><a id=\\"OpenSearch_279\\"></a><strong>OpenSearch</strong></h3>\\n<h3><a id=\\"Every_other_Tuesday_3pm_GMT_280\\"></a><strong>Every other Tuesday, 3pm GMT</strong></h3>\\n<p>This regular meet-up is for anyone interested in OpenSearch &amp; Open Distro. All skill levels are welcome and they cover and welcome talks on topics including: search, logging, log analytics, and data visualisation.</p>\n<p>Sign up to the next session, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/1az\\" target=\\"_blank\\">OpenSearch Community Meeting</a></ins></p>\n<h2><a id=\\"Stay_in_touch_with_open_source_at_Amazon_Web_Services_286\\"></a><strong>Stay in touch with open source at Amazon Web Services</strong></h2>\\n<p>I hope this summary has been useful. Remember to check out the <ins><a href=\\"https://aws.amazon.com/opensource/?opensource-all.sort-by=item.additionalFields.startDate&amp;opensource-all.sort-order=asc\\" target=\\"_blank\\">Open Source homepage</a></ins> to keep up to date with all our activity in open source by following us on <ins><a href=\\"https://twitter.com/AWSOpen\\" target=\\"_blank\\">@AWSOpen</a></ins></p>\n"}