Open source news and updates #82

{"value":"#### **September 20th, 2021 - Instalment #82**\nNewsletter #82.\n\nWelcome to issue #82 of this newsletter, bringing you the latests updates on open source on Amazon Web Services. This weeks featured new projects include cdk-nag (helping CDK developers to shift left and check their stacks against best practices), cfn-alarms (a nice tool to automate the creation of your CloudWatch alarms), Amazon Web Services-lambda-adapter (a project to help accelerate your web applications to serverless), as well as some new open source solutions including amazon-security-hub-automated-response-and-remediation and eks-preventative-controls.\n\nThere is another great collection of community and Amazon Web Services content, covering Apache Airflow, Apache Spark, Apache Kafka, Amazon Web Services CDK, Kubernetes, OpenTelemetry, OpenSearch, ROS, Amazon Web Services Data Wrangler, Apache Cassandra, PostgreSQL and more. For Java developers, check out the Spring Boot post and with the announcement last week of Java 17, make sure you check out the news on Amazon Corretto 17, a great openJDK based distribution.\n\nI have not had a video for a while, but this week is a good one, with the Containers from the couch team diving deep on EKS Anywhere. New events added this week, so make sure you check those out, especially the Secure Coding Virtual Summit with some great sessions covering open source security.\n\nFinally, we have a couple of new events added, so check those out and put them in your diary.\n\n##### **Celebrating open source contributors**\n\nThe articles posted in this series are only possible thanks to contributors and project maintainers and so I would like to shout out and thank those folks who really do power open source and enable us all to build on top of what they have created.\n\nSo thank you to the following open source heroes: Becki Lee, Igor Soroka, Lars Jacobsson, Raviraja Ganta, Tom Milner, Gunter Rotsaert, Jayden Aung, Tomas Christ, Danilo Poccia, Jasper Wang, Deenadayaalan Thirugnanasambandam, Sri Saran Balaji, Joseph Zhang, Andrew Love, Steven Heidel, Eric Kessler, Karen Xu, Kelvin Lo, Vadim Lyakhovich, Drew Rutledge, Mark Sailes, Justin Garrison, Vivek, Sheetal Joshi, Ugur KIRA, Ping Xiang, and Min Xia.\n\nMake sure you find and follow these builders and keep up to date with their open source projects and contributions.\n\n##### **Latest open source projects**\n###### **cdk-nag**\n\n++[cdk-nag](https://aws-oss.beachgeek.co.uk/wx)++ this was my favourite new open source project last week, and provides you with the ability to check your CDK applications for best practices using a combination of available rule packs. Inspired by cfn_nag, which does the same thing for Cloudformation. Awesome stuff.\n\n\n\n###### **cfn-alarms**\n\n++[cfn-alarms](https://aws-oss.beachgeek.co.uk/x4)++ this is a great open source project from the team at mhlabs (the MatHem teach team), which provides the ability to create CloudWatch alarms based on the resources in a CloudFormation stack. To dive a little deeper and show how this works, Lars Jacobsson put together this post, ++[An approach to loosely coupled CloudWatch alarms and contextual alerts](https://aws-oss.beachgeek.co.uk/x5)++ which also provides a reference project that you can explore in more detail.\n\n###### **MLOps-Basics**\n\n++[MLOps-Basics](https://aws-oss.beachgeek.co.uk/x1)++ this is a great project from Raviraja Ganta that has put together a nine week learning program to help you get started with MLOps. I have not gone through it yet, but it is certainly on my todo list. Each week has a corresponding blog post to help guide you through.\n\n\n\n###### **Amazon Web Services-lambda-adapter**\n\n++[amazon-lambda-adapter](https://aws-oss.beachgeek.co.uk/x2)++ this is a very handy tool that helps you to run web applications on Amazon Web Services Lambda without changing code. The project README walks you through how it works and how to get started, so why not give it a go.\n\n\n\n###### **lambda-java17-layer**\n\n++[lambda-java17-layer](https://aws-oss.beachgeek.co.uk/wy)++ why not celebrate the GA announcement of Java 17 by trying out this new Amazon Web Services Lambda layer from Mark Sailes to enable Java 17 support. Work in progress, so kick the tyres and let Mark know how you get on.\n\n###### **Amazon Web Services-security-hub-automated-response-and-remediation**\n\n++[amazon-security-hub-automated-response-and-remediation](https://aws-oss.beachgeek.co.uk/x3)++ this project is an add-on solution that works with Amazon Web Services Security Hub to provide a ready-to-deploy architecture and a library of automated playbooks. The solution makes it easier for Amazon Web Services Security Hub customers to resolve common security findings and to improve their security posture in Amazon Web Services. Detailed documentation including how to customise and create your own playbooks.\n\n\n\n###### **eks-preventative-controls**\n\n++[eks-preventative-controls](https://aws-oss.beachgeek.co.uk/wm)++ this open source project will help you setup a CI/CD pipeline, integrating Conftest and OPA Rego policies (open-source tools within CNCF Open Policy Agent suite) you can automate Kubernetes preventative controls, and help you maintain compliance in your Kubernetes environments. To help you get going, Jasper Wang and Deenadayaalan Thirugnanasambandam have put together this blog post, ++[How to automate Amazon EKS preventative controls in CI/CD using CDK and OPA/Conftest](https://aws-oss.beachgeek.co.uk/wn)++\n\n\n\n##### **Amazon Web Services and Community blog posts**\n###### **Amazon Corretto**\n\nAmazon Corretto is a no-cost, multi-platform, production-ready distribution of OpenJDK. Corretto is distributed by Amazon under an open source license. Amazon Corretto 17 is now generally available. This version supports the latest Java feature release JDK 17 and is available on Linux, Windows, and macOS, on x86 and arm.\n\nIt was great to see all the launch announcements last week, and I especially liked this ++[tweet](https://aws-oss.beachgeek.co.uk/ww)++.\n\n###### **Spring Boot**\n\nGunter Rotsaert has put together great content over the years, and last week I came across ++[How to Deploy a Spring Boot App on Amaozn ECS Cluster](https://aws-oss.beachgeek.co.uk/x7)++ which provides everything you need to know to get your Spring Boot apps running on (in my view) the best container service on Amazon Web Services.\n\n###### **Apache Kafka**\n\nKafka Connect is an open-source component of Apache Kafka that provides a framework for connecting with external systems such as databases, key-value stores, search indexes, and file systems. In the post, ++[Introducing Amazon MSK Connect – Stream Data to and from Your Apache Kafka Clusters Using Managed Connectors](https://aws-oss.beachgeek.co.uk/wo)++ Danilo Poccia introduces a new capability within Amazon Managed Streaming for Apache Kafka that makes it easier to manage Kafka Connect clusters. MSK Connect allows you to configure and deploy a connector using Kafka Connect with a just few clicks. To dive deeper, check out the post. [hands on]\n\n\n\n\n###### **Apache Spark**\n\nWhere you are running a self managed Apache Spark cluster or using Amazon EMR, you have a number of choices to help you optimise how you run those workloads on Amazon. In the post ++[Reduce costs and increase resource utilization of Apache Spark jobs on Kubernetes with Amazon EMR on Amazon EKS](https://aws-oss.beachgeek.co.uk/xd)++, Saurabh Bhutyani takes a look at how you can use the support in Amazon EMR on EKS for Spark’s pod template feature and how to use that for resource isolation and controlling costs.\n\n\n\n###### **Apache Airflow**\n\nTwo great posts this week on Apache Airflow. First up we have Tomas Christ, Solution Architect at eprimo GmbH, who provides some context and then dives into the details of why and how they migrated from self managed Apache Airflow to Amazon Managed Workflows for Apache Airflow in the post, ++[Migrating from self-managed Apache Airflow to Amazon Managed Workflows for Apache Airflow (MWAA)](https://aws-oss.beachgeek.co.uk/wp)++. If you looking to do the same, make sure you read Tomas' lessons learned and he provides some code to show you how they have deployed and automated running MWAA.\n\n\n\nFollowing that we have this post from Amazon Web Services Community Builder Tom Milner which I thought was very cool, and shows a creative way of how you can decouple but integrate workflows in Apache Airflow, using events. This is a must read post this week, so dive into ++[Decouple your DAGs with an event-driven architecture on Amazon Web Services](https://aws-oss.beachgeek.co.uk/x6)++\n\n\n\n###### **Apache Cassandra**\n\nVadim Lyakhovich shares how you can connect to your Amazon Keyspaces (for Apache Cassandra) from your local machine using some common developer tools in the post, ++[Connect to Amazon Keyspaces from your desktop using IntelliJ, PyCharm, or DataGrip IDEs](https://aws-oss.beachgeek.co.uk/wu)++\n\n###### **PostgreSQL**\n\nIn the past I have had to run benchmarks on a number of different systems, sometimes at the application level, other times at the infrastructure level. Recently, as part of some work to understand the performance characteristics of Amazon Web Services Graviton2 instances, I ran a number of benchmarks on Amazon Aurora RDS MySQl and Postgres to compare the performance. I would have loved to have had this post, ++[Automate benchmark tests for Amazon Aurora PostgreSQL](https://aws-oss.beachgeek.co.uk/ws)++ from Andrew Love. A great walk through of how to use tools like pgbench to run benchmarks, and make sure you are optimising your infrastructure for your workloads. [hands on]\n\n\n\n###### **OpenSearch**\n\nDrew Rutledge shares how you can use OpenSearch and OpenSearch Dashboards for aggregating, monitoring, and visualising application and server health, in the post ++[Monitoring VMware Cloud on Amazon Web Services Workloads with Amazon OpenSearch Service](https://aws-oss.beachgeek.co.uk/wv)++ [hands on]\n\n\n\n###### **Kubernetes**\n\nIf you have wanted to connect your Kubernetes clusters to Amazon Web Services and visualise it in Amazon EKS via the Amazon Web Services Management Console, then happy days. Sri Saran Balaji and Joseph Zhang share details on exactly how you can manage that in their post, ++[Connect any Kubernetes cluster to Amazon EKS ](https://aws-oss.beachgeek.co.uk/wq)++ [hands on]\n\n\n\n\n###### **OpenTelemetry**\n\nA couple of posts this week caught my eye.\n\nFirst up we have ++[Container Insights for Amazon EKS Support Amazon Web Services Distro for OpenTelemetry Collector](https://aws-oss.beachgeek.co.uk/xc)++, a collaboration between Ugur KIRA, Ping Xiang, and Min Xia. The post looks at the Amazon Web Services Distro for OpenTelemetry (ADOT) Collector integration with Container Insights for Amazon EKS (CloudWatch Container Insights collects, aggregates, and summaries metrics from your containerized applications and microservices). They cover architecture details, key components, and installation and setup verification details.\n\n\n\nNext, we have, ++[Amazon Web Services Lambda metrics support for Amazon Managed Service for Prometheus now available in Amazon Web Services Distro for OpenTelemetry](https://aws-oss.beachgeek.co.uk/wt)++ where intern engineers Karen Xu and Kelvin Lo describe how they added metric support to the OpenTelemetry and Amazon Web Services Distro for OpenTelemetry Lambda layers, and built and tested the metric pipeline to generate, collect, and export application metrics from Amaozn Lambda to Amazon Managed Service for Prometheus (AMP).\n\n\n\n###### **Quantum**\n\nUsing a great analogy of Java, Steven Heidel and Eric Kessler share details of what OpenQASM is, and how it is looking to help simplify and make more accessible writing and interacting with Quantum applications in the post ++[Amazon joins the OpenQASM 3.0 Technical Steering Committee](https://aws-oss.beachgeek.co.uk/wr)++. Amazon Web Services has been supporting the open source effort to develop OpenQASM 3.0 for the past year, including joining the technical steering committee.\n\n###### **Amazon Web Services Data Wrangler**\n\nThere is not enough written or said about the fantastic open source project that is Amazon Web Services Data Wrangler. In this quick start, ++[An Amazon Web Services Professional Service open source initiative](https://aws-oss.beachgeek.co.uk/x0),++ you will learn more and then try and understand how you managed without.\n\n###### **Amazon Web Services CDK**\n\nA number of posts this week covering one of my favourite infrastructure as code tools, Amazon Web Services CDK.\n\nWho doesn't like a good \"lessons learnt\" blog post? I know I am a complete sucker for these, so was very happy to read Igor Soroka's blog post, ++[My lessons after moving from CloudFormation to CDK](https://aws-oss.beachgeek.co.uk/xa)++. It will only take you a few minutes to read, and you will be all the wiser for it.\n\nAmazon Web Services Community builder Matt Morgan has put together an essential post, ++[Testing the Async Cloud with Amazon Web Services CDK](https://aws-oss.beachgeek.co.uk/x9)++ to help you navigate how you can leverage Amazon Web Services CDK together with a number of tools to automate tests for your asynchronous architectures. The post also covers some related reading on the topic of testing frameworks for asynchronous processing which was new to me and I am all the better for having read them. Matt has put together and shares a sample repository with examples for doing asynchronous testing of EventBridge and Step Functions.\n\nFinally we have Becki Lee with ++[Securing an Amazon Web Services Cloud Development Kit (CDK) App Using Regula and Open Policy Agent (OPA)](https://aws-oss.beachgeek.co.uk/xe)++ which takes a look at how you can use Regula (a security and compliance tool that works with CloudFormation and Terraform which I have featured in my newsletter before) with Amazon Web Services CDK. [hands on]\n\n\n###### **Terraform**\n\nFor folks who love and use Terraform, Jayden Aung has put together this post, ++[How To Automate Application Deployments Using Terraform and Amazon Web Services CodeDeploy](https://aws-oss.beachgeek.co.uk/x8)++ he walks you through how you can use Terraform to automate the creation of Amazon Web Services infrastructure, and work with Amazon CodeDeploy to automate deployment of a sample application.\n\n##### **Videos of the week**\nI mentioned this last week, but it was great to see it in the flesh. Justin Garrison, Vivek and Sheetal Joshi walk you through Amazon EKS Anywhere, and show you how to deploy your first local cluster.\n\n<video src=\"https://dev-media.amazoncloud.cn/0288b6f4f524471c898a1b0e0c001421_Introducing%20Amazon%20EKS%20Anywhere.mp4\" class=\"manvaVedio\" controls=\"controls\" style=\"width:160px;height:160px\"></video>\n\n##### **Quick updates**\n###### **Apache Kafka**\n\nAmazon Managed Streaming for Apache Kafka (Amazon MSK) now supports the simultaneous use of multiple authentication modes and updates to encryption-in-transit settings for Amazon MSK clusters. These features allow you to migrate your clients seamlessly from one authentication mode to another and update encryption settings to match those changes.\n\nWith this launch, you can now activate any combination of authentication modes (mutual TLS, SASL SCRAM, or IAM Access Control) on new or existing clusters, which is useful if you are migrating to a new authentication mode or need to run multiple authentication modes simultaneously. You also have the flexibility to update TLS encryption settings for data moving between clients and brokers to ensure that your encryption settings can evolve with your requirements. Additionally, you can update the Private Security Authority recognised by the cluster that can be used to sign certificates for mutual TLS authentication.\n\n\n###### **ROS**\nAmazon Web Services RoboMaker, a service that allows customers to simulate robotics applications at cloud scale, now supports container images. This feature enables customers to use the container tools that they are already familiar with to build and package their code for running simulations in RoboMaker. With container support, you can now take advantage of container features such as cross-environment execution and dependency package locking while using RoboMaker. To use this feature, you create a RoboMaker Robot Application and Simulation Application with OCI compliant images stored in Amazon ECR (Elastic Container Registry). You can then use the created applications to run simulation jobs in RoboMaker.\n\n\n##### **Events for your diary**\nComing up later this week we have...\n\n###### **Data in Motion: Combining the strengths of Amazon Web Services and Confluent**\n###### **September 22nd, 11am MDT**\n\nIn this webinar, Big Compass, Confluent, and Amazon will come together to explore the strengths of Confluent and Amazon Web Services, and how each technology can complement one another for various use cases. The webinar covers ways to combine Amazon Web Services and Confluent in a hybrid platform, how to build your serverless applications with Amazon Web Services and Confluent, and a look at real-world use cases for Amazon Web Services and Confluent.\n\nTo find out more and ++[register, click here](https://aws-oss.beachgeek.co.uk/vq)++.\n\n###### **Cloud Native Day**\n###### **23rd September, Bern Switzerland**\n\nWhat is this, an in person event returning? A stellar line up including our own Michael Hausenblas, an event looking at CNCF projects and the future of IT. Find out more and to view prices/register, by clicking ++[here](https://cloudnativeday.ch/en/#speakers)++.\n\n###### **Secure Coding Virtual Summit**\n###### **September 29, 2021**\n\nThe Secure Coding Virtual Summit is your source for everything you need to build secure code from the ground up. There are many interesting session, but check out the sessions covering how to secure and protect yourself when using open source.\n\nFull details, including speaker line up and ++[how to register, here](https://aws-oss.beachgeek.co.uk/xb)++.\n\n###### **GraphQL API security best practices with Amazon Web Services**\n###### **AppSync and Amazon Web Services Amplify**\n###### **14th October, 11am AEST**\n\nAs a developer, the most important parts of managing your applications should always include enhancing performance while strengthening security. In this webinar, we take you through security best practices for your GraphQL API’s with Amazon Web Services AppSync and Amazon Amplify, providing you with an understanding of how these can be applied to your applications. In this session, you will learn about:\n\n- GraphQL Protocol and how to configure a schema\n- Possible ways to authenticate and authorise access to GraphQL APIs\n- How to configure network security for your API\n- How to enable observability for your API with logging, tracing or auditing\n\nTo ++[register for this event, use this link](https://aws-oss.beachgeek.co.uk/ue)++.\n\n###### **Amazon SageMaker and Open-Source Tools for ML:**\n###### **Better Together**\n###### **October 7 | 11 AM PT | 2 PM ET**\n\nMany organisations rely on open-source tools to support the Machine Learning lifecycle. Amazon SageMaker has been rapidly evolving by introducing support and compatibility for various open-source frameworks. In this session, you will learn how to build a customisable ML Infrastructure based on Amazon SageMaker and open-source components. We will discuss pros and cons, the limitations of different tools that support specific stages of the ML workflow, and best practices for MLOps, to automate these stages into repeatable pipelines.\n\nTo read more and ++[register for this event, click here](https://aws-oss.beachgeek.co.uk/wz)++.\n\n###### **Flink Forwards Global 2021**\n###### **October 26th/27th**\n\nFlink Forward Global 2021 is a 2-day virtual conference for the Apache Flink and stream processing communities. Apache Flink is an open-source distributed engine for processing data streams that can support both streaming and batch workloads. Flink Forward has keynote presentations and talks on production Flink use cases, technical deep dive sessions, and the growth of the Flink ecosystem. You can meet core Flink committers, new and experienced users, and thought leaders who share experiences and best practices in stream processing, real-time analytics, and the management of mission-critical Flink deployments in production.\n\n++[Read more and sign up here](https://aws-oss.beachgeek.co.uk/wh)++.\n\n##### **Stay in touch with open source at Amazon Web Services**\nI hope this summary has been useful. Remember to check out the ++[Open Source homepage](https://aws.amazon.com/opensource/?opensource-all.sort-by=item.additionalFields.startDate&opensource-all.sort-order=asc)++ to keep up to date with all our activity in open source by following us on ++[@Amazon Web ServicesOpen](https://twitter.com/AWSOpen)++\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","render":"<h4><a id=\"September_20th_2021__Instalment_82_0\"></a><strong>September 20th, 2021 - Instalment #82</strong></h4>\n<p>Newsletter #82.</p>\n<p>Welcome to issue #82 of this newsletter, bringing you the latests updates on open source on Amazon Web Services. This weeks featured new projects include cdk-nag (helping CDK developers to shift left and check their stacks against best practices), cfn-alarms (a nice tool to automate the creation of your CloudWatch alarms), Amazon Web Services-lambda-adapter (a project to help accelerate your web applications to serverless), as well as some new open source solutions including amazon-security-hub-automated-response-and-remediation and eks-preventative-controls.</p>\n<p>There is another great collection of community and Amazon Web Services content, covering Apache Airflow, Apache Spark, Apache Kafka, Amazon Web Services CDK, Kubernetes, OpenTelemetry, OpenSearch, ROS, Amazon Web Services Data Wrangler, Apache Cassandra, PostgreSQL and more. For Java developers, check out the Spring Boot post and with the announcement last week of Java 17, make sure you check out the news on Amazon Corretto 17, a great openJDK based distribution.</p>\n<p>I have not had a video for a while, but this week is a good one, with the Containers from the couch team diving deep on EKS Anywhere. New events added this week, so make sure you check those out, especially the Secure Coding Virtual Summit with some great sessions covering open source security.</p>\n<p>Finally, we have a couple of new events added, so check those out and put them in your diary.</p>\n<h5><a id=\"Celebrating_open_source_contributors_11\"></a><strong>Celebrating open source contributors</strong></h5>\n<p>The articles posted in this series are only possible thanks to contributors and project maintainers and so I would like to shout out and thank those folks who really do power open source and enable us all to build on top of what they have created.</p>\n<p>So thank you to the following open source heroes: Becki Lee, Igor Soroka, Lars Jacobsson, Raviraja Ganta, Tom Milner, Gunter Rotsaert, Jayden Aung, Tomas Christ, Danilo Poccia, Jasper Wang, Deenadayaalan Thirugnanasambandam, Sri Saran Balaji, Joseph Zhang, Andrew Love, Steven Heidel, Eric Kessler, Karen Xu, Kelvin Lo, Vadim Lyakhovich, Drew Rutledge, Mark Sailes, Justin Garrison, Vivek, Sheetal Joshi, Ugur KIRA, Ping Xiang, and Min Xia.</p>\n<p>Make sure you find and follow these builders and keep up to date with their open source projects and contributions.</p>\n<h5><a id=\"Latest_open_source_projects_19\"></a><strong>Latest open source projects</strong></h5>\n<h6><a id=\"cdknag_20\"></a><strong>cdk-nag</strong></h6>\n<p><ins><a href=\"https://aws-oss.beachgeek.co.uk/wx\" target=\"_blank\">cdk-nag</a></ins> this was my favourite new open source project last week, and provides you with the ability to check your CDK applications for best practices using a combination of available rule packs. Inspired by cfn_nag, which does the same thing for Cloudformation. Awesome stuff.</p>\n<p><img src=\"1\" alt=\"cdk_nag.gif\" /></p>\n<h6><a id=\"cfnalarms_26\"></a><strong>cfn-alarms</strong></h6>\n<p><ins><a href=\"https://aws-oss.beachgeek.co.uk/x4\" target=\"_blank\">cfn-alarms</a></ins> this is a great open source project from the team at mhlabs (the MatHem teach team), which provides the ability to create CloudWatch alarms based on the resources in a CloudFormation stack. To dive a little deeper and show how this works, Lars Jacobsson put together this post, <ins><a href=\"https://aws-oss.beachgeek.co.uk/x5\" target=\"_blank\">An approach to loosely coupled CloudWatch alarms and contextual alerts</a></ins> which also provides a reference project that you can explore in more detail.</p>\n<h6><a id=\"MLOpsBasics_30\"></a><strong>MLOps-Basics</strong></h6>\n<p><ins><a href=\"https://aws-oss.beachgeek.co.uk/x1\" target=\"_blank\">MLOps-Basics</a></ins> this is a great project from Raviraja Ganta that has put together a nine week learning program to help you get started with MLOps. I have not gone through it yet, but it is certainly on my todo list. Each week has a corresponding blog post to help guide you through.</p>\n<p><img src=\"2\" alt=\"image.png\" /></p>\n<h6><a id=\"Amazon_Web_Serviceslambdaadapter_36\"></a><strong>Amazon Web Services-lambda-adapter</strong></h6>\n<p><ins><a href=\"https://aws-oss.beachgeek.co.uk/x2\" target=\"_blank\">amazon-lambda-adapter</a></ins> this is a very handy tool that helps you to run web applications on Amazon Web Services Lambda without changing code. The project README walks you through how it works and how to get started, so why not give it a go.</p>\n<p><img src=\"3\" alt=\"image.png\" /></p>\n<h6><a id=\"lambdajava17layer_42\"></a><strong>lambda-java17-layer</strong></h6>\n<p><ins><a href=\"https://aws-oss.beachgeek.co.uk/wy\" target=\"_blank\">lambda-java17-layer</a></ins> why not celebrate the GA announcement of Java 17 by trying out this new Amazon Web Services Lambda layer from Mark Sailes to enable Java 17 support. Work in progress, so kick the tyres and let Mark know how you get on.</p>\n<h6><a id=\"Amazon_Web_Servicessecurityhubautomatedresponseandremediation_46\"></a><strong>Amazon Web Services-security-hub-automated-response-and-remediation</strong></h6>\n<p><ins><a href=\"https://aws-oss.beachgeek.co.uk/x3\" target=\"_blank\">amazon-security-hub-automated-response-and-remediation</a></ins> this project is an add-on solution that works with Amazon Web Services Security Hub to provide a ready-to-deploy architecture and a library of automated playbooks. The solution makes it easier for Amazon Web Services Security Hub customers to resolve common security findings and to improve their security posture in Amazon Web Services. Detailed documentation including how to customise and create your own playbooks.</p>\n<p><img src=\"4\" alt=\"image.png\" /></p>\n<h6><a id=\"ekspreventativecontrols_52\"></a><strong>eks-preventative-controls</strong></h6>\n<p><ins><a href=\"https://aws-oss.beachgeek.co.uk/wm\" target=\"_blank\">eks-preventative-controls</a></ins> this open source project will help you setup a CI/CD pipeline, integrating Conftest and OPA Rego policies (open-source tools within CNCF Open Policy Agent suite) you can automate Kubernetes preventative controls, and help you maintain compliance in your Kubernetes environments. To help you get going, Jasper Wang and Deenadayaalan Thirugnanasambandam have put together this blog post, <ins><a href=\"https://aws-oss.beachgeek.co.uk/wn\" target=\"_blank\">How to automate Amazon EKS preventative controls in CI/CD using CDK and OPA/Conftest</a></ins></p>\n<p><img src=\"5\" alt=\"image.png\" /></p>\n<h5><a id=\"Amazon_Web_Services_and_Community_blog_posts_58\"></a><strong>Amazon Web Services and Community blog posts</strong></h5>\n<h6><a id=\"Amazon_Corretto_59\"></a><strong>Amazon Corretto</strong></h6>\n<p>Amazon Corretto is a no-cost, multi-platform, production-ready distribution of OpenJDK. Corretto is distributed by Amazon under an open source license. Amazon Corretto 17 is now generally available. This version supports the latest Java feature release JDK 17 and is available on Linux, Windows, and macOS, on x86 and arm.</p>\n<p>It was great to see all the launch announcements last week, and I especially liked this <ins><a href=\"https://aws-oss.beachgeek.co.uk/ww\" target=\"_blank\">tweet</a></ins>.</p>\n<h6><a id=\"Spring_Boot_65\"></a><strong>Spring Boot</strong></h6>\n<p>Gunter Rotsaert has put together great content over the years, and last week I came across <ins><a href=\"https://aws-oss.beachgeek.co.uk/x7\" target=\"_blank\">How to Deploy a Spring Boot App on Amaozn ECS Cluster</a></ins> which provides everything you need to know to get your Spring Boot apps running on (in my view) the best container service on Amazon Web Services.</p>\n<h6><a id=\"Apache_Kafka_69\"></a><strong>Apache Kafka</strong></h6>\n<p>Kafka Connect is an open-source component of Apache Kafka that provides a framework for connecting with external systems such as databases, key-value stores, search indexes, and file systems. In the post, <ins><a href=\"https://aws-oss.beachgeek.co.uk/wo\" target=\"_blank\">Introducing Amazon MSK Connect – Stream Data to and from Your Apache Kafka Clusters Using Managed Connectors</a></ins> Danilo Poccia introduces a new capability within Amazon Managed Streaming for Apache Kafka that makes it easier to manage Kafka Connect clusters. MSK Connect allows you to configure and deploy a connector using Kafka Connect with a just few clicks. To dive deeper, check out the post. [hands on]</p>\n<p><img src=\"6\" alt=\"image.png\" /></p>\n<h6><a id=\"Apache_Spark_76\"></a><strong>Apache Spark</strong></h6>\n<p>Where you are running a self managed Apache Spark cluster or using Amazon EMR, you have a number of choices to help you optimise how you run those workloads on Amazon. In the post <ins><a href=\"https://aws-oss.beachgeek.co.uk/xd\" target=\"_blank\">Reduce costs and increase resource utilization of Apache Spark jobs on Kubernetes with Amazon EMR on Amazon EKS</a></ins>, Saurabh Bhutyani takes a look at how you can use the support in Amazon EMR on EKS for Spark’s pod template feature and how to use that for resource isolation and controlling costs.</p>\n<p><img src=\"7\" alt=\"image.png\" /></p>\n<h6><a id=\"Apache_Airflow_82\"></a><strong>Apache Airflow</strong></h6>\n<p>Two great posts this week on Apache Airflow. First up we have Tomas Christ, Solution Architect at eprimo GmbH, who provides some context and then dives into the details of why and how they migrated from self managed Apache Airflow to Amazon Managed Workflows for Apache Airflow in the post, <ins><a href=\"https://aws-oss.beachgeek.co.uk/wp\" target=\"_blank\">Migrating from self-managed Apache Airflow to Amazon Managed Workflows for Apache Airflow (MWAA)</a></ins>. If you looking to do the same, make sure you read Tomas’ lessons learned and he provides some code to show you how they have deployed and automated running MWAA.</p>\n<p><img src=\"8\" alt=\"image.png\" /></p>\n<p>Following that we have this post from Amazon Web Services Community Builder Tom Milner which I thought was very cool, and shows a creative way of how you can decouple but integrate workflows in Apache Airflow, using events. This is a must read post this week, so dive into <ins><a href=\"https://aws-oss.beachgeek.co.uk/x6\" target=\"_blank\">Decouple your DAGs with an event-driven architecture on Amazon Web Services</a></ins></p>\n<p><img src=\"9\" alt=\"image.png\" /></p>\n<h6><a id=\"Apache_Cassandra_92\"></a><strong>Apache Cassandra</strong></h6>\n<p>Vadim Lyakhovich shares how you can connect to your Amazon Keyspaces (for Apache Cassandra) from your local machine using some common developer tools in the post, <ins><a href=\"https://aws-oss.beachgeek.co.uk/wu\" target=\"_blank\">Connect to Amazon Keyspaces from your desktop using IntelliJ, PyCharm, or DataGrip IDEs</a></ins></p>\n<h6><a id=\"PostgreSQL_96\"></a><strong>PostgreSQL</strong></h6>\n<p>In the past I have had to run benchmarks on a number of different systems, sometimes at the application level, other times at the infrastructure level. Recently, as part of some work to understand the performance characteristics of Amazon Web Services Graviton2 instances, I ran a number of benchmarks on Amazon Aurora RDS MySQl and Postgres to compare the performance. I would have loved to have had this post, <ins><a href=\"https://aws-oss.beachgeek.co.uk/ws\" target=\"_blank\">Automate benchmark tests for Amazon Aurora PostgreSQL</a></ins> from Andrew Love. A great walk through of how to use tools like pgbench to run benchmarks, and make sure you are optimising your infrastructure for your workloads. [hands on]</p>\n<p><img src=\"10\" alt=\"image.png\" /></p>\n<h6><a id=\"OpenSearch_102\"></a><strong>OpenSearch</strong></h6>\n<p>Drew Rutledge shares how you can use OpenSearch and OpenSearch Dashboards for aggregating, monitoring, and visualising application and server health, in the post <ins><a href=\"https://aws-oss.beachgeek.co.uk/wv\" target=\"_blank\">Monitoring VMware Cloud on Amazon Web Services Workloads with Amazon OpenSearch Service</a></ins> [hands on]</p>\n<p><img src=\"11\" alt=\"image.png\" /></p>\n<h6><a id=\"Kubernetes_108\"></a><strong>Kubernetes</strong></h6>\n<p>If you have wanted to connect your Kubernetes clusters to Amazon Web Services and visualise it in Amazon EKS via the Amazon Web Services Management Console, then happy days. Sri Saran Balaji and Joseph Zhang share details on exactly how you can manage that in their post, <ins><a href=\"https://aws-oss.beachgeek.co.uk/wq\" target=\"_blank\">Connect any Kubernetes cluster to Amazon EKS </a></ins> [hands on]</p>\n<p><img src=\"12\" alt=\"image.png\" /></p>\n<h6><a id=\"OpenTelemetry_115\"></a><strong>OpenTelemetry</strong></h6>\n<p>A couple of posts this week caught my eye.</p>\n<p>First up we have <ins><a href=\"https://aws-oss.beachgeek.co.uk/xc\" target=\"_blank\">Container Insights for Amazon EKS Support Amazon Web Services Distro for OpenTelemetry Collector</a></ins>, a collaboration between Ugur KIRA, Ping Xiang, and Min Xia. The post looks at the Amazon Web Services Distro for OpenTelemetry (ADOT) Collector integration with Container Insights for Amazon EKS (CloudWatch Container Insights collects, aggregates, and summaries metrics from your containerized applications and microservices). They cover architecture details, key components, and installation and setup verification details.</p>\n<p><img src=\"13\" alt=\"image.png\" /></p>\n<p>Next, we have, <ins><a href=\"https://aws-oss.beachgeek.co.uk/wt\" target=\"_blank\">Amazon Web Services Lambda metrics support for Amazon Managed Service for Prometheus now available in Amazon Web Services Distro for OpenTelemetry</a></ins> where intern engineers Karen Xu and Kelvin Lo describe how they added metric support to the OpenTelemetry and Amazon Web Services Distro for OpenTelemetry Lambda layers, and built and tested the metric pipeline to generate, collect, and export application metrics from Amaozn Lambda to Amazon Managed Service for Prometheus (AMP).</p>\n<p><img src=\"14\" alt=\"image.png\" /></p>\n<h6><a id=\"Quantum_127\"></a><strong>Quantum</strong></h6>\n<p>Using a great analogy of Java, Steven Heidel and Eric Kessler share details of what OpenQASM is, and how it is looking to help simplify and make more accessible writing and interacting with Quantum applications in the post <ins><a href=\"https://aws-oss.beachgeek.co.uk/wr\" target=\"_blank\">Amazon joins the OpenQASM 3.0 Technical Steering Committee</a></ins>. Amazon Web Services has been supporting the open source effort to develop OpenQASM 3.0 for the past year, including joining the technical steering committee.</p>\n<h6><a id=\"Amazon_Web_Services_Data_Wrangler_131\"></a><strong>Amazon Web Services Data Wrangler</strong></h6>\n<p>There is not enough written or said about the fantastic open source project that is Amazon Web Services Data Wrangler. In this quick start, <ins><a href=\"https://aws-oss.beachgeek.co.uk/x0\" target=\"_blank\">An Amazon Web Services Professional Service open source initiative</a>,</ins> you will learn more and then try and understand how you managed without.</p>\n<h6><a id=\"Amazon_Web_Services_CDK_135\"></a><strong>Amazon Web Services CDK</strong></h6>\n<p>A number of posts this week covering one of my favourite infrastructure as code tools, Amazon Web Services CDK.</p>\n<p>Who doesn’t like a good “lessons learnt” blog post? I know I am a complete sucker for these, so was very happy to read Igor Soroka’s blog post, <ins><a href=\"https://aws-oss.beachgeek.co.uk/xa\" target=\"_blank\">My lessons after moving from CloudFormation to CDK</a></ins>. It will only take you a few minutes to read, and you will be all the wiser for it.</p>\n<p>Amazon Web Services Community builder Matt Morgan has put together an essential post, <ins><a href=\"https://aws-oss.beachgeek.co.uk/x9\" target=\"_blank\">Testing the Async Cloud with Amazon Web Services CDK</a></ins> to help you navigate how you can leverage Amazon Web Services CDK together with a number of tools to automate tests for your asynchronous architectures. The post also covers some related reading on the topic of testing frameworks for asynchronous processing which was new to me and I am all the better for having read them. Matt has put together and shares a sample repository with examples for doing asynchronous testing of EventBridge and Step Functions.</p>\n<p>Finally we have Becki Lee with <ins><a href=\"https://aws-oss.beachgeek.co.uk/xe\" target=\"_blank\">Securing an Amazon Web Services Cloud Development Kit (CDK) App Using Regula and Open Policy Agent (OPA)</a></ins> which takes a look at how you can use Regula (a security and compliance tool that works with CloudFormation and Terraform which I have featured in my newsletter before) with Amazon Web Services CDK. [hands on]</p>\n<h6><a id=\"Terraform_146\"></a><strong>Terraform</strong></h6>\n<p>For folks who love and use Terraform, Jayden Aung has put together this post, <ins><a href=\"https://aws-oss.beachgeek.co.uk/x8\" target=\"_blank\">How To Automate Application Deployments Using Terraform and Amazon Web Services CodeDeploy</a></ins> he walks you through how you can use Terraform to automate the creation of Amazon Web Services infrastructure, and work with Amazon CodeDeploy to automate deployment of a sample application.</p>\n<h5><a id=\"Videos_of_the_week_150\"></a><strong>Videos of the week</strong></h5>\n<p>I mentioned this last week, but it was great to see it in the flesh. Justin Garrison, Vivek and Sheetal Joshi walk you through Amazon EKS Anywhere, and show you how to deploy your first local cluster.</p>\n<p><video src=\"https://dev-media.amazoncloud.cn/0288b6f4f524471c898a1b0e0c001421_Introducing%20Amazon%20EKS%20Anywhere.mp4\" controls=\"controls\"></video></p>\n<h5><a id=\"Quick_updates_155\"></a><strong>Quick updates</strong></h5>\n<h6><a id=\"Apache_Kafka_156\"></a><strong>Apache Kafka</strong></h6>\n<p>Amazon Managed Streaming for Apache Kafka (Amazon MSK) now supports the simultaneous use of multiple authentication modes and updates to encryption-in-transit settings for Amazon MSK clusters. These features allow you to migrate your clients seamlessly from one authentication mode to another and update encryption settings to match those changes.</p>\n<p>With this launch, you can now activate any combination of authentication modes (mutual TLS, SASL SCRAM, or IAM Access Control) on new or existing clusters, which is useful if you are migrating to a new authentication mode or need to run multiple authentication modes simultaneously. You also have the flexibility to update TLS encryption settings for data moving between clients and brokers to ensure that your encryption settings can evolve with your requirements. Additionally, you can update the Private Security Authority recognised by the cluster that can be used to sign certificates for mutual TLS authentication.</p>\n<h6><a id=\"ROS_163\"></a><strong>ROS</strong></h6>\n<p>Amazon Web Services RoboMaker, a service that allows customers to simulate robotics applications at cloud scale, now supports container images. This feature enables customers to use the container tools that they are already familiar with to build and package their code for running simulations in RoboMaker. With container support, you can now take advantage of container features such as cross-environment execution and dependency package locking while using RoboMaker. To use this feature, you create a RoboMaker Robot Application and Simulation Application with OCI compliant images stored in Amazon ECR (Elastic Container Registry). You can then use the created applications to run simulation jobs in RoboMaker.</p>\n<h5><a id=\"Events_for_your_diary_167\"></a><strong>Events for your diary</strong></h5>\n<p>Coming up later this week we have…</p>\n<h6><a id=\"Data_in_Motion_Combining_the_strengths_of_Amazon_Web_Services_and_Confluent_170\"></a><strong>Data in Motion: Combining the strengths of Amazon Web Services and Confluent</strong></h6>\n<h6><a id=\"September_22nd_11am_MDT_171\"></a><strong>September 22nd, 11am MDT</strong></h6>\n<p>In this webinar, Big Compass, Confluent, and Amazon will come together to explore the strengths of Confluent and Amazon Web Services, and how each technology can complement one another for various use cases. The webinar covers ways to combine Amazon Web Services and Confluent in a hybrid platform, how to build your serverless applications with Amazon Web Services and Confluent, and a look at real-world use cases for Amazon Web Services and Confluent.</p>\n<p>To find out more and <ins><a href=\"https://aws-oss.beachgeek.co.uk/vq\" target=\"_blank\">register, click here</a></ins>.</p>\n<h6><a id=\"Cloud_Native_Day_177\"></a><strong>Cloud Native Day</strong></h6>\n<h6><a id=\"23rd_September_Bern_Switzerland_178\"></a><strong>23rd September, Bern Switzerland</strong></h6>\n<p>What is this, an in person event returning? A stellar line up including our own Michael Hausenblas, an event looking at CNCF projects and the future of IT. Find out more and to view prices/register, by clicking <ins><a href=\"https://cloudnativeday.ch/en/#speakers\" target=\"_blank\">here</a></ins>.</p>\n<h6><a id=\"Secure_Coding_Virtual_Summit_182\"></a><strong>Secure Coding Virtual Summit</strong></h6>\n<h6><a id=\"September_29_2021_183\"></a><strong>September 29, 2021</strong></h6>\n<p>The Secure Coding Virtual Summit is your source for everything you need to build secure code from the ground up. There are many interesting session, but check out the sessions covering how to secure and protect yourself when using open source.</p>\n<p>Full details, including speaker line up and <ins><a href=\"https://aws-oss.beachgeek.co.uk/xb\" target=\"_blank\">how to register, here</a></ins>.</p>\n<h6><a id=\"GraphQL_API_security_best_practices_with_Amazon_Web_Services_189\"></a><strong>GraphQL API security best practices with Amazon Web Services</strong></h6>\n<h6><a id=\"AppSync_and_Amazon_Web_Services_Amplify_190\"></a><strong>AppSync and Amazon Web Services Amplify</strong></h6>\n<h6><a id=\"14th_October_11am_AEST_191\"></a><strong>14th October, 11am AEST</strong></h6>\n<p>As a developer, the most important parts of managing your applications should always include enhancing performance while strengthening security. In this webinar, we take you through security best practices for your GraphQL API’s with Amazon Web Services AppSync and Amazon Amplify, providing you with an understanding of how these can be applied to your applications. In this session, you will learn about:</p>\n<ul>\n<li>GraphQL Protocol and how to configure a schema</li>\n<li>Possible ways to authenticate and authorise access to GraphQL APIs</li>\n<li>How to configure network security for your API</li>\n<li>How to enable observability for your API with logging, tracing or auditing</li>\n</ul>\n<p>To <ins><a href=\"https://aws-oss.beachgeek.co.uk/ue\" target=\"_blank\">register for this event, use this link</a></ins>.</p>\n<h6><a id=\"Amazon_SageMaker_and_OpenSource_Tools_for_ML_202\"></a><strong>Amazon SageMaker and Open-Source Tools for ML:</strong></h6>\n<h6><a id=\"Better_Together_203\"></a><strong>Better Together</strong></h6>\n<h6><a id=\"October_7__11_AM_PT__2_PM_ET_204\"></a><strong>October 7 | 11 AM PT | 2 PM ET</strong></h6>\n<p>Many organisations rely on open-source tools to support the Machine Learning lifecycle. Amazon SageMaker has been rapidly evolving by introducing support and compatibility for various open-source frameworks. In this session, you will learn how to build a customisable ML Infrastructure based on Amazon SageMaker and open-source components. We will discuss pros and cons, the limitations of different tools that support specific stages of the ML workflow, and best practices for MLOps, to automate these stages into repeatable pipelines.</p>\n<p>To read more and <ins><a href=\"https://aws-oss.beachgeek.co.uk/wz\" target=\"_blank\">register for this event, click here</a></ins>.</p>\n<h6><a id=\"Flink_Forwards_Global_2021_210\"></a><strong>Flink Forwards Global 2021</strong></h6>\n<h6><a id=\"October_26th27th_211\"></a><strong>October 26th/27th</strong></h6>\n<p>Flink Forward Global 2021 is a 2-day virtual conference for the Apache Flink and stream processing communities. Apache Flink is an open-source distributed engine for processing data streams that can support both streaming and batch workloads. Flink Forward has keynote presentations and talks on production Flink use cases, technical deep dive sessions, and the growth of the Flink ecosystem. You can meet core Flink committers, new and experienced users, and thought leaders who share experiences and best practices in stream processing, real-time analytics, and the management of mission-critical Flink deployments in production.</p>\n<p><ins><a href=\"https://aws-oss.beachgeek.co.uk/wh\" target=\"_blank\">Read more and sign up here</a></ins>.</p>\n<h5><a id=\"Stay_in_touch_with_open_source_at_Amazon_Web_Services_217\"></a><strong>Stay in touch with open source at Amazon Web Services</strong></h5>\n<p>I hope this summary has been useful. Remember to check out the <ins><a href=\"https://aws.amazon.com/opensource/?opensource-all.sort-by=item.additionalFields.startDate&amp;opensource-all.sort-order=asc\" target=\"_blank\">Open Source homepage</a></ins> to keep up to date with all our activity in open source by following us on <ins><a href=\"https://twitter.com/AWSOpen\" target=\"_blank\">@Amazon Web ServicesOpen</a></ins></p>\n"}