Open source news and updates #137

海外精选
开源
海外精选的内容汇集了全球优质的亚马逊云科技相关技术内容。同时,内容中提到的“AWS” 是 “Amazon Web Services” 的缩写,在此网站不作为商标展示。
0
0
{"value":"## **November 25th, 2022 - Instalment #137**\n### Welcome\n\n\nWelcome to the Amazon Web Services open source newsletter, edition #137. As it is re:Invent next week, I will be publishing the newsletter early as I am heading out on Monday. I will be in Las Vegas talking with open source Builders, hanging out on the Open Source Kiosk in the Amazon Web Services Village, and doing some talks. If you are coming, I would love to meet some of you, so get in touch. I will also be taking a break for a week, so the next newsletter will be on December 12th.\n\nAs always, this week we have more new projects for you to practice your four freedoms on, including a couple of projects for those who are looking to perhaps stand up their own Mastadon instances. \"Amazon Web Services-vpc-flowlogs-enricher\" is a project to help you add additional data into your VPC Flow logs, \"Amazon Web Services-security-assessment-solution\" a solution that uses some open source security tools that you can use to assess your Amazon Web Services accounts, \"Amazon Web Services-backup-amplify-appsync\" a tool for all Amazon Web Services Amplify users need to know about, \"message-bus-bridge\" is a tool to help you copy messages between message bus', \"monitor-serverless-datalake\" keep on top of your data lakes with this solution, \"ec2-image-builder-send-approval-notifications-before-sharing-ami\" shows you how you can add a notification step in the AMI building workflow, \"amazon-ecs-fargate-cdk-v2-cicd\" is a nice demonstration on using Amazon Web Services CDKv2 with Flask, \"deploy-nth-to-eks\" a tool for Kubernetes admins, and a few more projects too!\n\nWith the run up to re:Invent, the Amazon Web Services Amplify team have been on fire, and we have lots of content for Amazon Web Services Amplify users and fans. We also have content covering your favourite open source projects, including GraphQL, Grafana, Prometheus, MariaDB, PostgreSQL, Flutter, React, Apache Iceberg, Apache Airflow, Apache Flink, Apache ShardingSphere, AutoGluon, Amazon Web Services ParallelCluster, Kubeflow, NGINX, Finch, [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail), Trino, Apache Hudi, O3DE, Apache Kafka, OpenSearch, MLFlow, and more.\n\nFinally, with re:Invent upon us, make sure you check the events section for everything you need to know to make sure you do not miss the best open source sessions.\n\n\n### **Amazon Web Services Copilot - have your say**\n\nThe Amazon Web Services Copilot project has created a new design proposal for overriding Copilot abstracted resources using the Amazon Web Services Cloud Development Kit (CDK). The goal is to provide a \"break the glass\" mechanism to access and configure functionality that is not surfaced by Copilot manifests by leveraging the expressive power of a programming language. Have your say by heading over to ++[Extending Copilot with the CDK](https://aws-oss.beachgeek.co.uk/2b9)++ and joining the discussion.\n\n### **Feedback**\n\nPlease let me know how we can improve this newsletter as well as how Amazon Web Services can better work with open source projects and technologies by completing ++[ this very short survey](https://eventbox.dev/survey/NUSZ91Z)++ that will take you probably less than 30 seconds to complete. Thank you so much!\n\n### **Celebrating open source contributors**\n\nThe articles and projects shared in this newsletter are only possible thanks to the many contributors in open source. I would like to shout out and thank those folks who really do power open source and enable us all to learn and build on top of what they have created.\n\nSo thank you to the following open source heroes: John Preston, Andreas Wittig, Michael Wittig, Uma Ramadoss, Boni Bruno, Eric Henderson, Chelluru Vidyadhar, Vijay Karumajji, Justin Lim, Krishna Sarabu, Chirag Dave, and Mark Townsend\n\n### **Latest open source projects**\n\n*The great thing about open source projects is that you can review the source code. If you like the look of these projects, make sure you that take a look at the code, and if it is useful to you, get in touch with the maintainer to provide feedback, suggestions or even submit a contribution.*\n\n### **Tools**\n### **Amazon Web Services-sam-cli-pipeline-init-templates**\n\n++[Amazon Web Services-sam-cli-pipeline-init-templates](https://aws-oss.beachgeek.co.uk/2av)++ This repository contains the pipeline init templates used in the Amazon Web Services SAM CLI for sam pipeline commands. Customers can now incrementally add services to their repository and automate the creation and execution of pipelines for each new #serverless service. The template creates the necessary supporting infrastructure to keep track of commit history and changes that occur in your directories, so only the modified service pipeline is triggered. Get started by simply choosing option 2 when you initialise and bootstrap and new pipeline.\n\n### **Amazon Web Services-security-assessment-solution**\n\n++[Amazon Web Services-security-assessment-solution](https://aws-oss.beachgeek.co.uk/2ak)++ Cybersecurity remains a very important topic and point of concern for many CIOs, CISOs, and their customers. To meet these important concerns, Amazon Web Services has developed a primary set of services customers should use to aid in protecting their accounts. [Amazon GuardDuty](https://aws.amazon.com/cn/guardduty/?trk=cndc-detail), Amazon Web Services Security Hub, Amazon Web Services Config, and Amazon Web Services Well-Architected reviews help customers maintain a strong security posture over their Amazon Web Services accounts. As more organizations deploy to the cloud, especially if they are doing so quickly, and they have not yet implemented the recommended Amazon Web Services Services, there may be a need to conduct a rapid security assessment of the cloud environment. With that in mind, we have worked to develop an inexpensive, easy to deploy, secure, and fast solution to provide our customers two (2) security assessment reports. These security assessments are from the open source projects “Prowler” and “ScoutSuite.” Each of these projects conduct an assessment based on Amazon Web Services best practices and can help quickly identify any potential risk areas in a customer’s deployed environment.\n\n### **Amazon Web Services-backup-amplify-appsync**\n\n++[Amazon Web Services-backup-amplify-appsync](https://aws-oss.beachgeek.co.uk/2ax)++ Amazon Web Services Amplify makes it easy to build full stack front end UI apps with backends and authentication. Amazon Web Services AppSync adds serverless GraphQL and DynamoDB tables to your application with no code. This project guides you on how to include the infrastructure as code to add Amazon Web Services Backup to an Amplify and AppSync application using to manage snapshots for your applications DynamoDB tables.\n\n![image.png](https://dev-media.amazoncloud.cn/7c31f278f6df416399c9776ad65a1534_image.png)\n\n### **monitor-serverless-datalake**\n\n++[monitor-serverless-datalake](https://aws-oss.beachgeek.co.uk/2ay)++ This repository serves as a launch pad for monitoring serverless data lakes in Amazon Web Services. The objective is to provide a plug and play mechanism for monitoring enterprise scale data lakes. Data lakes starts small and rapidly explodes with adoption. With growing adoption, the data pipelines also grows in number and complexity. It is pivotal to ensure that the data pipeline executes as per SLA and failures be mitigated. The solution provides mechanisms for the following, 1. Capture state changes across all tasks in the data lake 2. Quickly notify operations of failures as they happen 3. Measure service reliability across data lake – to identify opportunities for performance optimisation\n\n![image.png](https://dev-media.amazoncloud.cn/46d4dac545be489f8f9dc567dbaef997_image.png)\n\n### **message-bus-bridge**\n\n++[message-bus-bridge](https://aws-oss.beachgeek.co.uk/2az)++ is a relatively simple service that transfers messages between two different message buses. It was built for the purpose of providing users of WebSocket API services to have a quick and easy way to provide connectivity to their existing MQ bus systems without having to re-code to a WebSocket API. Effectively, it will listen to any message coming from the MQ bus and send it over to the WebSocket API, and vice-versa. While the service in this incarnation implements MQ to WebSockets, the code is modular so that the respective bus handling code can be swapped out for another bus, such as JMS or Kafka.\n\n\n### **Amazon Web Services-vpc-flowlogs-enricher**\n\n++[Amazon Web Services-vpc-flowlogs-enricher](https://aws-oss.beachgeek.co.uk/2aw)++ This repo contains a sample lambda function code that can be used in Kinesis Firehose stream to enrich VPC Flow Log record with additional metadata like resource tags for source and destination IP addresses and, VPC ID, Subnet ID, Interface ID, AZ for destination IP addresses. This data then can be used to identify flows for specific tags, or Source AZ to destination AZ traffic and many more scenarios.\n\n![image.png](https://dev-media.amazoncloud.cn/24c633b9b7b3403c8c3288027ba8162f_image.png)\n\n### **ec2-image-builder-send-approval-notifications-before-sharing-ami**\n\n++[ec2-image-builder-send-approval-notifications-before-sharing-ami](https://aws-oss.beachgeek.co.uk/2b0)++ You may be required to manually validate the Amazon Machine Image (AMI) built from an Amazon Elastic Compute Cloud (Amazon EC2) Image Builder pipeline before sharing this AMI to other Amazon Web Services accounts or to an Amazon Web Services organization. Currently, Image Builder provides an end-to-end pipeline that automatically shares AMIs after they’ve been built. This repo provides code and documentation to help you build a solution to enable approval notifications before AMIs are shared with other Amazon Web Services accounts.\n\n![image.png](https://dev-media.amazoncloud.cn/a85429e96412410aa1f03121b74aa9b5_image.png)\n\n\n### **deploy-nth-to-eks**\n\n++[deploy-nth-to-eks](https://aws-oss.beachgeek.co.uk/2b1)++ Amazon Web Services Node Termination Handler (nth) ensures that the Kubernetes control plane responds appropriately to events that can cause your EC2 instance to become unavailable, such as EC2 maintenance events, EC2 Spot interruptions, ASG Scale-In, ASG AZ Rebalance, and EC2 Instance Termination via the API or Console. If not handled, your application code may not stop gracefully, take longer to recover full availability, or accidentally schedule work to nodes that are going down.The Amazon Web Services-node-termination-handler (NTH) can operate in two different modes: Instance Metadata Service (IMDS) or the Queue Processor. The Amazon Web Services-node-termination-handler Instance Metadata Service Monitor will run a small pod on each host to perform monitoring of IMDS paths like /spot or /events and react accordingly to drain and/or cordon the corresponding node. The Amazon Web Services-node-termination-handler Queue Processor will monitor an SQS queue of events from [Amazon EventBridge](https://aws.amazon.com/cn/eventbridge/?trk=cndc-detail) for ASG lifecycle events, EC2 status change events, Spot Interruption Termination Notice events, and Spot Rebalance Recommendation events. When NTH detects an instance is going down, we use the Kubernetes API to cordon the node to ensure no new work is scheduled there, then drain it, removing any existing work. The termination handler Queue Processor requires Amazon Web Services IAM permissions to monitor and manage the SQS queue and to query the EC2 API. This pattern will automate the deployment of Node Termination Handler using Queue Processor through CICD Pipeline.\n\n![image.png](https://dev-media.amazoncloud.cn/a7e4b3b0893641989dd7c95d4da3f6b5_image.png)\n\n## **Demos, Samples, Solutions and Workshops**\n### **custom-provider-with-terraform-plugin-framework**\n\n++[custom-provider-with-terraform-plugin-framework](https://aws-oss.beachgeek.co.uk/2b5)++ This repository contains a complete implementation of a custom provider built using HashiCorp's latest SDK called Terraform plugin framework. It is used to teach, educate, and show the internals of a provider built with the latest SDK from HashiCorp. Even if you are not looking to learn how to build custom providers, you may dial your troubleshooting skills to an expert level if you learn how one works behind the scenes. Plus, this provider is lots of fun to play with. The provider is called buildonaws and it allows you to maintain characters from comic books such as heros, super-heros, and villains.\n\n\n### **mastodon-on-Amazon Web Services**\n\n++[mastodon-on-Amazon Web Services]()++ Andreas Wittig and Michael Wittig share details of how you can host your own Mastodon instance on Amazon Web Services. They have also put together this blog post, ++[Mastodon on Amazon Web Services: Host your own instance](https://aws-oss.beachgeek.co.uk/2b8)++ which you can read for more info.\n\n![image.png](https://dev-media.amazoncloud.cn/50f10162bd054c1d9e3790899b7b7bd6_image.png)\n\n\n### **mastodon-Amazon Web Services-architecture**\n\n++[mastodon-Amazon Web Services-architecture](https://aws-oss.beachgeek.co.uk/2b6)++ this repo provides details on how snapp.social Mastadon instance is being run on Amazon Web Services, and as more and more people explore whether this options is right for them, take a look and see how they have architected and deployed this on Amazon Web Services.\n\n\n### **amazon-ecs-fargate-cdk-v2-cicd**\n\n++[amazon-ecs-fargate-cdk-v2-cicd](https://aws-oss.beachgeek.co.uk/2b2)++ This project builds a complete sample containerised Flask application publicly available on Amazon Web Services, using Fargate, ECS, CodeBuild, and CodePipline to produce a fully functional pipeline to continuously roll out changes to your new app.\n\n### **ROSConDemo**\n\n++[ROSConDemo](https://aws-oss.beachgeek.co.uk/2b4)++ this repo contains code for a working robotic fruit picking demo project for O3DE with ROS 2 Gem.\n\n![image.png](https://dev-media.amazoncloud.cn/c0058d1eb3284ba78d74917b693814de_image.png)\n\n\n### **o3de-demo-project**\n\no3de-demo-projectThis project demonstrates how ROS2 Gem for O3DE can be used with a scene (The Loft project) and ROS 2 navigation stack.\n\n\n![image.png](https://dev-media.amazoncloud.cn/7a2281a62e854856a53d55d2fde0f744_image.png)\n\n## **Amazon Web Services and Community blog posts**\n### **Finch**\n\n\nPhil Estes and Chris Short put together this post,++[ Introducing Finch: An Open Source Client for Container Development](https://aws-oss.beachgeek.co.uk/2aj)++ to announce a new open source project, Finch. Finch is a new command line client for building, running, and publishing Linux containers. It provides for simple installation of a native macOS client, along with a curated set of de facto standard open source components including Lima, nerdctl, containerd, and BuildKit. With Finch, you can create and run containers locally, and build and publish Open Container Initiative (OCI) container images. One thing that really stands out from this post is this quote:\n\n\nRather than iterating in private and releasing a finished project, we feel open source is most successful when diverse voices come to the party. We have plans for features and innovations, but opening the project this early will lead to a more robust and useful solution for all. We are happy to address issues, and are ready to accept pull requests.\n\nSo check out this post and get hands on with Finch.\n\n### **Apache Hudi**\n\nHot off the heels of featuring Apache Hudi in the ++[last Build on Open Source show]()++, we have Suthan Phillips and Dylan Qu who have put together B ++[uild your Apache Hudi data lake on Amazon Web Services using Amazon EMR – Part 1](https://aws-oss.beachgeek.co.uk/2as)++, where they cover best practices when building Hudi data lakes on Amazon Web Services using [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail)\n\n![image.png](https://dev-media.amazoncloud.cn/e84938db7c26494180a2000b4a8d4cc1_image.png)\n\n### **Apache Kafka**\n\nWith so many choices for Builders on how they deploy Apache Kafka, how do you decide which is the right option for you? Well, Amazon Web Services Community Builder John Preston is here to provide his thoughts on this in his blog post,++[ Amazon Web Services MSK, Confluent Cloud, Aiven. How to chose your managed Kafka service provider](https://aws-oss.beachgeek.co.uk/2ba)++? After you have read the post, share your thoughts with John in the comments.\n\n### **Apache ShardingSphere**\n\nApache ShardingSphere follows Database Plus - our community's guiding development concept for creating a complete ecosystem that allows you to transform any database into a distributed database system, and easily enhance it with sharding, elastic scaling, data encryption features & more. It focuses on repurposing existing databases, by placing a standardized upper layer above existing and fragmented databases, rather than creating a new database. You can read more about this project in the post, ++[ShardingSphere-on-Cloud & Pisanix replace Sidecar for a true cloud-native experience](https://aws-oss.beachgeek.co.uk/2ah)++ and find out more about ++[ShardingSphere-on-Cloud](https://aws-oss.beachgeek.co.uk/2ai)++ that shows you how you can deploy ShardingSphere in a Kubernetes environment on Amazon Web Services.\n\n\n![image.png](https://dev-media.amazoncloud.cn/262203eeadd74b12a4cd5058e16edb04_image.png)\n\n### **MySQL and MariaDB**\n\nIn the post ++[Security best practices for Amazon RDS for MySQL and MariaDB instances](https://aws-oss.beachgeek.co.uk/2ap)++, Chelluru Vidyadhar discuss the different best practices you can follow in order to run [Amazon RDS](https://aws.amazon.com/cn/rds/?trk=cndc-detail) for MySQL and [Amazon RDS](https://aws.amazon.com/cn/rds/?trk=cndc-detail) for MariaDB databases securely. Chelluru look at the current good practices at network, database instance, and DB engine (MySQL and MariaDB) levels.\n\n\nSticking with MariaDB, Vijay Karumajji and Justin Lim have put together ++[Increase write throughput on Amazon RDS for MariaDB using the MyRocks storage engine](https://aws-oss.beachgeek.co.uk/2aq)++, where they explore the newly launched MyRocks storage engine architecture in [Amazon RDS](https://aws.amazon.com/cn/rds/?trk=cndc-detail) for MariaDB 10.6. They start by covering MyRocks and its architecture, use cases of MyRocks, and demonstrate our benchmarking results, so you can determine if the MyRocks storage engine can help you get increased performance for your workload.\n\n\n![image.png](https://dev-media.amazoncloud.cn/9a49f7178b3e408589c0351c5a3f30de_image.png)\n\n\n### **PostgreSQL**\n\npgBadger is an open source tool for identifying both slow-running and frequently running queries in your PostgreSQL applications, and helping guide you on how to improve their performance. In the blog post, ++[A serverless architecture for analyzing PostgreSQL logs with pgBadger](https://aws-oss.beachgeek.co.uk/2au)++ Krishna Sarabu, Chirag Dave, and Mark Townsend walk you through a solution design that enables the analysis of PostgreSQL database logs using no persistent compute resources. This allows you to use pgBadger without having to worry about provisioning, securing, and maintaining additional compute and storage resources. [hands on]\n\n![image.png](https://dev-media.amazoncloud.cn/8fd4363dcc06497c8506d04f89a89ae0_image.png)\n\n### **Kubernetes**\n\n We had a plethora of Kubernetes content in the run up to re:Invent, so here is a round up of the ones I found most interesting.\n\n- ++[How to detect security issues in Amazon EKS clusters using Amazon GuardDuty – Part 1](https://aws-oss.beachgeek.co.uk/2al)++ walks through the events leading up to a real-world security issue that occurred due to EKS cluster misconfiguration, and then looks at how those misconfigurations could be used by a malicious actor, and how [Amazon GuardDuty](https://aws.amazon.com/cn/guardduty/?trk=cndc-detail) monitors and identifies suspicious activity throughout the EKS security event\n- ++[Persistent storage for Kubernetes](https://aws-oss.beachgeek.co.uk/2am)++ the first of a two part post that covers the concepts of persistent storage for Kubernetes and how you can apply those concepts for a basic workload\n\n- ![image.png](https://dev-media.amazoncloud.cn/e41d523f2b7a4d1097399e1cdbf2c1b0_image.png)\n\n- ++[Exposing Kubernetes Applications, Part 3: NGINX Ingress Controller](https://aws-oss.beachgeek.co.uk/2an)++ the third in a series looking at ways to expose applications running in a Kubernetes cluster for external access, this post covers using an open-source implementation of an Ingress controller: NGINX Ingress Controller, exploring some of its features and the ways it differs from its Amazon Web Services Load Balancer Controller\n\n\n![image.png](https://dev-media.amazoncloud.cn/5e056fa4808f4973a1240c597a3eb6c8_image.png)\n\n- ++[Machine Learning with Kubeflow on Amazon EKS with Amazon EFS](https://aws-oss.beachgeek.co.uk/2ao)++ walks through how you can use Kubeflow on [Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail) to implement model parallelism and use [Amazon EFS](https://aws.amazon.com/cn/efs/?trk=cndc-detail) as persistent storage to share datasets [hands on]\n\n![image.png](https://dev-media.amazoncloud.cn/7bf61c09d27d4e01b6dda6f7e9a605c7_image.png)\n\n\n### **Other posts and quick reads**\n\n- ++[Using Authorizer with DynamoDB and EKS](https://aws-oss.beachgeek.co.uk/2bc)++ shows how to use the open source ++[Authorizer](https://aws-oss.beachgeek.co.uk/2bd)++ project to provide an auth solution when working with [Amazon DynamoDB](https://aws.amazon.com/cn/dynamodb/?trk=cndc-detail)\n\n![image.png](https://dev-media.amazoncloud.cn/a5d54ecb67e84806b8ac869749607854_image.png)\n\n\n- ++[Launch self-supervised training jobs in the cloud with Amazon Web Services ParallelCluster](https://aws-oss.beachgeek.co.uk/2a7)++ describes the process for creating a High Performance Compute (HPC) cluster that will launch large, self-supervised training jobs, primarily leveraging two technologies: Amazon Web Services ParallelCluster and the Vision Self-Supervised Learning (VISSL) library\n\n![image.png](https://dev-media.amazoncloud.cn/8fcf93537fb242d5a7a49b6cdfc03766_image.png)\n\n- ++[Getting started with JavaScript resolvers in Amazon Web Services AppSync GraphQL APIs](https://aws-oss.beachgeek.co.uk/2ab)++ takes a look at how you can now use JavaScript to write your AppSync pipeline resolver code and AppSync function code, as well as the existing Velocity Template Language (VTL)\n\n![image.png](https://dev-media.amazoncloud.cn/4965034b6c654668a4a65c09602c25f0_image.png)\n\n- ++[Easy and accurate forecasting with AutoGluon-TimeSeries ](https://aws-oss.beachgeek.co.uk/2ad)++ showcases AutoGluon-TimeSeries’s ease of use in quickly building a powerful forecaster [hands on]\n\n![image.png](https://dev-media.amazoncloud.cn/241deb7cb24a4b6ebad9d560d0d13116_image.png)\n\n- ++[Managing images in your NextJS app with Amazon Web Services AppSync and the Amazon Web Services CDK](https://aws-oss.beachgeek.co.uk/2at)++ shows how combining the Amazon Web Services CDK with the Amplify JavaScript library, they provide the flexibility needed for teams to scale independently and confidently, while still taking advantage of modern tooling [hands on]\n\n![image.png](https://dev-media.amazoncloud.cn/60f2acf47c014719b7e25cfa17096a3b_image.png)\n\n### **Case Studies**\n\n- ++[Announcing the winners of the inaugural Future of Government Awards: Celebrating digital transformation initiatives around the world](https://aws-oss.beachgeek.co.uk/2a3)++ includes details of the winners of Open Source Creation of the Year Award and Open Source Adaptation of the Year Award.\n\n- ++[DENT, the Open Source Network Operating System for Distributed Edge, Now Powers Amazon Web Services Just Walk Out Technology](https://aws-oss.beachgeek.co.uk/2bb)++ a look at how this networking open source project is being used by Amazon Web Services in it's [Just Walk Out](https://aws.amazon.com/cn/just-walk-out/?trk=cndc-detail) Technology\n\n### **Quick updates**\n### **Apache Iceberg**\n\n\n[Amazon Athena](https: //aws.amazon.com/cn/athena/?trk=cndc-detail) has added SQL commands and file formats that simplify the storage, transformation, and maintenance of data stored in Apache Iceberg tables. These new capabilities enable data engineers and analysts to combine more of the familiar conveniences of SQL with the transactional properties of Iceberg to enable efficient and robust analytics use cases.\n\nToday's launch adds CREATE TABLE AS SELECT (CTAS), MERGE, and VACUUM commands that streamline the lifecycle management of your Iceberg data: CTAS makes it fast and efficient to create tables, MERGE synchronises tables in one step to simplify your data preparation and update tasks, and VACUUM helps you manage storage footprint and delete records to meet regulatory requirements such as GDPR. We've also added support for AVRO and ORC so you can create Iceberg tables with a broader set of file formats. Lastly, you can now simplify access to Iceberg-managed data by using Views to hide complex joins, aggregations, and data types.\n\n### **Apache Airflow**\n\n[Amazon Managed Workflows for Apache Airflow](https://aws.amazon.com/cn/managed-workflows-for-apache-airflow/?trk=cndc-detail) (MWAA) now provides [Amazon CloudWatch](https://aws.amazon.com/cn/cloudwatch/?trk=cndc-detail) metrics for container, database, and queue utilisation. Amazon MWAA is a managed service for Apache Airflow that lets you use the same familiar Apache Airflow platform as you do today to orchestrate your workflows and enjoy improved scalability, availability, and security without the operational burden of having to manage the underlying infrastructure. With these additional metrics, customers have improved visibility into their Amazon MWAA performance to help them debug workloads and appropriately size their environments.\n\nCheck out the excellent post ++[Introducing container, database, and queue utilization metrics for the Amazon MWAA environment](https://aws-oss.beachgeek.co.uk/2ae)++, where Uma Ramadoss dives deep and shares details about the new metrics published for Amazon MWAA environment, build a sample application with a pre-built workflow, and explore the metrics using CloudWatch dashboard. [hands on]\n\n![image.png](https://dev-media.amazoncloud.cn/4339b25be09e49b49e343bf713a17e29_image.png)\n\n### **Apache Flink**\n\nApache Flink is a popular open source framework for stateful computations over data streams. It allows you to formulate queries that are continuously evaluated in near real time against an incoming stream of events. There were a couple of announcements this week featuring this open source project.\n\nFirst up was news that [Amazon Kinesis](https://aws.amazon.com/cn/kinesis/?trk=cndc-detail) Data Analytics for Apache Flink now supports Apache Flink version 1.15. This new version includes improvements to Flink's exactly-once processing semantics, Kinesis Data Streams and Kinesis Data Firehose connectors, Python User Defined Functions, Flink SQL, and more. The release also includes an Amazon Web Services-contributed capability, a new Async-Sink framework which simplifies the creation of custom sinks to deliver processed data. Read more about how we contributed to this release by checking out the post, ++[Making it Easier to Build Connectors with Apache Flink: Introducing the Async Sink](https://aws-oss.beachgeek.co.uk/2af)++ where Zichen Liu, Steffen Hausmann, and Ahmed Hamdy talk about a feature of Apache Flink, Async Sinks, and how the Async Sink works, how you can build a new sink based on the Async Sink, and discuss our plans to continue our contributions to Apache Flink.\n\n[Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) customers can now use Amazon Web Services Glue Data Catalog from their streaming and batch SQL workflows on Flink. The Amazon Web Services Glue Data Catalog is an Apache Hive metastore-compatible catalog. You can configure your Flink jobs on [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) to use the Data Catalog as an external Apache Hive metastore. With this release, You can then directly run Flink SQL queries against the tables stored in the Data Catalog.\n\nFlink supports on-cluster Hive metastore as the out-of-box persistent catalog. This means that metadata had to be recreated when clusters were shutdown and it was hard for multiple clusters to share the same metadata information. Starting with [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) 6.9, your Flink jobs on [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) can manage Flink’s metadata in Amazon Web Services Glue Data Catalog. You can use a persistent and fully managed Glue Data Catalog as a centralised repository. Each Data Catalog is a highly scalable collection of tables organised into databases.\n\nThe Amazon Web Services Glue Data Catalog provides a uniform repository where disparate systems can store and find metadata to keep track of data in data silos. You can then query the metadata and transform that data in a consistent manner across a wide variety of applications. With support for Amazon Web Services Glue Data Catalog, you can use Apache Flink on [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) for unified BATCH and STREAM processing of Apache Hive Tables or metadata of any Flink tablesource such as Iceberg, Kinesis or Kafka. You can specify the Amazon Web Services Glue Data Catalog as the metastore for Flink using the Amazon Web Services Management Console, Amazon Web Services CLI, or [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) API.\n\n### **[Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail)**\n\nA couple of [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) on [Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail) updates this week.\n\nThe ACK controller for [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) on Elastic Kubernetes Service (EKS) has graduated to generally available status. Using the ACK controller for EMR on EKS, you can declaratively define and manage EMR on EKS resources such as virtual clusters and job runs as Kubernetes custom resources. This lets you manage these resources directly using Kubernetes-native tools such as ‘kubectl’. EMR on EKS is a deployment option for EMR that allows you to run open-source big data frameworks on EKS clusters. You can consolidate analytical workloads with your Kubernetes-based applications on the same [Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail) cluster to improve resource utilisation and simplify infrastructure management and tooling. ACK is a collection of Kubernetes custom resource definitions (CRDs) and custom controllers working together to extend the Kubernetes API and manage Amazon Web Services resources on your behalf.\n\nFollowing that we had the announcement of support for configuring Spark properties within EMR Studio Jupyter Notebook sessions for interactive Spark workloads. [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) on EKS enables customers to efficiently run open-source big data frameworks such as Apache Spark on [Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail). [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) on EKS customers setup and use a managed endpoint (available in preview) to run interactive workloads using integrated development environments (IDEs) such as EMR Studio. Data scientists and engineers use EMR Studio Jupyter notebooks with EMR on EKS to develop, visualise and debug applications written in Python, PySpark, or Scala. With this release, customers can now customise their Spark settings, such as driver and executor CPU/memory, number of executors, and package dependencies, within their notebook session to handle different computational workloads or different amounts of data, using a single managed endpoint.\n\n### **Trino**\n\nTrino is an open source SQL query engine used to run interactive analytics on data stored in [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail). Announced last week was news that [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail) improves performance of queries running on Trino by up to 9x when using [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail) Select. With S3 Select, you “push down” the computational work to filter your S3 data instead of returning the entire object. By using Trino with S3 Select, you retrieve only a subset of data from an object, reducing the amount of data returned and accelerating query performance.\n\nAmazon Web Services’s upstream contribution to open source Trino, you can use Trino with S3 Select to improve your query performance. S3 Select offloads the heavy lifting of filtering and accessing data inside objects to [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail), which reduces the amount of data that has to be transferred and processed by Trino. For example, if you have a data lake built on [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail) and use Trino today, you can use S3 Select’s filtering capability to quickly and easily run interactive ad-hoc queries.\n\n\nYou can explore this in more detail by checking out this blog post, ++[Run queries up to 9x faster using Trino with Amazon S3 Select on Amazon EMR](https://aws-oss.beachgeek.co.uk/2ar)++ where Boni Bruno and Eric Henderson look at the performance benchmarks on Trino release 397 with S3 Select using TPC-DS-like benchmark queries at 3 TB scale.\n\n![image.png](https://dev-media.amazoncloud.cn/d6c98e652d92422e9447518a9f41f941_image.png)\n\n### **Amazon Web Services Amplify**\n\nAmplify DataStore provides frontend app developers the ability to build real-time apps with offline capabilities by storing data on-device (web browser or mobile device) and automatically synchronizing data to the cloud and across devices on an internet connection. Launched this week was the release of custom primary keys, also known as custom identifiers, for Amplify DataStore to provide additional flexibility for your data models. You can dive deeper into this update by reading along in the post, ++[New: Announcing custom primary key support for Amazon Web Services Amplify DataStore](https://aws-oss.beachgeek.co.uk/2a1)++\n\nWe had another Amplify DataStore post that looks at a number of other enhancements with Amplify DataStore that were released this week, that make working with relational data easier: lazy loading, nested query predicates, and type enhancements. To find out more about these new enhancements, check out ++[NEW: Lazy loading & nested query predicates for Amazon Web Services Amplify DataStore](https://aws-oss.beachgeek.co.uk/2aa)++ [hands on]\n\nAlso announced this week was the release of version 5.0.0 of the Amplify JavaScript library. This release is jam-packed with highly requested features, in addition to under the hood improvements to enhance stability and usability of the JavaScript library. Check out the post, ++[Announcing Amazon Web Services Amplify JavaScript library version 5](https://aws-oss.beachgeek.co.uk/2a2)++ which contains links to the GitHub repo.\n\nThe Amplify team have been super busy, as they also announced a developer preview to expand Flutter support to Web and Desktop for the API, Analytics, and Storage use cases. Developers can now build cross-platform Flutter apps with Amplify that target iOS, Android, Web, and Desktop (macOS, Windows, Linux) using a single codebase. Combined with the Authentication preview that was previously released, developers can now build cross-platform Flutter applications that include REST API or GraphQL API to interact with backend data, analytics to understand user behaviour, and storage for saving and retrieving files and media. This developer preview version was written fully in Dart, allowing developers to deploy their apps to all target platforms currently supported by Flutter. Amplify Flutter is designed to provide developers with consistent behaviour, regardless of the target platform. With these feature sets now available on Web and Desktop, Flutter developers can build experiences that target the platforms that matter most to their customers. Check out the post, ++[Announcing Flutter Web and Desktop support for Amazon Web Services Amplify Storage, Analytics and API libraries](https://aws-oss.beachgeek.co.uk/2ag)++, to find out more about this launch and how to use Amazon Web Services Amplify GraphQL API and Storage libraries by creating a grocery list application with Flutter that targets iOS, Android, Web, and Desktop. [hands on]\n\n![ezgif5a59cbd94fc.gif](https://dev-media.amazoncloud.cn/e00d98f5f6b145cb98727d40532bd5a9_ezgif-5-a59cbd94fc.gif)\n\nFinally, we also announced that Amazon Web Services Amplify is announcing support for GraphQL APIs without Conflict Resolution enabled! With this launch, it’s easier than ever to use custom mutations and queries, without needing to manage the underlying conflict resolution protocol. You can still model your data with the same easy-to-use graphical interface. And, we are also bringing improved GraphQL API testing to Studio through the open-source tool, GraphiQL.\n\nFind out more by reading the post, ++[Announcing new GraphQL API features in Amplify Studio](https://aws-oss.beachgeek.co.uk/2a4)++\n\n*Bonus Content*\n\nThere has been plenty of Amazon Web Services Amplify content posted this week, so why not check out some of these posts:\n\n- ++[NEW: Build React forms for any API in minutes with Amazon Web Services Amplify Studio (no Amazon Web Services Account required)](https://aws-oss.beachgeek.co.uk/2a5)++ looks at Amplify Studio form builder, the new way to build React form components for any API [hands on]\n\n![image.png](https://dev-media.amazoncloud.cn/da5cb8f3583e4526a0ec04e81c24b99e_image.png)\n\n- ++[Text to Speech on Android Using Amazon Web Services Amplify](https://aws-oss.beachgeek.co.uk/2a6)++ provides a nice example on how to use the Predictions category to implement text to speech in an Android app [hands on]\n\n### **Amazon Web Services Toolkits**\n\nAmazon Web Services Toolkits for JetBrains and VS Code released a faster code iteration experience for developing Amazon Web Services SAM applications. The Amazon Web Services Toolkits are open source plugins for JetBrains and VS Code IDEs that provide an integrated experience for developing Serverless applications, including assistance for getting started and local step-through debugging capabilities for Serverless applications. With today’s release, the Toolkits adds SAM CLI’s Lambda “sync” capabilities shipped as SAM Accelerate (check out the announcement). These new features in the Toolkits for JetBrains and VS Code provide customers with increased flexibility. Customers can either sync their entire Serverless application (i.e., infrastructure and the code), or sync just the code changes and skip Cloudformation deployments.\n\nRead more in the full blog post, ++[Faster iteration experience for Amazon Web Services SAM applications in the Amazon Web Services Toolkits for JetBrains and VS Code](https://aws-oss.beachgeek.co.uk/29y)++\n\n\n### **Grafana**\n\nLaunched this week was [Amazon Managed Grafana](https://aws.amazon.com/cn/grafana/?trk=cndc-detail)’s new alerting feature that allows customers to gain visibility into their Prometheus Alertmanager alerts from their Grafana workspace. Customers can continue to use classic Grafana Alerting in their [Amazon Managed Grafana](https://aws.amazon.com/cn/grafana/?trk=cndc-detail) workspaces if that experience better fits their needs. Customers using the [Amazon Managed Service for Prometheus](https://aws.amazon.com/cn/prometheus/?trk=cndc-detail) workspaces to collect Prometheus metrics utilise the fully managed Alert Manager and Ruler features in the service to configure alerting and recording rules. With this feature, they can visualise all their alert and recording rules configured in their [Amazon Managed Service for Prometheus](https://aws.amazon.com/cn/prometheus/?trk=cndc-detail) workspace.\n\nRead more in the hands on guide, ++[Announcing Prometheus Alertmanager rules in Amazon Managed Grafana](https://aws-oss.beachgeek.co.uk/29z)++\n\nAlso announced was [Amazon Managed Grafana](https://aws.amazon.com/cn/grafana/?trk=cndc-detail) support for connecting to data sources inside an Amazon Virtual Private Cloud ([Amazon VPC](https://aws.amazon.com/cn/vpc/?trk=cndc-detail)). Customers using [Amazon Managed Grafana](https://aws.amazon.com/cn/grafana/?trk=cndc-detail) have been asking for support to connect to data sources that reside in an [Amazon VPC](https://aws.amazon.com/cn/vpc/?trk=cndc-detail) and are not publicly accessible. Data in [Amazon OpenSearch Service](https://aws.amazon.com/cn/opensearch-service/?trk=cndc-detail) clusters, [Amazon RDS](https://aws.amazon.com/cn/rds/?trk=cndc-detail) instances, self-hosted data sources, and other data sensitive workloads often are only privately accessible. Customers have expressed the need to connect [Amazon Managed Grafana](https://aws.amazon.com/cn/grafana/?trk=cndc-detail) to these data sources securely while maintaining a strong security posture.\n\nRead more about this in the post, ++[Announcing Private VPC data source support for Amazon Managed Grafana](https://aws-oss.beachgeek.co.uk/2a0)++\n\n### **NodeJS**\n\nYou can now develop Amazon Web Services Lambda functions using the Node.js 18 runtime. This version is in active LTS status and considered ready for general use. When creating or updating functions, specify a runtime parameter value of nodejs18.x or use the appropriate container base image to use this new runtime. This runtime version is supported by functions running on either Arm-based Amazon Web Services Graviton2 processors or x86-based processors. Using the Graviton2 processor architecture option allows you to get up to 34% better price performance.\n\nRead the post ++[Node.js 18.x runtime now available in Amazon Web Services Lambda](https://aws-oss.beachgeek.co.uk/2a8)++, to find out more about the major changes available with the Node.js 18 runtime in Lambda. You should also check out ++[Why and how you should use Amazon Web Services SDK for JavaScript (v3) on Node.js 18](https://aws-oss.beachgeek.co.uk/2a9)++ as the Amazon Web Services SDK for JavaScript (v3) is included by default in Amazon Web Services Lambda Node.js 18 runtime.\n\n### **MariaDB**\n\nAmazon Relational Database Service ([Amazon RDS](https://aws.amazon.com/cn/rds/?trk=cndc-detail)) for MariaDB now supports MariaDB minor versions 10.6.11, 10.5.18, 10.4.27 and 10.3.37. We recommend that you upgrade to the latest minor versions to fix known security vulnerabilities in prior versions of MariaDB, and to benefit from the numerous bug fixes, performance improvements, and new functionality added by the MariaDB community.\n\n### **PostgreSQL**\n\nAmazon Relational Database Service ([Amazon RDS](https://aws.amazon.com/cn/rds/?trk=cndc-detail)) for PostgreSQL now supports PostgreSQL minor versions 14.5, 13.8, 12.12, 11.17, and 10.22. We recommend you upgrade to the latest minor version to fix known security vulnerabilities in prior versions of PostgreSQL, and to benefit from the bug fixes, performance improvements, and new functionality added by the PostgreSQL community. Please refer to the PostgreSQL community announcement for more details about the release. This release also includes support for [Amazon RDS](https://aws.amazon.com/cn/rds/?trk=cndc-detail) Multi-AZ with two readable standbys and updates for existing supported PostgreSQL extensions: PostGIS extension is updated to 3.1.7, pg_partman extension is updated to 4.6.2, and pgRouting extension is updated to 3.2.2. Please see the list of supported extensions in the [Amazon RDS](https://aws.amazon.com/cn/rds/?trk=cndc-detail) User Guide for specific versions.\n\n## **Videos of the week**\n### **Kubernetes and Amazon Web Services**\n\nIf you missed this, then it is well worth checking out the awesome Jay Pipes discuss Amazon Web Services' use of Kubernetes, as well as Amazon Web Services' contributions to the Kubernetes code base. The interview was recorded at KubeCon North America last month.\n\n<video src=\\"https://dev-media.amazoncloud.cn/227b20a3e20a4e15b13383c4939f6056_Kubernetes%20and%20Amazon%20Web%20Services.mp4\\" class=\\"manvaVedio\\" controls=\\"controls\\" style=\\"width:160px;height:160px\\"></video>\n\n### **OpenSearch**\n\nThe videos from OpenSearchCon that took place earlier this year are now available. You can see the ++[entire list here](https://aws-oss.beachgeek.co.uk/2be)++, and there are a number of great sessions covering a very broad range of topics. The one I spent time watching was this session from OpenSearch Core Codebase Nicholas Knize, OpenSearch Maintainer, Lucene Committer and PMC Member. If you are interested in contributing to OpenSearch and curious in how to get started, then this session will answer some of these questions and more by raising the hood and exploring the code base.\n\n<video src=\\"https://dev-media.amazoncloud.cn/ee20bae7a2bd4d869dfdde578b781b4e_Nick%20Knize%2C%20Getting%20Started%20with%20the%20OpenSearch%20Core%20Codebase%2C%20OpenSearchCon..mp4\\" class=\\"manvaVedio\\" controls=\\"controls\\" style=\\"width:160px;height:160px\\"></video>\n\n### **Kubeflow and MLFlow**\n\nJoin your hosts Antje Barth and Chris Fregley as they are joined by a number of guests to talk about some great open source projects such as Kubeflow, MLflow, datamesh.utils, and data.all\n\n<video src=\\"https://dev-media.amazoncloud.cn/2ec26da8d3c344f2a090b47ccb0d7668_Managed%20Kubeflow%20%2B%20Managed%20MLflow%20%2B%20Data%20Mesh%20architectures%20on%20AWS.mp4\\" class=\\"manvaVedio\\" controls=\\"controls\\" style=\\"width:160px;height:160px\\"></video>\n\n### **Build on Open Source**\n\nFor those unfamiliar with this show, Build on Open Source is where we go over this newsletter and then invite special guests to dive deep into their open source project. Expect plenty of code, demos and hopefully laughs. We have put together a playlist so that you can easily access all (seven) of the other episodes of the Build on Open Source show. ++[Build on Open Source playlist](https://aws-oss.beachgeek.co.uk/24u)++\n\n\n# **Events for your diary**\n### Apache Hudi Meetup - re:Invent\n### **November 28th - December 3rd, Las Vegas**\n\nApache Hudi is a data platform technology that helps build reliable and scalable data lakes. Hudi brings stream processing to big data, supercharging your data lakes, making them orders of magnitude more efficient.\n\nHudi is widely used by many companies like Uber, Walmart, Amazon.com, Robinhood, GE, Disney Hotstar, Alibaba, ByteDance that build transactional or streaming data lakes. Hudi also comes pre-built with [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) and is integrated with [Amazon Athena](https: //aws.amazon.com/cn/athena/?trk=cndc-detail), Amazon Web Services Glue as well as [Amazon Redshift](https://aws.amazon.com/cn/redshift/?trk=cndc-detail). It is also integrated in many other cloud providers such as Google cloud and Alibaba cloud.\n\nPlease join the Apache Hudi community for a Meetup hosted by Onehouse and the Apache Hudi community at the re:Invent site. Here are the different times and locations (local Vegas time):\n\n- Nov 28th [7:00 pm - 7:20 pm] Networking\n- Nov 28th [7:20 pm - 7:50 pm] Hudi 101 (Speaker TBA)\n- Nov 28th [7:50 pm - 8:20 pm] How Hudi supercharges your lake house architecture with streaming and historical data by Vinoth Chandar\n- Nov 28th [8:20 pm - 8:40 pm] Roadmap (Speaker TBA)\n- Nov 28th [8:40 pm - 9:00 pm] Open floor for Q&A\nIt will be hosted in Conference room “Chopin 2” at the Encore Hotel\n\n### **re:Invent**\n### **November 28th - December 3rd, Las Vegas**\n\nre:Invent is happening all this week, and there is plenty of great open source content for you, whether it is breakout sessions, chalk talks, open source vendors in the expo, and more.\n\nWe will be featuring open source projects in the Developer Lounge again, in the Amazon Web Services Modern Applications and Open Source Zone. We have published a schedule of the open source projects you can check out, so why not take a peek at ++[The Amazon Web Services Modern Applications and Open Source Zone: Learn, Play, and Relax at Amazon Web Services re:Invent 2022](https://aws-oss.beachgeek.co.uk/2ac)++ and come along. I will be there for a big chunk of time on Tuesday, Wednesday, and Thursday. If you have a good open source story to tell, or some SWAG to trade, I will be bringing our Build On Open Source challenge coins, so be sure to hunt me down!\n\nCheck out this handy way to look at all the amazing open source sessions, then check out this ++[dashboard](https://aws-oss.beachgeek.co.uk/252)++ [sign up required]. I would love to hear which ones you are excited about so please let me know in the comments or via Twitter. If you want to hear what my top three, must watch sessions, then this is what I would attend (sadly, as an Amazon Web Services employee I am not allowed to attend sessions)\n\n1. OPN306 Amazon Web Services Lambda Powertools: Lessons from the road to 10 million downloads - Heitor Lessa is going to deliver an amazing session on the journey from idea to one of the most loved and used open source tools for Amazon Web Services Lambda users\n2. BOA204 When security, safety, and urgency all matter: Handling Log4Shell - Cannot wait for this session from Abbey Fuller who will walk us through how we managed this incident\n3. OPN202 Maintaining the Amazon Web Services Amplify Framework in the open - Matt Auerbach and Ashish Nanda are going to share details on how Amplify engineering managers work with the OSS community to build open-source software\n\n\n## **OpenSearch**\n### Every other Tuesday, 3pm GMT\n\nThis regular meet-up is for anyone interested in OpenSearch & Open Distro. All skill levels are welcome and they cover and welcome talks on topics including: search, logging, log analytics, and data visualisation.\n\nSign up to the next session, ++[OpenSearch Community Meeting](https://aws-oss.beachgeek.co.uk/1az)++\n\n### **Stay in touch with open source at Amazon Web Services**\nI hope this summary has been useful. Remember to check out the ++[Open Source homepage](https://aws.amazon.com/opensource/?opensource-all.sort-by=item.additionalFields.startDate&opensource-all.sort-order=asc)++ to keep up to date with all our activity in open source by following us on ++[@AWSOpen](https://twitter.com/AWSOpen)++\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","render":"<h2><a id=\\"November_25th_2022__Instalment_137_0\\"></a><strong>November 25th, 2022 - Instalment #137</strong></h2>\\n<h3><a id=\\"Welcome_1\\"></a>Welcome</h3>\\n<p>Welcome to the Amazon Web Services open source newsletter, edition #137. As it is re:Invent next week, I will be publishing the newsletter early as I am heading out on Monday. I will be in Las Vegas talking with open source Builders, hanging out on the Open Source Kiosk in the Amazon Web Services Village, and doing some talks. If you are coming, I would love to meet some of you, so get in touch. I will also be taking a break for a week, so the next newsletter will be on December 12th.</p>\n<p>As always, this week we have more new projects for you to practice your four freedoms on, including a couple of projects for those who are looking to perhaps stand up their own Mastadon instances. “Amazon Web Services-vpc-flowlogs-enricher” is a project to help you add additional data into your VPC Flow logs, “Amazon Web Services-security-assessment-solution” a solution that uses some open source security tools that you can use to assess your Amazon Web Services accounts, “Amazon Web Services-backup-amplify-appsync” a tool for all Amazon Web Services Amplify users need to know about, “message-bus-bridge” is a tool to help you copy messages between message bus’, “monitor-serverless-datalake” keep on top of your data lakes with this solution, “ec2-image-builder-send-approval-notifications-before-sharing-ami” shows you how you can add a notification step in the AMI building workflow, “amazon-ecs-fargate-cdk-v2-cicd” is a nice demonstration on using Amazon Web Services CDKv2 with Flask, “deploy-nth-to-eks” a tool for Kubernetes admins, and a few more projects too!</p>\n<p>With the run up to re:Invent, the Amazon Web Services Amplify team have been on fire, and we have lots of content for Amazon Web Services Amplify users and fans. We also have content covering your favourite open source projects, including GraphQL, Grafana, Prometheus, MariaDB, PostgreSQL, Flutter, React, Apache Iceberg, Apache Airflow, Apache Flink, Apache ShardingSphere, AutoGluon, Amazon Web Services ParallelCluster, Kubeflow, NGINX, Finch, Amazon EMR, Trino, Apache Hudi, O3DE, Apache Kafka, OpenSearch, MLFlow, and more.</p>\n<p>Finally, with re:Invent upon us, make sure you check the events section for everything you need to know to make sure you do not miss the best open source sessions.</p>\n<h3><a id=\\"Amazon_Web_Services_Copilot__have_your_say_13\\"></a><strong>Amazon Web Services Copilot - have your say</strong></h3>\\n<p>The Amazon Web Services Copilot project has created a new design proposal for overriding Copilot abstracted resources using the Amazon Web Services Cloud Development Kit (CDK). The goal is to provide a “break the glass” mechanism to access and configure functionality that is not surfaced by Copilot manifests by leveraging the expressive power of a programming language. Have your say by heading over to <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2b9\\" target=\\"_blank\\">Extending Copilot with the CDK</a></ins> and joining the discussion.</p>\n<h3><a id=\\"Feedback_17\\"></a><strong>Feedback</strong></h3>\\n<p>Please let me know how we can improve this newsletter as well as how Amazon Web Services can better work with open source projects and technologies by completing <ins><a href=\\"https://eventbox.dev/survey/NUSZ91Z\\" target=\\"_blank\\"> this very short survey</a></ins> that will take you probably less than 30 seconds to complete. Thank you so much!</p>\n<h3><a id=\\"Celebrating_open_source_contributors_21\\"></a><strong>Celebrating open source contributors</strong></h3>\\n<p>The articles and projects shared in this newsletter are only possible thanks to the many contributors in open source. I would like to shout out and thank those folks who really do power open source and enable us all to learn and build on top of what they have created.</p>\n<p>So thank you to the following open source heroes: John Preston, Andreas Wittig, Michael Wittig, Uma Ramadoss, Boni Bruno, Eric Henderson, Chelluru Vidyadhar, Vijay Karumajji, Justin Lim, Krishna Sarabu, Chirag Dave, and Mark Townsend</p>\n<h3><a id=\\"Latest_open_source_projects_27\\"></a><strong>Latest open source projects</strong></h3>\\n<p><em>The great thing about open source projects is that you can review the source code. If you like the look of these projects, make sure you that take a look at the code, and if it is useful to you, get in touch with the maintainer to provide feedback, suggestions or even submit a contribution.</em></p>\\n<h3><a id=\\"Tools_31\\"></a><strong>Tools</strong></h3>\\n<h3><a id=\\"Amazon_Web_Servicessamclipipelineinittemplates_32\\"></a><strong>Amazon Web Services-sam-cli-pipeline-init-templates</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2av\\" target=\\"_blank\\">Amazon Web Services-sam-cli-pipeline-init-templates</a></ins> This repository contains the pipeline init templates used in the Amazon Web Services SAM CLI for sam pipeline commands. Customers can now incrementally add services to their repository and automate the creation and execution of pipelines for each new #serverless service. The template creates the necessary supporting infrastructure to keep track of commit history and changes that occur in your directories, so only the modified service pipeline is triggered. Get started by simply choosing option 2 when you initialise and bootstrap and new pipeline.</p>\n<h3><a id=\\"Amazon_Web_Servicessecurityassessmentsolution_36\\"></a><strong>Amazon Web Services-security-assessment-solution</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2ak\\" target=\\"_blank\\">Amazon Web Services-security-assessment-solution</a></ins> Cybersecurity remains a very important topic and point of concern for many CIOs, CISOs, and their customers. To meet these important concerns, Amazon Web Services has developed a primary set of services customers should use to aid in protecting their accounts. Amazon GuardDuty, Amazon Web Services Security Hub, Amazon Web Services Config, and Amazon Web Services Well-Architected reviews help customers maintain a strong security posture over their Amazon Web Services accounts. As more organizations deploy to the cloud, especially if they are doing so quickly, and they have not yet implemented the recommended Amazon Web Services Services, there may be a need to conduct a rapid security assessment of the cloud environment. With that in mind, we have worked to develop an inexpensive, easy to deploy, secure, and fast solution to provide our customers two (2) security assessment reports. These security assessments are from the open source projects “Prowler” and “ScoutSuite.” Each of these projects conduct an assessment based on Amazon Web Services best practices and can help quickly identify any potential risk areas in a customer’s deployed environment.</p>\n<h3><a id=\\"Amazon_Web_Servicesbackupamplifyappsync_40\\"></a><strong>Amazon Web Services-backup-amplify-appsync</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2ax\\" target=\\"_blank\\">Amazon Web Services-backup-amplify-appsync</a></ins> Amazon Web Services Amplify makes it easy to build full stack front end UI apps with backends and authentication. Amazon Web Services AppSync adds serverless GraphQL and DynamoDB tables to your application with no code. This project guides you on how to include the infrastructure as code to add Amazon Web Services Backup to an Amplify and AppSync application using to manage snapshots for your applications DynamoDB tables.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/7c31f278f6df416399c9776ad65a1534_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"monitorserverlessdatalake_46\\"></a><strong>monitor-serverless-datalake</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2ay\\" target=\\"_blank\\">monitor-serverless-datalake</a></ins> This repository serves as a launch pad for monitoring serverless data lakes in Amazon Web Services. The objective is to provide a plug and play mechanism for monitoring enterprise scale data lakes. Data lakes starts small and rapidly explodes with adoption. With growing adoption, the data pipelines also grows in number and complexity. It is pivotal to ensure that the data pipeline executes as per SLA and failures be mitigated. The solution provides mechanisms for the following, 1. Capture state changes across all tasks in the data lake 2. Quickly notify operations of failures as they happen 3. Measure service reliability across data lake – to identify opportunities for performance optimisation</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/46d4dac545be489f8f9dc567dbaef997_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"messagebusbridge_52\\"></a><strong>message-bus-bridge</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2az\\" target=\\"_blank\\">message-bus-bridge</a></ins> is a relatively simple service that transfers messages between two different message buses. It was built for the purpose of providing users of WebSocket API services to have a quick and easy way to provide connectivity to their existing MQ bus systems without having to re-code to a WebSocket API. Effectively, it will listen to any message coming from the MQ bus and send it over to the WebSocket API, and vice-versa. While the service in this incarnation implements MQ to WebSockets, the code is modular so that the respective bus handling code can be swapped out for another bus, such as JMS or Kafka.</p>\n<h3><a id=\\"Amazon_Web_Servicesvpcflowlogsenricher_57\\"></a><strong>Amazon Web Services-vpc-flowlogs-enricher</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2aw\\" target=\\"_blank\\">Amazon Web Services-vpc-flowlogs-enricher</a></ins> This repo contains a sample lambda function code that can be used in Kinesis Firehose stream to enrich VPC Flow Log record with additional metadata like resource tags for source and destination IP addresses and, VPC ID, Subnet ID, Interface ID, AZ for destination IP addresses. This data then can be used to identify flows for specific tags, or Source AZ to destination AZ traffic and many more scenarios.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/24c633b9b7b3403c8c3288027ba8162f_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"ec2imagebuildersendapprovalnotificationsbeforesharingami_63\\"></a><strong>ec2-image-builder-send-approval-notifications-before-sharing-ami</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2b0\\" target=\\"_blank\\">ec2-image-builder-send-approval-notifications-before-sharing-ami</a></ins> You may be required to manually validate the Amazon Machine Image (AMI) built from an Amazon Elastic Compute Cloud (Amazon EC2) Image Builder pipeline before sharing this AMI to other Amazon Web Services accounts or to an Amazon Web Services organization. Currently, Image Builder provides an end-to-end pipeline that automatically shares AMIs after they’ve been built. This repo provides code and documentation to help you build a solution to enable approval notifications before AMIs are shared with other Amazon Web Services accounts.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/a85429e96412410aa1f03121b74aa9b5_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"deploynthtoeks_70\\"></a><strong>deploy-nth-to-eks</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2b1\\" target=\\"_blank\\">deploy-nth-to-eks</a></ins> Amazon Web Services Node Termination Handler (nth) ensures that the Kubernetes control plane responds appropriately to events that can cause your EC2 instance to become unavailable, such as EC2 maintenance events, EC2 Spot interruptions, ASG Scale-In, ASG AZ Rebalance, and EC2 Instance Termination via the API or Console. If not handled, your application code may not stop gracefully, take longer to recover full availability, or accidentally schedule work to nodes that are going down.The Amazon Web Services-node-termination-handler (NTH) can operate in two different modes: Instance Metadata Service (IMDS) or the Queue Processor. The Amazon Web Services-node-termination-handler Instance Metadata Service Monitor will run a small pod on each host to perform monitoring of IMDS paths like /spot or /events and react accordingly to drain and/or cordon the corresponding node. The Amazon Web Services-node-termination-handler Queue Processor will monitor an SQS queue of events from Amazon EventBridge for ASG lifecycle events, EC2 status change events, Spot Interruption Termination Notice events, and Spot Rebalance Recommendation events. When NTH detects an instance is going down, we use the Kubernetes API to cordon the node to ensure no new work is scheduled there, then drain it, removing any existing work. The termination handler Queue Processor requires Amazon Web Services IAM permissions to monitor and manage the SQS queue and to query the EC2 API. This pattern will automate the deployment of Node Termination Handler using Queue Processor through CICD Pipeline.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/a7e4b3b0893641989dd7c95d4da3f6b5_image.png\\" alt=\\"image.png\\" /></p>\n<h2><a id=\\"Demos_Samples_Solutions_and_Workshops_76\\"></a><strong>Demos, Samples, Solutions and Workshops</strong></h2>\\n<h3><a id=\\"customproviderwithterraformpluginframework_77\\"></a><strong>custom-provider-with-terraform-plugin-framework</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2b5\\" target=\\"_blank\\">custom-provider-with-terraform-plugin-framework</a></ins> This repository contains a complete implementation of a custom provider built using HashiCorp’s latest SDK called Terraform plugin framework. It is used to teach, educate, and show the internals of a provider built with the latest SDK from HashiCorp. Even if you are not looking to learn how to build custom providers, you may dial your troubleshooting skills to an expert level if you learn how one works behind the scenes. Plus, this provider is lots of fun to play with. The provider is called buildonaws and it allows you to maintain characters from comic books such as heros, super-heros, and villains.</p>\n<h3><a id=\\"mastodononAmazon_Web_Services_82\\"></a><strong>mastodon-on-Amazon Web Services</strong></h3>\\n<p><ins><a href=\\"\\" target=\\"_blank\\">mastodon-on-Amazon Web Services</a></ins> Andreas Wittig and Michael Wittig share details of how you can host your own Mastodon instance on Amazon Web Services. They have also put together this blog post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2b8\\" target=\\"_blank\\">Mastodon on Amazon Web Services: Host your own instance</a></ins> which you can read for more info.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/50f10162bd054c1d9e3790899b7b7bd6_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"mastodonAmazon_Web_Servicesarchitecture_89\\"></a><strong>mastodon-Amazon Web Services-architecture</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2b6\\" target=\\"_blank\\">mastodon-Amazon Web Services-architecture</a></ins> this repo provides details on how snapp.social Mastadon instance is being run on Amazon Web Services, and as more and more people explore whether this options is right for them, take a look and see how they have architected and deployed this on Amazon Web Services.</p>\n<h3><a id=\\"amazonecsfargatecdkv2cicd_94\\"></a><strong>amazon-ecs-fargate-cdk-v2-cicd</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2b2\\" target=\\"_blank\\">amazon-ecs-fargate-cdk-v2-cicd</a></ins> This project builds a complete sample containerised Flask application publicly available on Amazon Web Services, using Fargate, ECS, CodeBuild, and CodePipline to produce a fully functional pipeline to continuously roll out changes to your new app.</p>\n<h3><a id=\\"ROSConDemo_98\\"></a><strong>ROSConDemo</strong></h3>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2b4\\" target=\\"_blank\\">ROSConDemo</a></ins> this repo contains code for a working robotic fruit picking demo project for O3DE with ROS 2 Gem.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/c0058d1eb3284ba78d74917b693814de_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"o3dedemoproject_105\\"></a><strong>o3de-demo-project</strong></h3>\\n<p>o3de-demo-projectThis project demonstrates how ROS2 Gem for O3DE can be used with a scene (The Loft project) and ROS 2 navigation stack.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/7a2281a62e854856a53d55d2fde0f744_image.png\\" alt=\\"image.png\\" /></p>\n<h2><a id=\\"Amazon_Web_Services_and_Community_blog_posts_112\\"></a><strong>Amazon Web Services and Community blog posts</strong></h2>\\n<h3><a id=\\"Finch_113\\"></a><strong>Finch</strong></h3>\\n<p>Phil Estes and Chris Short put together this post,<ins><a href=\\"https://aws-oss.beachgeek.co.uk/2aj\\" target=\\"_blank\\"> Introducing Finch: An Open Source Client for Container Development</a></ins> to announce a new open source project, Finch. Finch is a new command line client for building, running, and publishing Linux containers. It provides for simple installation of a native macOS client, along with a curated set of de facto standard open source components including Lima, nerdctl, containerd, and BuildKit. With Finch, you can create and run containers locally, and build and publish Open Container Initiative (OCI) container images. One thing that really stands out from this post is this quote:</p>\n<p>Rather than iterating in private and releasing a finished project, we feel open source is most successful when diverse voices come to the party. We have plans for features and innovations, but opening the project this early will lead to a more robust and useful solution for all. We are happy to address issues, and are ready to accept pull requests.</p>\n<p>So check out this post and get hands on with Finch.</p>\n<h3><a id=\\"Apache_Hudi_123\\"></a><strong>Apache Hudi</strong></h3>\\n<p>Hot off the heels of featuring Apache Hudi in the <ins><a href=\\"\\" target=\\"_blank\\">last Build on Open Source show</a></ins>, we have Suthan Phillips and Dylan Qu who have put together B <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2as\\" target=\\"_blank\\">uild your Apache Hudi data lake on Amazon Web Services using Amazon EMR – Part 1</a></ins>, where they cover best practices when building Hudi data lakes on Amazon Web Services using Amazon EMR</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/e84938db7c26494180a2000b4a8d4cc1_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"Apache_Kafka_129\\"></a><strong>Apache Kafka</strong></h3>\\n<p>With so many choices for Builders on how they deploy Apache Kafka, how do you decide which is the right option for you? Well, Amazon Web Services Community Builder John Preston is here to provide his thoughts on this in his blog post,<ins><a href=\\"https://aws-oss.beachgeek.co.uk/2ba\\" target=\\"_blank\\"> Amazon Web Services MSK, Confluent Cloud, Aiven. How to chose your managed Kafka service provider</a></ins>? After you have read the post, share your thoughts with John in the comments.</p>\n<h3><a id=\\"Apache_ShardingSphere_133\\"></a><strong>Apache ShardingSphere</strong></h3>\\n<p>Apache ShardingSphere follows Database Plus - our community’s guiding development concept for creating a complete ecosystem that allows you to transform any database into a distributed database system, and easily enhance it with sharding, elastic scaling, data encryption features &amp; more. It focuses on repurposing existing databases, by placing a standardized upper layer above existing and fragmented databases, rather than creating a new database. You can read more about this project in the post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2ah\\" target=\\"_blank\\">ShardingSphere-on-Cloud &amp; Pisanix replace Sidecar for a true cloud-native experience</a></ins> and find out more about <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2ai\\" target=\\"_blank\\">ShardingSphere-on-Cloud</a></ins> that shows you how you can deploy ShardingSphere in a Kubernetes environment on Amazon Web Services.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/262203eeadd74b12a4cd5058e16edb04_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"MySQL_and_MariaDB_140\\"></a><strong>MySQL and MariaDB</strong></h3>\\n<p>In the post <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2ap\\" target=\\"_blank\\">Security best practices for Amazon RDS for MySQL and MariaDB instances</a></ins>, Chelluru Vidyadhar discuss the different best practices you can follow in order to run Amazon RDS for MySQL and Amazon RDS for MariaDB databases securely. Chelluru look at the current good practices at network, database instance, and DB engine (MySQL and MariaDB) levels.</p>\n<p>Sticking with MariaDB, Vijay Karumajji and Justin Lim have put together <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2aq\\" target=\\"_blank\\">Increase write throughput on Amazon RDS for MariaDB using the MyRocks storage engine</a></ins>, where they explore the newly launched MyRocks storage engine architecture in Amazon RDS for MariaDB 10.6. They start by covering MyRocks and its architecture, use cases of MyRocks, and demonstrate our benchmarking results, so you can determine if the MyRocks storage engine can help you get increased performance for your workload.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/9a49f7178b3e408589c0351c5a3f30de_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"PostgreSQL_151\\"></a><strong>PostgreSQL</strong></h3>\\n<p>pgBadger is an open source tool for identifying both slow-running and frequently running queries in your PostgreSQL applications, and helping guide you on how to improve their performance. In the blog post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2au\\" target=\\"_blank\\">A serverless architecture for analyzing PostgreSQL logs with pgBadger</a></ins> Krishna Sarabu, Chirag Dave, and Mark Townsend walk you through a solution design that enables the analysis of PostgreSQL database logs using no persistent compute resources. This allows you to use pgBadger without having to worry about provisioning, securing, and maintaining additional compute and storage resources. [hands on]</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/8fd4363dcc06497c8506d04f89a89ae0_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"Kubernetes_157\\"></a><strong>Kubernetes</strong></h3>\\n<p>We had a plethora of Kubernetes content in the run up to re:Invent, so here is a round up of the ones I found most interesting.</p>\n<ul>\\n<li>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2al\\" target=\\"_blank\\">How to detect security issues in Amazon EKS clusters using Amazon GuardDuty – Part 1</a></ins> walks through the events leading up to a real-world security issue that occurred due to EKS cluster misconfiguration, and then looks at how those misconfigurations could be used by a malicious actor, and how Amazon GuardDuty monitors and identifies suspicious activity throughout the EKS security event</p>\n</li>\\n<li>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2am\\" target=\\"_blank\\">Persistent storage for Kubernetes</a></ins> the first of a two part post that covers the concepts of persistent storage for Kubernetes and how you can apply those concepts for a basic workload</p>\n</li>\\n<li>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/e41d523f2b7a4d1097399e1cdbf2c1b0_image.png\\" alt=\\"image.png\\" /></p>\n</li>\\n<li>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2an\\" target=\\"_blank\\">Exposing Kubernetes Applications, Part 3: NGINX Ingress Controller</a></ins> the third in a series looking at ways to expose applications running in a Kubernetes cluster for external access, this post covers using an open-source implementation of an Ingress controller: NGINX Ingress Controller, exploring some of its features and the ways it differs from its Amazon Web Services Load Balancer Controller</p>\n</li>\\n</ul>\n<p><img src=\\"https://dev-media.amazoncloud.cn/5e056fa4808f4973a1240c597a3eb6c8_image.png\\" alt=\\"image.png\\" /></p>\n<ul>\\n<li><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2ao\\" target=\\"_blank\\">Machine Learning with Kubeflow on Amazon EKS with Amazon EFS</a></ins> walks through how you can use Kubeflow on Amazon EKS to implement model parallelism and use Amazon EFS as persistent storage to share datasets [hands on]</li>\n</ul>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/7bf61c09d27d4e01b6dda6f7e9a605c7_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"Other_posts_and_quick_reads_176\\"></a><strong>Other posts and quick reads</strong></h3>\\n<ul>\\n<li><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2bc\\" target=\\"_blank\\">Using Authorizer with DynamoDB and EKS</a></ins> shows how to use the open source <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2bd\\" target=\\"_blank\\">Authorizer</a></ins> project to provide an auth solution when working with Amazon DynamoDB</li>\n</ul>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/a5d54ecb67e84806b8ac869749607854_image.png\\" alt=\\"image.png\\" /></p>\n<ul>\\n<li><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2a7\\" target=\\"_blank\\">Launch self-supervised training jobs in the cloud with Amazon Web Services ParallelCluster</a></ins> describes the process for creating a High Performance Compute (HPC) cluster that will launch large, self-supervised training jobs, primarily leveraging two technologies: Amazon Web Services ParallelCluster and the Vision Self-Supervised Learning (VISSL) library</li>\n</ul>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/8fcf93537fb242d5a7a49b6cdfc03766_image.png\\" alt=\\"image.png\\" /></p>\n<ul>\\n<li><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2ab\\" target=\\"_blank\\">Getting started with JavaScript resolvers in Amazon Web Services AppSync GraphQL APIs</a></ins> takes a look at how you can now use JavaScript to write your AppSync pipeline resolver code and AppSync function code, as well as the existing Velocity Template Language (VTL)</li>\n</ul>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/4965034b6c654668a4a65c09602c25f0_image.png\\" alt=\\"image.png\\" /></p>\n<ul>\\n<li><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2ad\\" target=\\"_blank\\">Easy and accurate forecasting with AutoGluon-TimeSeries </a></ins> showcases AutoGluon-TimeSeries’s ease of use in quickly building a powerful forecaster [hands on]</li>\n</ul>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/241deb7cb24a4b6ebad9d560d0d13116_image.png\\" alt=\\"image.png\\" /></p>\n<ul>\\n<li><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2at\\" target=\\"_blank\\">Managing images in your NextJS app with Amazon Web Services AppSync and the Amazon Web Services CDK</a></ins> shows how combining the Amazon Web Services CDK with the Amplify JavaScript library, they provide the flexibility needed for teams to scale independently and confidently, while still taking advantage of modern tooling [hands on]</li>\n</ul>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/60f2acf47c014719b7e25cfa17096a3b_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"Case_Studies_199\\"></a><strong>Case Studies</strong></h3>\\n<ul>\\n<li>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2a3\\" target=\\"_blank\\">Announcing the winners of the inaugural Future of Government Awards: Celebrating digital transformation initiatives around the world</a></ins> includes details of the winners of Open Source Creation of the Year Award and Open Source Adaptation of the Year Award.</p>\n</li>\\n<li>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2bb\\" target=\\"_blank\\">DENT, the Open Source Network Operating System for Distributed Edge, Now Powers Amazon Web Services Just Walk Out Technology</a></ins> a look at how this networking open source project is being used by Amazon Web Services in it’s Just Walk Out Technology</p>\n</li>\\n</ul>\n<h3><a id=\\"Quick_updates_205\\"></a><strong>Quick updates</strong></h3>\\n<h3><a id=\\"Apache_Iceberg_206\\"></a><strong>Apache Iceberg</strong></h3>\\n<p>Amazon Athena has added SQL commands and file formats that simplify the storage, transformation, and maintenance of data stored in Apache Iceberg tables. These new capabilities enable data engineers and analysts to combine more of the familiar conveniences of SQL with the transactional properties of Iceberg to enable efficient and robust analytics use cases.</p>\n<p>Today’s launch adds CREATE TABLE AS SELECT (CTAS), MERGE, and VACUUM commands that streamline the lifecycle management of your Iceberg data: CTAS makes it fast and efficient to create tables, MERGE synchronises tables in one step to simplify your data preparation and update tasks, and VACUUM helps you manage storage footprint and delete records to meet regulatory requirements such as GDPR. We’ve also added support for AVRO and ORC so you can create Iceberg tables with a broader set of file formats. Lastly, you can now simplify access to Iceberg-managed data by using Views to hide complex joins, aggregations, and data types.</p>\n<h3><a id=\\"Apache_Airflow_213\\"></a><strong>Apache Airflow</strong></h3>\\n<p>Amazon Managed Workflows for Apache Airflow (MWAA) now provides Amazon CloudWatch metrics for container, database, and queue utilisation. Amazon MWAA is a managed service for Apache Airflow that lets you use the same familiar Apache Airflow platform as you do today to orchestrate your workflows and enjoy improved scalability, availability, and security without the operational burden of having to manage the underlying infrastructure. With these additional metrics, customers have improved visibility into their Amazon MWAA performance to help them debug workloads and appropriately size their environments.</p>\n<p>Check out the excellent post <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2ae\\" target=\\"_blank\\">Introducing container, database, and queue utilization metrics for the Amazon MWAA environment</a></ins>, where Uma Ramadoss dives deep and shares details about the new metrics published for Amazon MWAA environment, build a sample application with a pre-built workflow, and explore the metrics using CloudWatch dashboard. [hands on]</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/4339b25be09e49b49e343bf713a17e29_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"Apache_Flink_221\\"></a><strong>Apache Flink</strong></h3>\\n<p>Apache Flink is a popular open source framework for stateful computations over data streams. It allows you to formulate queries that are continuously evaluated in near real time against an incoming stream of events. There were a couple of announcements this week featuring this open source project.</p>\n<p>First up was news that Amazon Kinesis Data Analytics for Apache Flink now supports Apache Flink version 1.15. This new version includes improvements to Flink’s exactly-once processing semantics, Kinesis Data Streams and Kinesis Data Firehose connectors, Python User Defined Functions, Flink SQL, and more. The release also includes an Amazon Web Services-contributed capability, a new Async-Sink framework which simplifies the creation of custom sinks to deliver processed data. Read more about how we contributed to this release by checking out the post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2af\\" target=\\"_blank\\">Making it Easier to Build Connectors with Apache Flink: Introducing the Async Sink</a></ins> where Zichen Liu, Steffen Hausmann, and Ahmed Hamdy talk about a feature of Apache Flink, Async Sinks, and how the Async Sink works, how you can build a new sink based on the Async Sink, and discuss our plans to continue our contributions to Apache Flink.</p>\n<p>Amazon EMR customers can now use Amazon Web Services Glue Data Catalog from their streaming and batch SQL workflows on Flink. The Amazon Web Services Glue Data Catalog is an Apache Hive metastore-compatible catalog. You can configure your Flink jobs on Amazon EMR to use the Data Catalog as an external Apache Hive metastore. With this release, You can then directly run Flink SQL queries against the tables stored in the Data Catalog.</p>\n<p>Flink supports on-cluster Hive metastore as the out-of-box persistent catalog. This means that metadata had to be recreated when clusters were shutdown and it was hard for multiple clusters to share the same metadata information. Starting with Amazon EMR 6.9, your Flink jobs on Amazon EMR can manage Flink’s metadata in Amazon Web Services Glue Data Catalog. You can use a persistent and fully managed Glue Data Catalog as a centralised repository. Each Data Catalog is a highly scalable collection of tables organised into databases.</p>\n<p>The Amazon Web Services Glue Data Catalog provides a uniform repository where disparate systems can store and find metadata to keep track of data in data silos. You can then query the metadata and transform that data in a consistent manner across a wide variety of applications. With support for Amazon Web Services Glue Data Catalog, you can use Apache Flink on Amazon EMR for unified BATCH and STREAM processing of Apache Hive Tables or metadata of any Flink tablesource such as Iceberg, Kinesis or Kafka. You can specify the Amazon Web Services Glue Data Catalog as the metastore for Flink using the Amazon Web Services Management Console, Amazon Web Services CLI, or Amazon EMR API.</p>\n<h3><a id=\\"Amazon_EMR_233\\"></a><strong>Amazon EMR</strong></h3>\\n<p>A couple of Amazon EMR on Amazon EKS updates this week.</p>\n<p>The ACK controller for Amazon EMR on Elastic Kubernetes Service (EKS) has graduated to generally available status. Using the ACK controller for EMR on EKS, you can declaratively define and manage EMR on EKS resources such as virtual clusters and job runs as Kubernetes custom resources. This lets you manage these resources directly using Kubernetes-native tools such as ‘kubectl’. EMR on EKS is a deployment option for EMR that allows you to run open-source big data frameworks on EKS clusters. You can consolidate analytical workloads with your Kubernetes-based applications on the same Amazon EKS cluster to improve resource utilisation and simplify infrastructure management and tooling. ACK is a collection of Kubernetes custom resource definitions (CRDs) and custom controllers working together to extend the Kubernetes API and manage Amazon Web Services resources on your behalf.</p>\n<p>Following that we had the announcement of support for configuring Spark properties within EMR Studio Jupyter Notebook sessions for interactive Spark workloads. Amazon EMR on EKS enables customers to efficiently run open-source big data frameworks such as Apache Spark on Amazon EKS. Amazon EMR on EKS customers setup and use a managed endpoint (available in preview) to run interactive workloads using integrated development environments (IDEs) such as EMR Studio. Data scientists and engineers use EMR Studio Jupyter notebooks with EMR on EKS to develop, visualise and debug applications written in Python, PySpark, or Scala. With this release, customers can now customise their Spark settings, such as driver and executor CPU/memory, number of executors, and package dependencies, within their notebook session to handle different computational workloads or different amounts of data, using a single managed endpoint.</p>\n<h3><a id=\\"Trino_241\\"></a><strong>Trino</strong></h3>\\n<p>Trino is an open source SQL query engine used to run interactive analytics on data stored in Amazon S3. Announced last week was news that Amazon S3 improves performance of queries running on Trino by up to 9x when using Amazon S3 Select. With S3 Select, you “push down” the computational work to filter your S3 data instead of returning the entire object. By using Trino with S3 Select, you retrieve only a subset of data from an object, reducing the amount of data returned and accelerating query performance.</p>\n<p>Amazon Web Services’s upstream contribution to open source Trino, you can use Trino with S3 Select to improve your query performance. S3 Select offloads the heavy lifting of filtering and accessing data inside objects to Amazon S3, which reduces the amount of data that has to be transferred and processed by Trino. For example, if you have a data lake built on Amazon S3 and use Trino today, you can use S3 Select’s filtering capability to quickly and easily run interactive ad-hoc queries.</p>\n<p>You can explore this in more detail by checking out this blog post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2ar\\" target=\\"_blank\\">Run queries up to 9x faster using Trino with Amazon S3 Select on Amazon EMR</a></ins> where Boni Bruno and Eric Henderson look at the performance benchmarks on Trino release 397 with S3 Select using TPC-DS-like benchmark queries at 3 TB scale.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/d6c98e652d92422e9447518a9f41f941_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"Amazon_Web_Services_Amplify_252\\"></a><strong>Amazon Web Services Amplify</strong></h3>\\n<p>Amplify DataStore provides frontend app developers the ability to build real-time apps with offline capabilities by storing data on-device (web browser or mobile device) and automatically synchronizing data to the cloud and across devices on an internet connection. Launched this week was the release of custom primary keys, also known as custom identifiers, for Amplify DataStore to provide additional flexibility for your data models. You can dive deeper into this update by reading along in the post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2a1\\" target=\\"_blank\\">New: Announcing custom primary key support for Amazon Web Services Amplify DataStore</a></ins></p>\n<p>We had another Amplify DataStore post that looks at a number of other enhancements with Amplify DataStore that were released this week, that make working with relational data easier: lazy loading, nested query predicates, and type enhancements. To find out more about these new enhancements, check out <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2aa\\" target=\\"_blank\\">NEW: Lazy loading &amp; nested query predicates for Amazon Web Services Amplify DataStore</a></ins> [hands on]</p>\n<p>Also announced this week was the release of version 5.0.0 of the Amplify JavaScript library. This release is jam-packed with highly requested features, in addition to under the hood improvements to enhance stability and usability of the JavaScript library. Check out the post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2a2\\" target=\\"_blank\\">Announcing Amazon Web Services Amplify JavaScript library version 5</a></ins> which contains links to the GitHub repo.</p>\n<p>The Amplify team have been super busy, as they also announced a developer preview to expand Flutter support to Web and Desktop for the API, Analytics, and Storage use cases. Developers can now build cross-platform Flutter apps with Amplify that target iOS, Android, Web, and Desktop (macOS, Windows, Linux) using a single codebase. Combined with the Authentication preview that was previously released, developers can now build cross-platform Flutter applications that include REST API or GraphQL API to interact with backend data, analytics to understand user behaviour, and storage for saving and retrieving files and media. This developer preview version was written fully in Dart, allowing developers to deploy their apps to all target platforms currently supported by Flutter. Amplify Flutter is designed to provide developers with consistent behaviour, regardless of the target platform. With these feature sets now available on Web and Desktop, Flutter developers can build experiences that target the platforms that matter most to their customers. Check out the post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2ag\\" target=\\"_blank\\">Announcing Flutter Web and Desktop support for Amazon Web Services Amplify Storage, Analytics and API libraries</a></ins>, to find out more about this launch and how to use Amazon Web Services Amplify GraphQL API and Storage libraries by creating a grocery list application with Flutter that targets iOS, Android, Web, and Desktop. [hands on]</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/e00d98f5f6b145cb98727d40532bd5a9_ezgif-5-a59cbd94fc.gif\\" alt=\\"ezgif5a59cbd94fc.gif\\" /></p>\n<p>Finally, we also announced that Amazon Web Services Amplify is announcing support for GraphQL APIs without Conflict Resolution enabled! With this launch, it’s easier than ever to use custom mutations and queries, without needing to manage the underlying conflict resolution protocol. You can still model your data with the same easy-to-use graphical interface. And, we are also bringing improved GraphQL API testing to Studio through the open-source tool, GraphiQL.</p>\n<p>Find out more by reading the post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2a4\\" target=\\"_blank\\">Announcing new GraphQL API features in Amplify Studio</a></ins></p>\n<p><em>Bonus Content</em></p>\\n<p>There has been plenty of Amazon Web Services Amplify content posted this week, so why not check out some of these posts:</p>\n<ul>\\n<li><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2a5\\" target=\\"_blank\\">NEW: Build React forms for any API in minutes with Amazon Web Services Amplify Studio (no Amazon Web Services Account required)</a></ins> looks at Amplify Studio form builder, the new way to build React form components for any API [hands on]</li>\n</ul>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/da5cb8f3583e4526a0ec04e81c24b99e_image.png\\" alt=\\"image.png\\" /></p>\n<ul>\\n<li><ins><a href=\\"https://aws-oss.beachgeek.co.uk/2a6\\" target=\\"_blank\\">Text to Speech on Android Using Amazon Web Services Amplify</a></ins> provides a nice example on how to use the Predictions category to implement text to speech in an Android app [hands on]</li>\n</ul>\\n<h3><a id=\\"Amazon_Web_Services_Toolkits_278\\"></a><strong>Amazon Web Services Toolkits</strong></h3>\\n<p>Amazon Web Services Toolkits for JetBrains and VS Code released a faster code iteration experience for developing Amazon Web Services SAM applications. The Amazon Web Services Toolkits are open source plugins for JetBrains and VS Code IDEs that provide an integrated experience for developing Serverless applications, including assistance for getting started and local step-through debugging capabilities for Serverless applications. With today’s release, the Toolkits adds SAM CLI’s Lambda “sync” capabilities shipped as SAM Accelerate (check out the announcement). These new features in the Toolkits for JetBrains and VS Code provide customers with increased flexibility. Customers can either sync their entire Serverless application (i.e., infrastructure and the code), or sync just the code changes and skip Cloudformation deployments.</p>\n<p>Read more in the full blog post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/29y\\" target=\\"_blank\\">Faster iteration experience for Amazon Web Services SAM applications in the Amazon Web Services Toolkits for JetBrains and VS Code</a></ins></p>\n<h3><a id=\\"Grafana_285\\"></a><strong>Grafana</strong></h3>\\n<p>Launched this week was Amazon Managed Grafana’s new alerting feature that allows customers to gain visibility into their Prometheus Alertmanager alerts from their Grafana workspace. Customers can continue to use classic Grafana Alerting in their Amazon Managed Grafana workspaces if that experience better fits their needs. Customers using the Amazon Managed Service for Prometheus workspaces to collect Prometheus metrics utilise the fully managed Alert Manager and Ruler features in the service to configure alerting and recording rules. With this feature, they can visualise all their alert and recording rules configured in their Amazon Managed Service for Prometheus workspace.</p>\n<p>Read more in the hands on guide, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/29z\\" target=\\"_blank\\">Announcing Prometheus Alertmanager rules in Amazon Managed Grafana</a></ins></p>\n<p>Also announced was Amazon Managed Grafana support for connecting to data sources inside an Amazon Virtual Private Cloud (Amazon VPC). Customers using Amazon Managed Grafana have been asking for support to connect to data sources that reside in an Amazon VPC and are not publicly accessible. Data in Amazon OpenSearch Service clusters, Amazon RDS instances, self-hosted data sources, and other data sensitive workloads often are only privately accessible. Customers have expressed the need to connect Amazon Managed Grafana to these data sources securely while maintaining a strong security posture.</p>\n<p>Read more about this in the post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2a0\\" target=\\"_blank\\">Announcing Private VPC data source support for Amazon Managed Grafana</a></ins></p>\n<h3><a id=\\"NodeJS_295\\"></a><strong>NodeJS</strong></h3>\\n<p>You can now develop Amazon Web Services Lambda functions using the Node.js 18 runtime. This version is in active LTS status and considered ready for general use. When creating or updating functions, specify a runtime parameter value of nodejs18.x or use the appropriate container base image to use this new runtime. This runtime version is supported by functions running on either Arm-based Amazon Web Services Graviton2 processors or x86-based processors. Using the Graviton2 processor architecture option allows you to get up to 34% better price performance.</p>\n<p>Read the post <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2a8\\" target=\\"_blank\\">Node.js 18.x runtime now available in Amazon Web Services Lambda</a></ins>, to find out more about the major changes available with the Node.js 18 runtime in Lambda. You should also check out <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2a9\\" target=\\"_blank\\">Why and how you should use Amazon Web Services SDK for JavaScript (v3) on Node.js 18</a></ins> as the Amazon Web Services SDK for JavaScript (v3) is included by default in Amazon Web Services Lambda Node.js 18 runtime.</p>\n<h3><a id=\\"MariaDB_301\\"></a><strong>MariaDB</strong></h3>\\n<p>Amazon Relational Database Service (Amazon RDS) for MariaDB now supports MariaDB minor versions 10.6.11, 10.5.18, 10.4.27 and 10.3.37. We recommend that you upgrade to the latest minor versions to fix known security vulnerabilities in prior versions of MariaDB, and to benefit from the numerous bug fixes, performance improvements, and new functionality added by the MariaDB community.</p>\n<h3><a id=\\"PostgreSQL_305\\"></a><strong>PostgreSQL</strong></h3>\\n<p>Amazon Relational Database Service (Amazon RDS) for PostgreSQL now supports PostgreSQL minor versions 14.5, 13.8, 12.12, 11.17, and 10.22. We recommend you upgrade to the latest minor version to fix known security vulnerabilities in prior versions of PostgreSQL, and to benefit from the bug fixes, performance improvements, and new functionality added by the PostgreSQL community. Please refer to the PostgreSQL community announcement for more details about the release. This release also includes support for Amazon RDS Multi-AZ with two readable standbys and updates for existing supported PostgreSQL extensions: PostGIS extension is updated to 3.1.7, pg_partman extension is updated to 4.6.2, and pgRouting extension is updated to 3.2.2. Please see the list of supported extensions in the Amazon RDS User Guide for specific versions.</p>\n<h2><a id=\\"Videos_of_the_week_309\\"></a><strong>Videos of the week</strong></h2>\\n<h3><a id=\\"Kubernetes_and_Amazon_Web_Services_310\\"></a><strong>Kubernetes and Amazon Web Services</strong></h3>\\n<p>If you missed this, then it is well worth checking out the awesome Jay Pipes discuss Amazon Web Services’ use of Kubernetes, as well as Amazon Web Services’ contributions to the Kubernetes code base. The interview was recorded at KubeCon North America last month.</p>\n<p><video src=\\"https://dev-media.amazoncloud.cn/227b20a3e20a4e15b13383c4939f6056_Kubernetes%20and%20Amazon%20Web%20Services.mp4\\" controls=\\"controls\\"></video></p>\\n<h3><a id=\\"OpenSearch_316\\"></a><strong>OpenSearch</strong></h3>\\n<p>The videos from OpenSearchCon that took place earlier this year are now available. You can see the <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2be\\" target=\\"_blank\\">entire list here</a></ins>, and there are a number of great sessions covering a very broad range of topics. The one I spent time watching was this session from OpenSearch Core Codebase Nicholas Knize, OpenSearch Maintainer, Lucene Committer and PMC Member. If you are interested in contributing to OpenSearch and curious in how to get started, then this session will answer some of these questions and more by raising the hood and exploring the code base.</p>\n<p><video src=\\"https://dev-media.amazoncloud.cn/ee20bae7a2bd4d869dfdde578b781b4e_Nick%20Knize%2C%20Getting%20Started%20with%20the%20OpenSearch%20Core%20Codebase%2C%20OpenSearchCon..mp4\\" controls=\\"controls\\"></video></p>\\n<h3><a id=\\"Kubeflow_and_MLFlow_322\\"></a><strong>Kubeflow and MLFlow</strong></h3>\\n<p>Join your hosts Antje Barth and Chris Fregley as they are joined by a number of guests to talk about some great open source projects such as Kubeflow, MLflow, datamesh.utils, and data.all</p>\n<p><video src=\\"https://dev-media.amazoncloud.cn/2ec26da8d3c344f2a090b47ccb0d7668_Managed%20Kubeflow%20%2B%20Managed%20MLflow%20%2B%20Data%20Mesh%20architectures%20on%20AWS.mp4\\" controls=\\"controls\\"></video></p>\\n<h3><a id=\\"Build_on_Open_Source_328\\"></a><strong>Build on Open Source</strong></h3>\\n<p>For those unfamiliar with this show, Build on Open Source is where we go over this newsletter and then invite special guests to dive deep into their open source project. Expect plenty of code, demos and hopefully laughs. We have put together a playlist so that you can easily access all (seven) of the other episodes of the Build on Open Source show. <ins><a href=\\"https://aws-oss.beachgeek.co.uk/24u\\" target=\\"_blank\\">Build on Open Source playlist</a></ins></p>\n<h1><a id=\\"Events_for_your_diary_333\\"></a><strong>Events for your diary</strong></h1>\\n<h3><a id=\\"Apache_Hudi_Meetup__reInvent_334\\"></a>Apache Hudi Meetup - re:Invent</h3>\\n<h3><a id=\\"November_28th__December_3rd_Las_Vegas_335\\"></a><strong>November 28th - December 3rd, Las Vegas</strong></h3>\\n<p>Apache Hudi is a data platform technology that helps build reliable and scalable data lakes. Hudi brings stream processing to big data, supercharging your data lakes, making them orders of magnitude more efficient.</p>\n<p>Hudi is widely used by many companies like Uber, Walmart, Amazon.com, Robinhood, GE, Disney Hotstar, Alibaba, ByteDance that build transactional or streaming data lakes. Hudi also comes pre-built with Amazon EMR and is integrated with Amazon Athena, Amazon Web Services Glue as well as Amazon Redshift. It is also integrated in many other cloud providers such as Google cloud and Alibaba cloud.</p>\n<p>Please join the Apache Hudi community for a Meetup hosted by Onehouse and the Apache Hudi community at the re:Invent site. Here are the different times and locations (local Vegas time):</p>\n<ul>\\n<li>Nov 28th [7:00 pm - 7:20 pm] Networking</li>\n<li>Nov 28th [7:20 pm - 7:50 pm] Hudi 101 (Speaker TBA)</li>\n<li>Nov 28th [7:50 pm - 8:20 pm] How Hudi supercharges your lake house architecture with streaming and historical data by Vinoth Chandar</li>\n<li>Nov 28th [8:20 pm - 8:40 pm] Roadmap (Speaker TBA)</li>\n<li>Nov 28th [8:40 pm - 9:00 pm] Open floor for Q&amp;A<br />\\nIt will be hosted in Conference room “Chopin 2” at the Encore Hotel</li>\n</ul>\\n<h3><a id=\\"reInvent_350\\"></a><strong>re:Invent</strong></h3>\\n<h3><a id=\\"November_28th__December_3rd_Las_Vegas_351\\"></a><strong>November 28th - December 3rd, Las Vegas</strong></h3>\\n<p>re:Invent is happening all this week, and there is plenty of great open source content for you, whether it is breakout sessions, chalk talks, open source vendors in the expo, and more.</p>\n<p>We will be featuring open source projects in the Developer Lounge again, in the Amazon Web Services Modern Applications and Open Source Zone. We have published a schedule of the open source projects you can check out, so why not take a peek at <ins><a href=\\"https://aws-oss.beachgeek.co.uk/2ac\\" target=\\"_blank\\">The Amazon Web Services Modern Applications and Open Source Zone: Learn, Play, and Relax at Amazon Web Services re:Invent 2022</a></ins> and come along. I will be there for a big chunk of time on Tuesday, Wednesday, and Thursday. If you have a good open source story to tell, or some SWAG to trade, I will be bringing our Build On Open Source challenge coins, so be sure to hunt me down!</p>\n<p>Check out this handy way to look at all the amazing open source sessions, then check out this <ins><a href=\\"https://aws-oss.beachgeek.co.uk/252\\" target=\\"_blank\\">dashboard</a></ins> [sign up required]. I would love to hear which ones you are excited about so please let me know in the comments or via Twitter. If you want to hear what my top three, must watch sessions, then this is what I would attend (sadly, as an Amazon Web Services employee I am not allowed to attend sessions)</p>\n<ol>\\n<li>OPN306 Amazon Web Services Lambda Powertools: Lessons from the road to 10 million downloads - Heitor Lessa is going to deliver an amazing session on the journey from idea to one of the most loved and used open source tools for Amazon Web Services Lambda users</li>\n<li>BOA204 When security, safety, and urgency all matter: Handling Log4Shell - Cannot wait for this session from Abbey Fuller who will walk us through how we managed this incident</li>\n<li>OPN202 Maintaining the Amazon Web Services Amplify Framework in the open - Matt Auerbach and Ashish Nanda are going to share details on how Amplify engineering managers work with the OSS community to build open-source software</li>\n</ol>\\n<h2><a id=\\"OpenSearch_364\\"></a><strong>OpenSearch</strong></h2>\\n<h3><a id=\\"Every_other_Tuesday_3pm_GMT_365\\"></a>Every other Tuesday, 3pm GMT</h3>\\n<p>This regular meet-up is for anyone interested in OpenSearch &amp; Open Distro. All skill levels are welcome and they cover and welcome talks on topics including: search, logging, log analytics, and data visualisation.</p>\n<p>Sign up to the next session, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/1az\\" target=\\"_blank\\">OpenSearch Community Meeting</a></ins></p>\n<h3><a id=\\"Stay_in_touch_with_open_source_at_Amazon_Web_Services_371\\"></a><strong>Stay in touch with open source at Amazon Web Services</strong></h3>\\n<p>I hope this summary has been useful. Remember to check out the <ins><a href=\\"https://aws.amazon.com/opensource/?opensource-all.sort-by=item.additionalFields.startDate&amp;opensource-all.sort-order=asc\\" target=\\"_blank\\">Open Source homepage</a></ins> to keep up to date with all our activity in open source by following us on <ins><a href=\\"https://twitter.com/AWSOpen\\" target=\\"_blank\\">@AWSOpen</a></ins></p>\n"}
目录
亚马逊云科技解决方案 基于行业客户应用场景及技术领域的解决方案
联系亚马逊云科技专家
亚马逊云科技解决方案
基于行业客户应用场景及技术领域的解决方案
联系专家
0
目录
关闭