Amazon Web Services open source newsletter, #185

## Edition #185 Welcome to issue #185 of the Amazon Web Services open source newsletter, the newsletter where we try and provide you the best open source on Amazon Web Services content. As always, this week we start with a round up of some freshly baked new projects for you to practice your four freedoms. This week we have projects that allow you to export your Partyrock applications, a tool to help you reduce hallucinations in your large language models, a new client for Redis, a tool to help you access the Amazon Web Services Partner Network, as well as sample projects that look at how you can use large langue models to build a new reader and building pipelines using Cloudformation. We also feature posts that include some of your favourite open source technologies, which this issue include Redis, Backstage, Amazon CDK, Kubernetes, [Amazon Corretto](https://aws.amazon.com/cn/corretto/?trk=cndc-detail), SnapStart, Dash, FluxCD, OpenSearch, Apache Airflow, Apache Kafka, Apache Hudi, [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail), MySQL, OpenShift, Linux, ROSA, and more. When I was small I was always told to never skip my vegetables, so heed this excellent advice and do not skip the Events section at the end! I have added a new link to a repo that lists open source events that are coming up in 2024, which is well worth checking out. **Feedback** Please please please take 1 minute to [complete this short survey](https://www.pulse.aws/promotion/10NT4XZQ?trk=cndc-detail). ### Celebrating open source contributors The articles and projects shared in this newsletter are only possible thanks to the many contributors in open source. I would like to shout out and thank those folks who really do power open source and enable us all to learn and build on top of what they have created. So thank you to the following open source heroes: Stephen Sennett, Zhuo-Wei Lee, Alontay Ellis, Rajarshi Das, Nearform, Wolfgang Unger, John McCloskey, Adam Keller, Vadym Kazulkin, Paulo Siecola, Sunny Bhambhani, Daniel Rivera, David Nalley, Parnab Basak, Chandan Rupakheti, Vinod Jayendra, Rupesh Tiwari, Arnav Mediratta, Dumlu Timuralp, Pratik R. Mankad, Praseeda Sathaye, Ovidiu Valeanu, and Olly Pomeroy. ### Latest open source projects *The great thing about open source projects is that you can review the source code. If you like the look of these projects, make sure you that take a look at the code, and if it is useful to you, get in touch with the maintainer to provide feedback, suggestions or even submit a contribution. The projects mentioned here do not represent any formal recommendation or endorsement, I am just sharing for greater awareness as I think they look useful and interesting!* ### Tools **partysmith** [partysmith](https://aws-oss.beachgeek.co.uk/3l4?trk=cndc-detail) is an awesome project from Amazon Web Services Community Builder Stephen Sennett, that provides an unofficial way to transform your Amazon Web Services PartyRock apps into deployable full-stack SvelteKit applications. Users can enter the URL of a publicly published PartyRock app, select the desired settings, and PartySmith will forge an application into a ZIP archive which will be downloaded to your machine, and ready for use. How cool is that! (Very in case you were wondering). Find out more by reading the supporting blog post, [PartySmith - Bring PartyRock apps to your place](https://aws-oss.beachgeek.co.uk/3l5?trk=cndc-detail). **RefChecker** [RefChecker](https://aws-oss.beachgeek.co.uk/3l3?trk=cndc-detail) For all their remarkable abilities, large language models (LLMs) have an Achilles heel, which is their tendency to hallucinate, or make assertions that sound plausible but are factually inaccurate. RefChecker provides automatic checking pipeline and benchmark dataset for detecting fine-grained hallucinations generated by Large Language Models. Check out the supporting post for this tool, [New tool, dataset help detect hallucinations in large language models](https://aws-oss.beachgeek.co.uk/3l7?trk=cndc-detail) an example of RefChecker pipeline **glide-for-redis** [glide-for-redis](https://aws-oss.beachgeek.co.uk/3l2?trk=cndc-detail) or General Language Independent Driver for the Enterprise (GLIDE) for Redis (mayeb GLIDER would have been cooler :-) is a new open source client for Redis that works with any Redis distribution that adheres to the Redis Serialization Protocol (RESP) specification. The client is optimised for security, performance, minimal downtime, and observability, and comes pre-configured with best practices learned from over a decade of operating Redis-compatible services used by hundreds of thousands of customers. **aws-signer-oci-artifacts** [aws-signer-oci-artifacts](https://aws-oss.beachgeek.co.uk/3km?trk=cndc-detail) this project is used to demonstrate how OCI artefacts can be signed and verified in a development pipeline. Zhuo-Wei Lee, Alontay Ellis, and Rajarshi Das have put together a blog post to help you get started, so if this project interests you, make sure you dive into [Signing and Validating OCI Artifacts with Amazon Web Services Signer](https://aws-oss.beachgeek.co.uk/3kn?trk=cndc-detail). **codecatalyst-blueprints** [codecatalyst-blueprints](https://aws-oss.beachgeek.co.uk/3kr?trk=cndc-detail) This repository contains common blueprint components, the base blueprint constructs and several public blueprints. Blueprints are code generators used to create and maintain projects in [Amazon CodeCatalyst](https://aws.amazon.com/cn/codecatalyst/?trk=cndc-detail). **aws-apn-connector** [aws-apn-connector](https://aws-oss.beachgeek.co.uk/3l1?trk=cndc-detail) this project from the folks at Nearform provides a way of interacting with the Amazon APN (Amazon Partner Network) programatically, as this does not provide an API. If you are looking to automate your interactions with the Amazon APN, you should check this project out. ### Demos, Samples, Solutions and Workshops **get-the-news-rss-atom-feed-summary** [get-the-news-rss-atom-feed-summary](https://aws-oss.beachgeek.co.uk/3l6?trk=cndc-detail) is a fantastic demo that demonstrates some of the capabilities that using large language models can help you introduce into your applications. The demo code provides a summary of the most recent news from an RSS or Atom feed using [Amazon Bedrock](https://aws.amazon.com/cn/bedrock/?trk=cndc-detail).  **cfn-pipeline** [cfn-pipeline](https://aws-oss.beachgeek.co.uk/3kv?trk=cndc-detail) is a repo from Wolfgang Unger that contains an Amazon Web Services Codepipeline that will allow automated Cloudformation deployments from within Amazon Web Services Codepipeline. To help you get started, Wolfgang has put together a detailed blog post that includes videos. Go check it out, [Pipeline for automatic CloudFormation Deployments](https://aws-oss.beachgeek.co.uk/3kw?trk=cndc-detail) ### Amazon Web Services and Community blog posts **The best from around the Community** What is new from around the community this week? John McCloskey from Bazaarvoice gets us going this week with his post, [Can You Keep a Secret?](https://aws-oss.beachgeek.co.uk/3ks?trk=cndc-detail)which is a must read for anyone who is using or looking at using backstage, an open platform for building developer portals. If you are looking for a good resource for understanding and getting started with the Amazon Cloud Development Kit (CDK), then Adam Keller has your back with the post, [The evolution of Infrastructure as Code with the Amazon Cloud Development Kit (CDK)](https://aws-oss.beachgeek.co.uk/3kt?trk=cndc-detail). Amazon Web Services Community Builder Vadym Kazulkin continues his quest to help Java developers everywhere better understand modern Java applications in the thirteenth (yes, thats right!!) instalment of his series, [Amazon SnapStart - Part 13 Measuring warm starts with Java 21 using different Lambda memory settings](https://aws-oss.beachgeek.co.uk/3ku?trk=cndc-detail). This is a must read series to dive deeper into how you can optimise your Java applications on Amazon Web Services, so check it out. Next up we have Paulo Siecola with [How to host a Plotly Dash app on Amazon ECS](https://aws-oss.beachgeek.co.uk/3kx?trk=cndc-detail)that shows you how to deploy [Dash](https://aws-oss.beachgeek.co.uk/3ky?trk=cndc-detail), a popular open source Python framework for building ML & data science web apps, using Amazon CDK. If you are a Kubernetes developer who likes to use helm charts to deploy your applications, then [Installing multiple helm charts in one go using helmfile](https://aws-oss.beachgeek.co.uk/3kz?trk=cndc-detail) from Sunny Bhambhani will be of interest. In this post he shows you how to install multiple helm charts. To finish up this week we have Daniel Rivera who has put together [Smart-Cash Project - GitOps with FluxCD](https://aws-oss.beachgeek.co.uk/3l0?trk=cndc-detail), the second post in a series that provides plenty of memes, but even more code, that shows you how you get started with GitOps using FluxCD. **OpenSearch** The OpenSearch project has achieved another milestone, launching the inaugural OpenSearch Leadership Committee. David Nalley has written about this in his post, [OpenSearch Expands Leadership Beyond Amazon Web Services](https://aws-oss.beachgeek.co.uk/3ki?trk=cndc-detail). The committee is another step toward open governance in which a broad community of users, contributors, and committers is included in a publicly visible and accessible process that determines an open source project’s direction. This weeks must read post, so check it out to find out more. **Apache Airflow** This is a post that is going to make a lot of Apache Airflow folks very happy. In [Disaster recovery strategies for Amazon MWAA – Part 1](https://aws-oss.beachgeek.co.uk/3kj?trk=cndc-detail), Parnab Basak, Chandan Rupakheti, Vinod Jayendra, and Rupesh Tiwari embark on an epic journey to explore the need for [Amazon Managed Workflows for Apache Airflow](https://aws.amazon.com/cn/managed-workflows-for-apache-airflow/?trk=cndc-detail) (MWAA) disaster recovery and prescribe solutions that will sustain Amazon MWAA environments against unintended disruptions. Essential reading for Airflow fans.  **Kubernetes** Important news this week from the [Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail) team, so make sure you read this. In [Amazon EKS extended support for Kubernetes versions pricing](https://aws-oss.beachgeek.co.uk/3kg?trk=cndc-detail) Arnav Mediratta talks about your extended support options, including pricing, for those of you wanting to understand more about running older versions of Kubernetes in your [Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail) clusters. This post covers what it is in more details, use cases, and cost. Sticking with Kubernetes we have [How to leverage Application Load Balancer’s advanced request routing to route application traffic across multiple Amazon EKS clusters](https://aws-oss.beachgeek.co.uk/3kl?trk=cndc-detail) where Dumlu Timuralp, Pratik R. Mankad, and Praseeda Sathaye show you how to use the advanced request routing capability of an existing Application Load Balancer to route traffic to micro services spread across multiple [Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail) clusters in a given Amazon Web Services Region. \[hands on] Also touching on the Kubernetes universe was this post, [Deploying and scaling Apache Kafka on Amazon EKS](https://aws-oss.beachgeek.co.uk/3kk?trk=cndc-detail) where Ovidiu Valeanu dives deep and walks you through how to build and deploy a Kafka cluster with [Amazon Elastic Kubernetes Service](https://aws.amazon.com/cn/eks/?trk=cndc-detail) ([Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail)) using Data on EKS (DoEKS). \[hands on]  **Other posts and quick reads** * [The Journey to IPv6 on Amazon EKS: Foundation (Part 1)](https://aws-oss.beachgeek.co.uk/3kd?trk=cndc-detail) is a series of three posts that looks to provide everything you need to know to make informed decisions around moving to IpV6 for your Kubernetes clusters  * [Real-time data distribution with Amazon MSK and Amazon AppSync](https://aws-oss.beachgeek.co.uk/3ke?trk=cndc-detail) demonstrate how to leverage Amazon Web Services services for ingesting and serving streaming data to client applications with minimal operational complexity \[hands on] * [Real-time serverless data ingestion from your Kafka clusters into Amazon Timestream using Kafka Connect](https://aws-oss.beachgeek.co.uk/3kh?trk=cndc-detail) is a hands on guide on how to stream events from your Amazon MSK clusters to [Amazon Timestream](https://aws.amazon.com/cn/timestream/?trk=cndc-detail) using the Timestream sink connector that helps you further analyse your time-series data \[hands on] * [Enforce fine-grained access control on Open Table Formats via Amazon EMR integrated with Amazon Web Services Lake Formation](https://aws-oss.beachgeek.co.uk/3kf?trk=cndc-detail) shows you how to implement fine-grained access controls on Apache Hudi tables using [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) integrated with Lake Formation \[hands on]  * [Scientific Data Management on Amazon Web Services with Open Source Quilt Data Packages](https://aws-oss.beachgeek.co.uk/3kq?trk=cndc-detail)looks at what Quilt Data Packages are (they were new to me!), and explores the concept of data packages for managing scientific data, both for wet-lab and computational data ### Quick updates **Amazon SDK End of Support Announcements** There were two end of support announcements last week that you should pay attention to. First up we had [Announcing end-of-support for Amazon SDK for Go (v1) effective July 31, 2025](https://aws-oss.beachgeek.co.uk/3ko?trk=cndc-detail) that provides a time line and suggestions options for those of you who might be using this. Following this, we had [Announcing end-of-support for Amazon SDK for Java v1.x effective December 31, 2025](https://aws-oss.beachgeek.co.uk/3kp?trk=cndc-detail) that provides similar help around showing you a time line and making recommendations for you. **[Amazon Corretto](https://aws.amazon.com/cn/corretto/?trk=cndc-detail)** On Jan 16, 2024 Amazon announced quarterly security and critical updates for [Amazon Corretto](https://aws.amazon.com/cn/corretto/?trk=cndc-detail) Long-Term Supported (LTS) and Feature (FR) versions of OpenJDK. Corretto 21.0.2, 17.0.10, 11.0.22, 8u402 are now available for download. [Amazon Corretto](https://aws.amazon.com/cn/corretto/?trk=cndc-detail) is a no-cost, multi-platform, production-ready distribution of OpenJDK. You can get your downloads over at the [Amazon Corretto download page](https://aws-oss.beachgeek.co.uk/2fh?trk=cndc-detail). **MySQL** Amazon Relational Database Service ([Amazon RDS](https://aws.amazon.com/cn/rds/?trk=cndc-detail)) for MySQL now supports multi-source replication, which allows you to configure multiple RDS for MySQL database instances as sources for a single RDS for MySQL target database instance. Multi-source replication on RDS for MySQL enables you to merge multiple shards into a single target, or consolidate data in one RDS for MySQL instance for analytics, or long-term backups. For every target RDS for MySQL instance, you can configure up to 15 RDS for MySQL source database instances. For read scaling, you can configure read replicas on the multi-source replication target instance. Multi-source replication is available on RDS for MySQL versions 5.7.44, 8.0.35 and higher minor versions. **OpenShift** The Red Hat OpenShift Service on Amazon Web Services (ROSA) with hosted control planes (HCP) deployment model is generally available. Under the classic ROSA deployment model, Amazon Web Services infrastructure required to run the ROSA control plane is hosted on your Amazon Web Services account. Now, you can create ROSA clusters with the control plane hosted and managed on a service account. Like ROSA classic, ROSA with HCP clusters incur on-demand service fees per worker node vCPU. Additionally, ROSA with HCP clusters incur an on-demand service fee per cluster hour. For service fee discounts, you can optionally purchase upfront ROSA with HCP contracts from the ROSA console. ROSA with HCP reduces the cost of running ROSA, speeds up cluster creation times, improves version upgrade flexibility, and uses Amazon Web Services managed policies to provide ROSA IAM service roles with narrowly-scoped permissions. ROSA with HCP helps reduce the cost of running ROSA clusters, as ROSA control planes are hosted and managed by Red Hat on Amazon Web Services. This approach mitigates the risk that actions taken on your account lead to a degraded ROSA control plane. The new deployment model reduces ROSA cluster creation times, and you can separately schedule OpenShift version upgrades for the control plane and the worker node machine pools. **Linux News** Amazon Web Services Systems Manager now supports instances running Ubuntu 23.04, Debian 12, and SUSE SP5. Systems Manager customers running these operating systems versions now have access to all Amazon Web Services Systems Manager Node Management capabilities, including Fleet Manager, Compliance, Inventory, Hybrid Activations, Session Manager, Run Command, State Manager, Patch Manager, and Distributor. ### Videos of the week **[Amazon ECS](https://aws.amazon.com/cn/ecs/?trk=cndc-detail): Amazon Fargate with Seekable OCI** Seekable OCI (SOCI) is open sourced technology that allows you to launch containers faster by lazily loading container images. In this video, Olly Pomeroy walks you through how this is used in [Amazon ECS](https://aws.amazon.com/cn/ecs/?trk=cndc-detail) Fargate to help applications deploy and scale out faster by enabling containers to start without waiting to download the entire container image. https://youtu.be/9W045sDuYKw?trk=cndc-detail ### Events for your diary If you are planning any events in 2024, either virtual, in person, or hybrid, get in touch as I would love to share details of your event with readers. I recently found this GitHub repo, [open-source-events](https://aws-oss.beachgeek.co.uk/3jt?trk=cndc-detail) that is a curated set of open source events for 2024. Go check it out and see what 2024 is looking like. **FOSDEM**\ **Brussels, Belgium 3rd and 4th February, 2024** FOSDEM is a free event for software developers to meet, share ideas and collaborate. Every year, thousands of developers of free and open source software from all over the world gather at the event in Brussels. There are plenty of folks from Amazon Web Services that will be there, so drop me a DM if you are going and want to meet up. You don't need to register. Just turn up and join in! Find out more by heading to the site home page, [FOSDEM 24](https://fosdem.org/2024/?trk=cndc-detail) **SOOCon 24**\ **London, UK 7th and 8th February, 2024** The State of Open Conference is back in the new year, and what better opportunity to combine with your trip to Fosdem, than come along to the wonderful Brewery in the City of London and catch up with the hundreds of open source developers, enthusiasts, hackers, and more. Check out the event page, and if you are so inclined, there is still chance to submit a talk as the call for papers (cfp) is open until the 15th of December. You can view more at the site page, [SOOCon24](https://stateofopencon.com/?trk=cndc-detail) **Cortex**\ **Every other Thursday, next one 16th February** The Cortex community call happens every two weeks on Thursday, alternating at 1200 UTC and 1700 UTC. You can check out the GitHub project for more details, go to the [Community Meetings](https://aws-oss.beachgeek.co.uk/2h5?trk=cndc-detail) section. The community calls keep a rolling doc of previous meetings, so you can catch up on the previous discussions. Check the [Cortex Community Meetings Notes](https://aws-oss.beachgeek.co.uk/2h6?trk=cndc-detail) for more info. **OpenSearch**\ **Every other Tuesday, 3pm GMT** This regular meet-up is for anyone interested in OpenSearch & Open Distro. All skill levels are welcome and they cover and welcome talks on topics including: search, logging, log analytics, and data visualisation. Sign up to the next session, [OpenSearch Community Meeting](https://aws-oss.beachgeek.co.uk/1az?trk=cndc-detail) ### Stay in touch with open source at Amazon Web Services Remember to check out the [Open Source homepage](https://aws.amazon.com/opensource/?opensource-all.sort-by=item.additionalFields.startDate\\&opensource-all.sort-order=asc?trk=cndc-detail) for more open source goodness. Also a reminder, we now have a GitHub repo that just lists all the great open source code that is shared within this newsletter. Where I can hear you all ask? Well as you ask so nicely, you can meander over to[ newsletter-oss-projects](https://aws-oss.beachgeek.co.uk/3l8?trk=cndc-detail). Made with ♥ from DevRel