Open source news and updates, #54

{"value":"### **February 1st, 2021 - Instalment #54**\nNewsletter #54. Lots to choose from this week. Catch up with the latest from the Open Distro for Elasticsearch community, an event for your diary in the shape of Innovate AI/ML where I will be covering open source tools in your AI/ML pipeline, and the usual round up of Amazon Web Services and community open source posts. We have some great open source projects that you need to check out - from tools and utilities that solve real pain points to complete solutions you can use as the base for your projects. Also this week, if you are looking for a fresh challenge, why not take a look at the opening within the JuypterLab team.\n\nAs always, I am always eager to hear from you about your projects or blogs/solutions you have been working on so please get in touch if you would like me to share your projects here.\n\n#### **Open Distro for Elasticsearch**\n\n++[Update from the fork team: Jan 28](https://aws-oss.beachgeek.co.uk/3v)++ Kyle Davis provides an update on where we are at with the getting the public forks ready. The team had a community call which quickly became oversubscribed, but you can see details on the ++[Open Distro site, Community Meeting - Jan 26, 2021](https://aws-oss.beachgeek.co.uk/3w)++\n\nIf you want to stay up to date then make sure you check out the meetup schedule - next meeting is on 8th Feb. You can check details ++[here](https://www.meetup.com/Open-Distro-for-Elasticsearch-Meetup-Group/events/)++.\n\nFinally, with the recent news of the forking of Elasticsearch and Kibana, check out the ++[FAQ](https://aws-oss.beachgeek.co.uk/4n)++ that the Open Distro for Elasticsearch community have put together.\n\n#### **Job**\n\nInterested in Open Source and Machine Learning? Amazon Web Services SageMaker open source JupyterLab contribution team is looking for Front End Engineers to join the team in Palo Alto. If you are interested, then take a look at the job description: ++[Frontend Software Development Engineer - Open Source JupyterLab](https://aws-oss.beachgeek.co.uk/3q)++\n\n#### **Innovate AI/ML**\n\nI have been putting together some blog posts to support my session at the ++[Innovate AI/ML event](https://aws.amazon.com/events/aws-innovate/machine-learning/online/emea/?sc_channel=em&sc_campaign=EMEA_FIELD_WEBINAR_innovate-AIML_20210224_7014z000001MJbu&sc_medium=em_&sc_content=REG_t1_field&sc_geo=emea&sc_country=mult&sc_outcome=reg&sc_publisher=aws&trkCampaign=emea21_innovatemlq1&trk=em_emea21_innovatemlq1_ricardosueiras)++ happening on Feb, 24th. I will be covering using open source tools in your AI/ML pipeline, and taking a look at Apache Airflow specifically.\n\nHere are the blog posts:\n\n- ++[Automating the installation and configuration of Amazon Managed Workflows for Apache Airflow](https://aws-oss.beachgeek.co.uk/3h)++\n- ++[Working with Permissions](https://aws-oss.beachgeek.co.uk/3n)++\n- ++[Accessing Amazon Managed Workflows for Apache Airflow](https://aws-oss.beachgeek.co.uk/3o)++\n- Interacting with [Amazon Managed Workflows for Apache Airflow](https://aws.amazon.com/cn/managed-workflows-for-apache-airflow/?trk=cndc-detail) via the command line\n\nI will be adding more before my session on the 24th. Use the ++[sign up link here](https://aws.amazon.com/events/aws-innovate/machine-learning/online/emea/?sc_channel=em&sc_campaign=EMEA_FIELD_WEBINAR_innovate-AIML_20210224_7014z000001MJbu&sc_medium=em_&sc_content=REG_t1_field&sc_geo=emea&sc_country=mult&sc_outcome=reg&sc_publisher=aws&trkCampaign=emea21_innovatemlq1&trk=em_emea21_innovatemlq1_ricardosueiras)++ to register for this event.\n\n#### **Celebrating open source contributors**\nThe articles posted in this series are only possible thanks to contributors and project maintainers and so I would like to shout out and thank those folks who really do power open source and enable us all to build on top of what they have created.\n\nSo thank you to the following open source heros: Chang Ho Jung, Emile Nijssen, Sofian Hamiti, Shreyas Subramanian, Keith Andruch, Andrew Park, Fei Lang, Ray Liu, Nikolai Kolesnikov, Ugur KIRA, Dejun Hu, TP Kohli, Jim Mlodgenski, Kinnar Sen, Tarun Kumar, Frank Liu, Matt Briggs, Mischa Spiegelmock, Rehan van der Merwe, Ian McKay and Artyom Pervukhin.\n\nMake sure you find and follow these builders and keep up to date with their open source projects and contributions.\n\n#### **Latest from open source projects**\n##### **cfn-diagram**\n\ncfn-diagram is an open source tool from the MatHem team that provides a cli tool to visualise CloudFormation/SAM/CDK templates as diagrams. I shared this project a while ago, but it came up again this week so if you missed it the first time, make sure you check it out this time.\n\n\n\n##### **cfn-stack-rename**\n\n++[cfn-stack-rename](https://aws-oss.beachgeek.co.uk/4d)++ Amazon Web Services Hero and prolific open source builder Ian McKay has released a new project that lets you rename your CloudFormation stack. Ian provides a quick overview of how it works as well as some gotchas to look out for when trying this out.\n\n##### **serverless-stack**\n\n++[serverless-stack](https://aws-oss.beachgeek.co.uk/4e)++ Serverless Stack Toolkit is an open source tool that allows you to build serverless apps using Amazon Web Services CDK. Check out the detailed documentation with lots of examples ++[here](https://aws-oss.beachgeek.co.uk/4f)++\n\n##### **awsautodns**\n\n++[awsautodns](https://aws-oss.beachgeek.co.uk/4q)++ awsautodns is a tool that allows an Amazon Web Services EC2 Instance to automatically register it's IP address in a Route53 Hosted Zone at startup. The install script creates a oneshot service, awsautodns.service, and sets it to start at boot, by installing it to multi-user.target. awsautodns is useful for avoiding fees associated with using an elastic IP address.\n\n##### **Amazon Web Services-iot-device-client**\n\n++[Amazon Web Services-iot-device-client](https://aws-oss.beachgeek.co.uk/4g)++ The Amazon Web Services IoT Device Client is an open-source, modular software written in C++ that you can compile and install on your Embedded Linux based IoT devices to access Amazon Web Services IoT Core, Amazon Web Services IoT Device Management, and Amazon Web Services IoT Device Defender features by default. It serves as a reference implementation for your IoT devices to work with Amazon Web Services IoT services, with operational best practices baked in\n\n##### **s3fetch**\n\n++[s3fetch](https://aws-oss.beachgeek.co.uk/4h)++ s3fetch is a simple & fast multi-threaded S3 download tool, with a breakdown on the project page of how they have achieved the speed. This is a different project from last week.\n\n##### **cloudfront-autoindex**\n\n++[cloudfront-autoindex](https://aws-oss.beachgeek.co.uk/4i)++ is a project from Artyom Pervukhin that solves a very particular pain point. Read the problem space that he has documented and see if you are having the same problems - he might just save you some time.\n\n##### **porter**\n\n++[porter](https://aws-oss.beachgeek.co.uk/4j)++ this looks like a very nice project that anyone familiar with the Heroku model will be interested in. Porter is a Kubernetes-powered PaaS that runs in your own cloud provider such as Amazon Web Services. Porter brings the Heroku experience to Kubernetes. Check out this ++[thread](https://aws-oss.beachgeek.co.uk/4k)++ provides some additional discussion on this topic that is well worth reading.\n\n##### **amazon-kinesis-video-streams-media-screencast-android**\n\n++[amazon-kinesis-video-streams-media-screencast-android](https://aws-oss.beachgeek.co.uk/4o)++ this open source project will show you how you can screencast your mobile app using Kinesis Video Stream with WebRTC in Android. Chang Ho Jung has also put together a detailed blog post to walk you through the process, ++[How to screencast apps using Amazon Kinesis Video Streams with WebRTC on Android](https://aws-oss.beachgeek.co.uk/4p)++\n\n#### **Video of the week**\n##### **Juypter Notbooks on Amazon Lightsail**\n\nThis is a great new video blog from Julien Simon, always focused on helping you optimise whether that is performance or cost. In this post, he shows you how you can run Juypter notebooks on [Amazon Lightsail ](https://aws.amazon.com/cn/lightsail/?trk=cndc-detail)for a very small amount indeed.\n\n<video src=\\"https://dev-media.amazoncloud.cn/dccc5518521f4186b3b8c1657c0d8a42_Running%20Jupyter%20notebooks%20on%20Amazon%20Lightsail.mp4\\" class=\\"manvaVedio\\" controls=\\"controls\\" style=\\"width:160px;height:160px\\"></video>\n\n##### **ROS**\nCheck out this great video tutorial on how to run a simulation on Amazon Web Services RoboMaker. Perfect if you are just starting out in ROS and want to know some of the tools and terminology.\n\n<video src=\\"https://dev-media.amazoncloud.cn/445cad11be1a4ea8a6d18eabcfce9b5e_Run%20a%20Simulation%20in%20AWS%20RoboMaker.mp4\\" class=\\"manvaVedio\\" controls=\\"controls\\" style=\\"width:160px;height:160px\\"></video>\n\n##### **Blog posts from the community and customers**\n##### **Amazon Web Services CDK**\n\nA couple of Amazon Web Services CDK posts this week. First up we have ++[Web Services with Amazon Web Services CDK](https://aws-oss.beachgeek.co.uk/48)++ Mischa Spiegelmock with a post that brought a smile to my face, introducing an open source application called Cursed Webring, featuring \"the worst of the internet\". So what I hear you say. Well, Mischa shows you how you can use the Amazon Web Services CDK to build this application. Full source code is available in the ++[GitHub repository](https://aws-oss.beachgeek.co.uk/49)++ too. I guess the question I have is will this Newsletter ever make it into the Cursed Webring!\n\nFollowing that we have Amazon Web Services Community Hero Rehan van der Merwe with a post, ++[4 Methods to configure multiple environments in the Amazon Web Services CDK](https://aws-oss.beachgeek.co.uk/4a)++ in which you will learn different methods that can be used to pass configuration values to the Amazon Web Services CDK. I will not give away what those four methods are, you will just have to read the post. Like all good open source posts, source code over at ++[his GitHub repo.](https://aws-oss.beachgeek.co.uk/4b)++\n\n\n##### **Amazon Web Services Amplify**\n\n++[Building a Ron Swanson Quote App on Amplify with Javascript and GraphQL](https://aws-oss.beachgeek.co.uk/4c)++ from my friends at Steamhaus, Paul Flowers with a great post with a hint of comedy (a winning combination) that shows you how easy it is to create a simple API that will be return Ron Swanson quotes to be displayed by a React frontend. Who (Why?) Ron Swanson, well only Paul will know that but this is a great fun post and a great way to learn how you can use Amazon Web Services Amplify to quickly put something like this together.\n\n##### **Linux OS for Kubernetes**\n\nIn this post from Talos Systems CEO, Steve Francis, ++[A Guide to Linux Operating Systems for Kubernetes](https://aws-oss.beachgeek.co.uk/40)++ he provides some insights into the specific needs around Linux operating systems within Kubernetes. If you want to know a little more as to what the Amazon Web Services open source project Bottlerocket is helping to address, read this post and hopefully it will be clearer.\n\n##### **Grafana**\n\n++[Monitoring Amazon Web Services Elastic Beanstalk Memory and Disk Usage in Grafana](https://aws-oss.beachgeek.co.uk/4m)++ Emile Nijssen shows you how easy it is to setup a Grafana dashboard to display your CloudWatch metrics, in this case, metrics that the CloudWatch agent in an Elastic Beanstalk instance is generating.\n\n#### **Amazon Web Services open source posts**\n##### **Bottlerocket**\n\n++[How the Bottlerocket build system works](https://aws-oss.beachgeek.co.uk/47)++ is great post from Matt Briggs on Bottlerocket. Bottlerocket is an open source, special-purpose operating system designed for hosting Linux containers. This post dives deep into the build system for Bottlerocket, taking a look at how it uses Cargo, the Rust package manager and some of the tooling used. This is a great way to understand in more detail how this works for your own Rust projects potentially.\n\n##### **MLflow**\n\n++[Managing your machine learning lifecycle with MLflow and Amazon SageMaker](https://aws-oss.beachgeek.co.uk/3x)++ Sofian Hamiti and Shreyas Subramanian take a look at an open source solution to helping data scientists manage their machine learning experiments. I took a look at this early in the year and hope to do some blog posts around it soon. In the meantime, this post shows how the open-source platform MLflow helps data scientists collaborate and share experiment results in order to find a solution to a business need. Great post, so read on to find out more about MLflow and how you can integrate this with [Amazon SageMaker](https://aws.amazon.com/cn/sagemaker/?trk=cndc-detail).\n\n\n\n##### **Deep Java Library**\n\n++[Model serving in Java with Amazon Web Services Elastic Beanstalk made easy with Deep Java Library](https://aws-oss.beachgeek.co.uk/46)++ Frank Liu has put together this guide to show you how to deploy a model on Elastic Beanstalk using the Deep Java Library (DJL), using an example of sending an image through a post call to get inference results on what the image contains.\n\n##### **Apache Airflow**\n\n++[Orchestrating analytics jobs on Amazon EMR Notebooks using Amazon MWAA](https://aws-oss.beachgeek.co.uk/3y)++ Fei Lang and Ray Liu have put together another great post that shows how to use [Amazon Managed Workflows for Apache Airflow](https://aws.amazon.com/cn/managed-workflows-for-apache-airflow/?trk=cndc-detail) (Amazon MWAA) to orchestrate analytics jobs on EMR Notebooks. They walk you through an end to end solution and is essential reading if you are just getting to grips with what Apache Airflow can do for you.\n\n\n\n##### **Apache Cassandra**\n\n++[Compress and conquer with Amazon Keyspaces (for Apache Cassandra)](https://aws-oss.beachgeek.co.uk/41)++ Nikolai Kolesnikov with a helpful post that shows you how to compress your data using freely available compression tools and store that compressed data in [Amazon Keyspaces](https://aws.amazon.com/cn/keyspaces/?trk=cndc-detail). To help deliver fast performance, [Amazon Keyspaces](https://aws.amazon.com/cn/keyspaces/?trk=cndc-detail) has a 1 MB row-size quota. What if you already have rows larger than 1 MB in your existing Cassandra tables? To reduce the size of these rows, you can compress one or more large columns. Compressing large columns reduces your storage costs, improves performance, reduces I/O and network usage, and enables you to fit the data within the [Amazon Keyspaces](https://aws.amazon.com/cn/keyspaces/?trk=cndc-detail) row quota. At that is what Nikolai will walk you through in this post.\n\n##### **Apache Spark**\n\n++[Running cost optimized Spark workloads on Kubernetes using EC2 Spot Instances](https://aws-oss.beachgeek.co.uk/44)++ Kinnar Sen takes a look at how to run Apache Spark workloads in a resilient, scalable and cost optimised way using [Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail) service in conjunction with Spot Instances. Apache Spark is an open-source, distributed processing system used for big data workloads, and supports Kubernetes as one of its cluster managers allowing you to allocate and schedule your analytics resources. This is a detailed walkthrough, so set some time aside as you will cover a lot of ground.\n\n\n\n\n##### **Fluent Bit**\n\n++[Fluent Bit Integration in CloudWatch Container Insights for EKS](https://aws-oss.beachgeek.co.uk/42)++ Ugur KIRA, Dejun Hu and TP Kohli collaborate on this post that introduces CloudWatch Container Insights, which enables you to explore, analyze, and visualize your container metrics, Prometheus metrics, application logs, and performance log events through automated dashboards in the CloudWatch console. Fluent Bit is an open source and multi-platform log processor and forwarder that allows you to collect data and logs from different sources, and unify and send them to different destinations including CloudWatch Logs. In this post find out how these two integrate to help you considerably enhances your observability and debugging capabilities in your container workloads.\n\n##### **PostgreSQL**\n\n++[Getting more with PostgreSQL purpose-built data types](https://aws-oss.beachgeek.co.uk/43)++ Jim Mlodgenski explores some of PostgreSQL’s purpose-built types, covering UUID and Network Addresses, Geospatial and Chemistry. This is just the tip however, and as Jim says in this post \"PostgreSQL’s extensible architecture will allow new types to be created in the future as PostgreSQL moves into more industries requiring specialized data handling.\"\n\n##### **Amazon Web Services SAM**\n\n++[Building a Jenkins Pipeline with Amazon Web Services SAM](https://aws-oss.beachgeek.co.uk/45)++ Tarun Kumar is here with a post that shows how to set up a multi-stage pipeline on a Jenkins host for a serverless application, using the Amazon Web Services Serverless Application Model (Amazon Web Services SAM).\n\n\n\n#### **Amazon Web Services Partners**\n##### **ancher and [Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail)**\n\n++[Optimizing Your Kubernetes Clusters with Rancher and Amazon EKS](https://aws-oss.beachgeek.co.uk/3z)++ Keith Andruch and Andrew Park take a look at how combining Rancher with [Amazon EKS](https://aws.amazon.com/cn/eks/?trk=cndc-detail) provides a solution that allows enterprises to overcome common technical challenges associated with Kubernetes, including learning and resourcing gaps, onboarding, and operational difficulties.\n\n\n\n#### **Amazon Web Services Case Studies**\n++[Expedia Group Speeds Up Vendor Payment Transactions from 1 Day to Seconds](https://aws-oss.beachgeek.co.uk/3t)++ is a case study on how they are using open source technologies, and how these are helping them break free of legacy, proprietary databases in order to achieve significant business value. To help migrate they are using a tool that has an open source community edition, Flyway. Check them out if you are also thinking about migrating and want to explore your database migration options.\n\n++[How INESC TEC tests and deploys ROS applications with Amazon Web Services RoboMaker](https://aws-oss.beachgeek.co.uk/4l)++ Rafael Arrais, Project Manager at INESC TEC describes how robotics researchers at INESC TEC in Portugal are using Amazon Web Services RoboMaker to set up a simulation-based regression testing CI/CD pipeline, showing you how they used simulation to scale testing, speed up dev velocity and reduce errors.\n\n\n\n#### **Quick updates**\n##### **PostgreSQL**\n\n[Amazon Aurora](https://aws.amazon.com/cn/rds/aurora/?trk=cndc-detail) with PostgreSQL compatibility now supports major version 12. PostgreSQL 12 includes better index management, improved partitioning capabilities, and the ability to execute JSON path queries per SQL/JSON specifications. It also enables nondeterministic collations that support case-insensitive and accent-insensitive comparisons for ICU provided collations, most common-value statistics for improved query plans, creation of generated columns that compute values with an expression, and many additional features.\n\nThis release also updates extensions including Address_standardizer, Address_standardizer_data_us, Amcheck, Citext, Hll, Hstore, Ip4r, Pg_repack, Pg_stat_statements, Pgaudit, Pglogical, Pgrouting, Plv8, Postgis, Postgis_tiger_geocoder, Postgis_topology.\n\n##### **In place upgrades**\n\nYou can perform an in-place upgrade of your [Amazon Aurora](https://aws.amazon.com/cn/rds/aurora/?trk=cndc-detail) database cluster from PostgreSQL major version 11 to 12. Instead of backing up and restoring the database to the new version, you can upgrade with just a few clicks in the [Amazon RDS](https://aws.amazon.com/cn/rds/?trk=cndc-detail) Management Console or by using the Amazon Web Services SDK or CLI.\n\n##### **Fluent Bit**\n\nAmazon Web Services announces the availability of Fluent Bit support for [Amazon CloudWatch](https://aws.amazon.com/cn/cloudwatch/?trk=cndc-detail), a fully managed, pay-as-you-go monitoring and observability service for resources running on Amazon Web Services and on-premises. CloudWatch Container Insights and Log Insights enables you to explore, analyse, and visualise your container logs collected from the Fluent Bit processor. Container Insights also provides access to the automated dashboards that summarise the performance and availability of the FluentBit Daemon set running on [Amazon Elastic Kubernetes Service](https://aws.amazon.com/cn/eks/?trk=cndc-detail) (EKS) and Kubernetes. With few clicks in the Amazon Web Services Management Console, you can start collecting Fluent Bit logs in CloudWatch to monitor and troubleshoot containers.\n\n\n##### **Elasticsearch**\n\nAmazon Elasticsearch Service now supports encryption of data at rest and node-to-node encryption on existing domains, enabling organisations hosting sensitive workloads to meet stringent security and compliance requirements.\n\nRead more about the announcement, ++[Amazon Elasticsearch Service extends encryption at rest and node-to-node encryption to existing domains](https://aws-oss.beachgeek.co.uk/3s)++\n\n#### **Stay in touch with open source at Amazon Web Services**\nI hope this summary has been useful. Remember to check out the ++[Open Source homepage](https://aws.amazon.com/opensource/?opensource-all.sort-by=item.additionalFields.startDate&opensource-all.sort-order=asc)++ to keep up to date with all our activity in open source by following us on ++[@AWSOpen](https://twitter.com/AWSOpen)++.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","render":"<h3><a id=\\"February_1st_2021__Instalment_54_0\\"></a><strong>February 1st, 2021 - Instalment #54</strong></h3>\\n<p>Newsletter #54. Lots to choose from this week. Catch up with the latest from the Open Distro for Elasticsearch community, an event for your diary in the shape of Innovate AI/ML where I will be covering open source tools in your AI/ML pipeline, and the usual round up of Amazon Web Services and community open source posts. We have some great open source projects that you need to check out - from tools and utilities that solve real pain points to complete solutions you can use as the base for your projects. Also this week, if you are looking for a fresh challenge, why not take a look at the opening within the JuypterLab team.</p>\n<p>As always, I am always eager to hear from you about your projects or blogs/solutions you have been working on so please get in touch if you would like me to share your projects here.</p>\n<h4><a id=\\"Open_Distro_for_Elasticsearch_5\\"></a><strong>Open Distro for Elasticsearch</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/3v\\" target=\\"_blank\\">Update from the fork team: Jan 28</a></ins> Kyle Davis provides an update on where we are at with the getting the public forks ready. The team had a community call which quickly became oversubscribed, but you can see details on the <ins><a href=\\"https://aws-oss.beachgeek.co.uk/3w\\" target=\\"_blank\\">Open Distro site, Community Meeting - Jan 26, 2021</a></ins></p>\n<p>If you want to stay up to date then make sure you check out the meetup schedule - next meeting is on 8th Feb. You can check details <ins><a href=\\"https://www.meetup.com/Open-Distro-for-Elasticsearch-Meetup-Group/events/\\" target=\\"_blank\\">here</a></ins>.</p>\n<p>Finally, with the recent news of the forking of Elasticsearch and Kibana, check out the <ins><a href=\\"https://aws-oss.beachgeek.co.uk/4n\\" target=\\"_blank\\">FAQ</a></ins> that the Open Distro for Elasticsearch community have put together.</p>\n<h4><a id=\\"Job_13\\"></a><strong>Job</strong></h4>\\n<p>Interested in Open Source and Machine Learning? Amazon Web Services SageMaker open source JupyterLab contribution team is looking for Front End Engineers to join the team in Palo Alto. If you are interested, then take a look at the job description: <ins><a href=\\"https://aws-oss.beachgeek.co.uk/3q\\" target=\\"_blank\\">Frontend Software Development Engineer - Open Source JupyterLab</a></ins></p>\n<h4><a id=\\"Innovate_AIML_17\\"></a><strong>Innovate AI/ML</strong></h4>\\n<p>I have been putting together some blog posts to support my session at the <ins><a href=\\"https://aws.amazon.com/events/aws-innovate/machine-learning/online/emea/?sc_channel=em&amp;sc_campaign=EMEA_FIELD_WEBINAR_innovate-AIML_20210224_7014z000001MJbu&amp;sc_medium=em_&amp;sc_content=REG_t1_field&amp;sc_geo=emea&amp;sc_country=mult&amp;sc_outcome=reg&amp;sc_publisher=aws&amp;trkCampaign=emea21_innovatemlq1&amp;trk=em_emea21_innovatemlq1_ricardosueiras\\" target=\\"_blank\\">Innovate AI/ML event</a></ins> happening on Feb, 24th. I will be covering using open source tools in your AI/ML pipeline, and taking a look at Apache Airflow specifically.</p>\n<p>Here are the blog posts:</p>\n<ul>\\n<li><ins><a href=\\"https://aws-oss.beachgeek.co.uk/3h\\" target=\\"_blank\\">Automating the installation and configuration of Amazon Managed Workflows for Apache Airflow</a></ins></li>\n<li><ins><a href=\\"https://aws-oss.beachgeek.co.uk/3n\\" target=\\"_blank\\">Working with Permissions</a></ins></li>\n<li><ins><a href=\\"https://aws-oss.beachgeek.co.uk/3o\\" target=\\"_blank\\">Accessing Amazon Managed Workflows for Apache Airflow</a></ins></li>\n<li>Interacting with Amazon Managed Workflows for Apache Airflow via the command line</li>\n</ul>\\n<p>I will be adding more before my session on the 24th. Use the <ins><a href=\\"https://aws.amazon.com/events/aws-innovate/machine-learning/online/emea/?sc_channel=em&amp;sc_campaign=EMEA_FIELD_WEBINAR_innovate-AIML_20210224_7014z000001MJbu&amp;sc_medium=em_&amp;sc_content=REG_t1_field&amp;sc_geo=emea&amp;sc_country=mult&amp;sc_outcome=reg&amp;sc_publisher=aws&amp;trkCampaign=emea21_innovatemlq1&amp;trk=em_emea21_innovatemlq1_ricardosueiras\\" target=\\"_blank\\">sign up link here</a></ins> to register for this event.</p>\n<h4><a id=\\"Celebrating_open_source_contributors_30\\"></a><strong>Celebrating open source contributors</strong></h4>\\n<p>The articles posted in this series are only possible thanks to contributors and project maintainers and so I would like to shout out and thank those folks who really do power open source and enable us all to build on top of what they have created.</p>\n<p>So thank you to the following open source heros: Chang Ho Jung, Emile Nijssen, Sofian Hamiti, Shreyas Subramanian, Keith Andruch, Andrew Park, Fei Lang, Ray Liu, Nikolai Kolesnikov, Ugur KIRA, Dejun Hu, TP Kohli, Jim Mlodgenski, Kinnar Sen, Tarun Kumar, Frank Liu, Matt Briggs, Mischa Spiegelmock, Rehan van der Merwe, Ian McKay and Artyom Pervukhin.</p>\n<p>Make sure you find and follow these builders and keep up to date with their open source projects and contributions.</p>\n<h4><a id=\\"Latest_from_open_source_projects_37\\"></a><strong>Latest from open source projects</strong></h4>\\n<h5><a id=\\"cfndiagram_38\\"></a><strong>cfn-diagram</strong></h5>\\n<p>cfn-diagram is an open source tool from the MatHem team that provides a cli tool to visualise CloudFormation/SAM/CDK templates as diagrams. I shared this project a while ago, but it came up again this week so if you missed it the first time, make sure you check it out this time.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/928a8801f66e41ea90db2140984e5db8_demo.gif\\" alt=\\"demo.gif\\" /></p>\n<h5><a id=\\"cfnstackrename_44\\"></a><strong>cfn-stack-rename</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/4d\\" target=\\"_blank\\">cfn-stack-rename</a></ins> Amazon Web Services Hero and prolific open source builder Ian McKay has released a new project that lets you rename your CloudFormation stack. Ian provides a quick overview of how it works as well as some gotchas to look out for when trying this out.</p>\n<h5><a id=\\"serverlessstack_48\\"></a><strong>serverless-stack</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/4e\\" target=\\"_blank\\">serverless-stack</a></ins> Serverless Stack Toolkit is an open source tool that allows you to build serverless apps using Amazon Web Services CDK. Check out the detailed documentation with lots of examples <ins><a href=\\"https://aws-oss.beachgeek.co.uk/4f\\" target=\\"_blank\\">here</a></ins></p>\n<h5><a id=\\"awsautodns_52\\"></a><strong>awsautodns</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/4q\\" target=\\"_blank\\">awsautodns</a></ins> awsautodns is a tool that allows an Amazon Web Services EC2 Instance to automatically register it’s IP address in a Route53 Hosted Zone at startup. The install script creates a oneshot service, awsautodns.service, and sets it to start at boot, by installing it to multi-user.target. awsautodns is useful for avoiding fees associated with using an elastic IP address.</p>\n<h5><a id=\\"Amazon_Web_Servicesiotdeviceclient_56\\"></a><strong>Amazon Web Services-iot-device-client</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/4g\\" target=\\"_blank\\">Amazon Web Services-iot-device-client</a></ins> The Amazon Web Services IoT Device Client is an open-source, modular software written in C++ that you can compile and install on your Embedded Linux based IoT devices to access Amazon Web Services IoT Core, Amazon Web Services IoT Device Management, and Amazon Web Services IoT Device Defender features by default. It serves as a reference implementation for your IoT devices to work with Amazon Web Services IoT services, with operational best practices baked in</p>\n<h5><a id=\\"s3fetch_60\\"></a><strong>s3fetch</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/4h\\" target=\\"_blank\\">s3fetch</a></ins> s3fetch is a simple &amp; fast multi-threaded S3 download tool, with a breakdown on the project page of how they have achieved the speed. This is a different project from last week.</p>\n<h5><a id=\\"cloudfrontautoindex_64\\"></a><strong>cloudfront-autoindex</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/4i\\" target=\\"_blank\\">cloudfront-autoindex</a></ins> is a project from Artyom Pervukhin that solves a very particular pain point. Read the problem space that he has documented and see if you are having the same problems - he might just save you some time.</p>\n<h5><a id=\\"porter_68\\"></a><strong>porter</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/4j\\" target=\\"_blank\\">porter</a></ins> this looks like a very nice project that anyone familiar with the Heroku model will be interested in. Porter is a Kubernetes-powered PaaS that runs in your own cloud provider such as Amazon Web Services. Porter brings the Heroku experience to Kubernetes. Check out this <ins><a href=\\"https://aws-oss.beachgeek.co.uk/4k\\" target=\\"_blank\\">thread</a></ins> provides some additional discussion on this topic that is well worth reading.</p>\n<h5><a id=\\"amazonkinesisvideostreamsmediascreencastandroid_72\\"></a><strong>amazon-kinesis-video-streams-media-screencast-android</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/4o\\" target=\\"_blank\\">amazon-kinesis-video-streams-media-screencast-android</a></ins> this open source project will show you how you can screencast your mobile app using Kinesis Video Stream with WebRTC in Android. Chang Ho Jung has also put together a detailed blog post to walk you through the process, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/4p\\" target=\\"_blank\\">How to screencast apps using Amazon Kinesis Video Streams with WebRTC on Android</a></ins></p>\n<h4><a id=\\"Video_of_the_week_76\\"></a><strong>Video of the week</strong></h4>\\n<h5><a id=\\"Juypter_Notbooks_on_Amazon_Lightsail_77\\"></a><strong>Juypter Notbooks on Amazon Lightsail</strong></h5>\\n<p>This is a great new video blog from Julien Simon, always focused on helping you optimise whether that is performance or cost. In this post, he shows you how you can run Juypter notebooks on Amazon Lightsail for a very small amount indeed.</p>\n<p><video src=\\"https://dev-media.amazoncloud.cn/dccc5518521f4186b3b8c1657c0d8a42_Running%20Jupyter%20notebooks%20on%20Amazon%20Lightsail.mp4\\" controls=\\"controls\\"></video></p>\\n<h5><a id=\\"ROS_83\\"></a><strong>ROS</strong></h5>\\n<p>Check out this great video tutorial on how to run a simulation on Amazon Web Services RoboMaker. Perfect if you are just starting out in ROS and want to know some of the tools and terminology.</p>\n<p><video src=\\"https://dev-media.amazoncloud.cn/445cad11be1a4ea8a6d18eabcfce9b5e_Run%20a%20Simulation%20in%20AWS%20RoboMaker.mp4\\" controls=\\"controls\\"></video></p>\\n<h5><a id=\\"Blog_posts_from_the_community_and_customers_88\\"></a><strong>Blog posts from the community and customers</strong></h5>\\n<h5><a id=\\"Amazon_Web_Services_CDK_89\\"></a><strong>Amazon Web Services CDK</strong></h5>\\n<p>A couple of Amazon Web Services CDK posts this week. First up we have <ins><a href=\\"https://aws-oss.beachgeek.co.uk/48\\" target=\\"_blank\\">Web Services with Amazon Web Services CDK</a></ins> Mischa Spiegelmock with a post that brought a smile to my face, introducing an open source application called Cursed Webring, featuring “the worst of the internet”. So what I hear you say. Well, Mischa shows you how you can use the Amazon Web Services CDK to build this application. Full source code is available in the <ins><a href=\\"https://aws-oss.beachgeek.co.uk/49\\" target=\\"_blank\\">GitHub repository</a></ins> too. I guess the question I have is will this Newsletter ever make it into the Cursed Webring!</p>\n<p>Following that we have Amazon Web Services Community Hero Rehan van der Merwe with a post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/4a\\" target=\\"_blank\\">4 Methods to configure multiple environments in the Amazon Web Services CDK</a></ins> in which you will learn different methods that can be used to pass configuration values to the Amazon Web Services CDK. I will not give away what those four methods are, you will just have to read the post. Like all good open source posts, source code over at <ins><a href=\\"https://aws-oss.beachgeek.co.uk/4b\\" target=\\"_blank\\">his GitHub repo.</a></ins></p>\n<h5><a id=\\"Amazon_Web_Services_Amplify_96\\"></a><strong>Amazon Web Services Amplify</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/4c\\" target=\\"_blank\\">Building a Ron Swanson Quote App on Amplify with Javascript and GraphQL</a></ins> from my friends at Steamhaus, Paul Flowers with a great post with a hint of comedy (a winning combination) that shows you how easy it is to create a simple API that will be return Ron Swanson quotes to be displayed by a React frontend. Who (Why?) Ron Swanson, well only Paul will know that but this is a great fun post and a great way to learn how you can use Amazon Web Services Amplify to quickly put something like this together.</p>\n<h5><a id=\\"Linux_OS_for_Kubernetes_100\\"></a><strong>Linux OS for Kubernetes</strong></h5>\\n<p>In this post from Talos Systems CEO, Steve Francis, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/40\\" target=\\"_blank\\">A Guide to Linux Operating Systems for Kubernetes</a></ins> he provides some insights into the specific needs around Linux operating systems within Kubernetes. If you want to know a little more as to what the Amazon Web Services open source project Bottlerocket is helping to address, read this post and hopefully it will be clearer.</p>\n<h5><a id=\\"Grafana_104\\"></a><strong>Grafana</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/4m\\" target=\\"_blank\\">Monitoring Amazon Web Services Elastic Beanstalk Memory and Disk Usage in Grafana</a></ins> Emile Nijssen shows you how easy it is to setup a Grafana dashboard to display your CloudWatch metrics, in this case, metrics that the CloudWatch agent in an Elastic Beanstalk instance is generating.</p>\n<h4><a id=\\"Amazon_Web_Services_open_source_posts_108\\"></a><strong>Amazon Web Services open source posts</strong></h4>\\n<h5><a id=\\"Bottlerocket_109\\"></a><strong>Bottlerocket</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/47\\" target=\\"_blank\\">How the Bottlerocket build system works</a></ins> is great post from Matt Briggs on Bottlerocket. Bottlerocket is an open source, special-purpose operating system designed for hosting Linux containers. This post dives deep into the build system for Bottlerocket, taking a look at how it uses Cargo, the Rust package manager and some of the tooling used. This is a great way to understand in more detail how this works for your own Rust projects potentially.</p>\n<h5><a id=\\"MLflow_113\\"></a><strong>MLflow</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/3x\\" target=\\"_blank\\">Managing your machine learning lifecycle with MLflow and Amazon SageMaker</a></ins> Sofian Hamiti and Shreyas Subramanian take a look at an open source solution to helping data scientists manage their machine learning experiments. I took a look at this early in the year and hope to do some blog posts around it soon. In the meantime, this post shows how the open-source platform MLflow helps data scientists collaborate and share experiment results in order to find a solution to a business need. Great post, so read on to find out more about MLflow and how you can integrate this with Amazon SageMaker.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/a82b361157df4f0985112fdcde9d89ff_image.png\\" alt=\\"image.png\\" /></p>\n<h5><a id=\\"Deep_Java_Library_119\\"></a><strong>Deep Java Library</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/46\\" target=\\"_blank\\">Model serving in Java with Amazon Web Services Elastic Beanstalk made easy with Deep Java Library</a></ins> Frank Liu has put together this guide to show you how to deploy a model on Elastic Beanstalk using the Deep Java Library (DJL), using an example of sending an image through a post call to get inference results on what the image contains.</p>\n<h5><a id=\\"Apache_Airflow_123\\"></a><strong>Apache Airflow</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/3y\\" target=\\"_blank\\">Orchestrating analytics jobs on Amazon EMR Notebooks using Amazon MWAA</a></ins> Fei Lang and Ray Liu have put together another great post that shows how to use Amazon Managed Workflows for Apache Airflow (Amazon MWAA) to orchestrate analytics jobs on EMR Notebooks. They walk you through an end to end solution and is essential reading if you are just getting to grips with what Apache Airflow can do for you.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/d825a8315e6942a8a53ce76f402a7a62_image.png\\" alt=\\"image.png\\" /></p>\n<h5><a id=\\"Apache_Cassandra_129\\"></a><strong>Apache Cassandra</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/41\\" target=\\"_blank\\">Compress and conquer with Amazon Keyspaces (for Apache Cassandra)</a></ins> Nikolai Kolesnikov with a helpful post that shows you how to compress your data using freely available compression tools and store that compressed data in Amazon Keyspaces. To help deliver fast performance, Amazon Keyspaces has a 1 MB row-size quota. What if you already have rows larger than 1 MB in your existing Cassandra tables? To reduce the size of these rows, you can compress one or more large columns. Compressing large columns reduces your storage costs, improves performance, reduces I/O and network usage, and enables you to fit the data within the Amazon Keyspaces row quota. At that is what Nikolai will walk you through in this post.</p>\n<h5><a id=\\"Apache_Spark_133\\"></a><strong>Apache Spark</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/44\\" target=\\"_blank\\">Running cost optimized Spark workloads on Kubernetes using EC2 Spot Instances</a></ins> Kinnar Sen takes a look at how to run Apache Spark workloads in a resilient, scalable and cost optimised way using Amazon EKS service in conjunction with Spot Instances. Apache Spark is an open-source, distributed processing system used for big data workloads, and supports Kubernetes as one of its cluster managers allowing you to allocate and schedule your analytics resources. This is a detailed walkthrough, so set some time aside as you will cover a lot of ground.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/0f75431ac1b84c128e6bc0d2f60b6517_image.png\\" alt=\\"image.png\\" /></p>\n<h5><a id=\\"Fluent_Bit_140\\"></a><strong>Fluent Bit</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/42\\" target=\\"_blank\\">Fluent Bit Integration in CloudWatch Container Insights for EKS</a></ins> Ugur KIRA, Dejun Hu and TP Kohli collaborate on this post that introduces CloudWatch Container Insights, which enables you to explore, analyze, and visualize your container metrics, Prometheus metrics, application logs, and performance log events through automated dashboards in the CloudWatch console. Fluent Bit is an open source and multi-platform log processor and forwarder that allows you to collect data and logs from different sources, and unify and send them to different destinations including CloudWatch Logs. In this post find out how these two integrate to help you considerably enhances your observability and debugging capabilities in your container workloads.</p>\n<h5><a id=\\"PostgreSQL_144\\"></a><strong>PostgreSQL</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/43\\" target=\\"_blank\\">Getting more with PostgreSQL purpose-built data types</a></ins> Jim Mlodgenski explores some of PostgreSQL’s purpose-built types, covering UUID and Network Addresses, Geospatial and Chemistry. This is just the tip however, and as Jim says in this post “PostgreSQL’s extensible architecture will allow new types to be created in the future as PostgreSQL moves into more industries requiring specialized data handling.”</p>\n<h5><a id=\\"Amazon_Web_Services_SAM_148\\"></a><strong>Amazon Web Services SAM</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/45\\" target=\\"_blank\\">Building a Jenkins Pipeline with Amazon Web Services SAM</a></ins> Tarun Kumar is here with a post that shows how to set up a multi-stage pipeline on a Jenkins host for a serverless application, using the Amazon Web Services Serverless Application Model (Amazon Web Services SAM).</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/139fdbb3999d4f58b1de70fed0a9183c_image.png\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"Amazon_Web_Services_Partners_154\\"></a><strong>Amazon Web Services Partners</strong></h4>\\n<h5><a id=\\"ancher_and_Amazon_EKS_155\\"></a><strong>ancher and Amazon EKS</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/3z\\" target=\\"_blank\\">Optimizing Your Kubernetes Clusters with Rancher and Amazon EKS</a></ins> Keith Andruch and Andrew Park take a look at how combining Rancher with Amazon EKS provides a solution that allows enterprises to overcome common technical challenges associated with Kubernetes, including learning and resourcing gaps, onboarding, and operational difficulties.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/a0988909369d4b5e880340f2eeb6f1f9_image.png\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"Amazon_Web_Services_Case_Studies_161\\"></a><strong>Amazon Web Services Case Studies</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/3t\\" target=\\"_blank\\">Expedia Group Speeds Up Vendor Payment Transactions from 1 Day to Seconds</a></ins> is a case study on how they are using open source technologies, and how these are helping them break free of legacy, proprietary databases in order to achieve significant business value. To help migrate they are using a tool that has an open source community edition, Flyway. Check them out if you are also thinking about migrating and want to explore your database migration options.</p>\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/4l\\" target=\\"_blank\\">How INESC TEC tests and deploys ROS applications with Amazon Web Services RoboMaker</a></ins> Rafael Arrais, Project Manager at INESC TEC describes how robotics researchers at INESC TEC in Portugal are using Amazon Web Services RoboMaker to set up a simulation-based regression testing CI/CD pipeline, showing you how they used simulation to scale testing, speed up dev velocity and reduce errors.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/f8d6712aa0b54ec7bb5bbe1a2071a48a_demo.gif\\" alt=\\"demo.gif\\" /></p>\n<h4><a id=\\"Quick_updates_168\\"></a><strong>Quick updates</strong></h4>\\n<h5><a id=\\"PostgreSQL_169\\"></a><strong>PostgreSQL</strong></h5>\\n<p>Amazon Aurora with PostgreSQL compatibility now supports major version 12. PostgreSQL 12 includes better index management, improved partitioning capabilities, and the ability to execute JSON path queries per SQL/JSON specifications. It also enables nondeterministic collations that support case-insensitive and accent-insensitive comparisons for ICU provided collations, most common-value statistics for improved query plans, creation of generated columns that compute values with an expression, and many additional features.</p>\n<p>This release also updates extensions including Address_standardizer, Address_standardizer_data_us, Amcheck, Citext, Hll, Hstore, Ip4r, Pg_repack, Pg_stat_statements, Pgaudit, Pglogical, Pgrouting, Plv8, Postgis, Postgis_tiger_geocoder, Postgis_topology.</p>\n<h5><a id=\\"In_place_upgrades_175\\"></a><strong>In place upgrades</strong></h5>\\n<p>You can perform an in-place upgrade of your Amazon Aurora database cluster from PostgreSQL major version 11 to 12. Instead of backing up and restoring the database to the new version, you can upgrade with just a few clicks in the Amazon RDS Management Console or by using the Amazon Web Services SDK or CLI.</p>\n<h5><a id=\\"Fluent_Bit_179\\"></a><strong>Fluent Bit</strong></h5>\\n<p>Amazon Web Services announces the availability of Fluent Bit support for Amazon CloudWatch, a fully managed, pay-as-you-go monitoring and observability service for resources running on Amazon Web Services and on-premises. CloudWatch Container Insights and Log Insights enables you to explore, analyse, and visualise your container logs collected from the Fluent Bit processor. Container Insights also provides access to the automated dashboards that summarise the performance and availability of the FluentBit Daemon set running on Amazon Elastic Kubernetes Service (EKS) and Kubernetes. With few clicks in the Amazon Web Services Management Console, you can start collecting Fluent Bit logs in CloudWatch to monitor and troubleshoot containers.</p>\n<h5><a id=\\"Elasticsearch_184\\"></a><strong>Elasticsearch</strong></h5>\\n<p>Amazon Elasticsearch Service now supports encryption of data at rest and node-to-node encryption on existing domains, enabling organisations hosting sensitive workloads to meet stringent security and compliance requirements.</p>\n<p>Read more about the announcement, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/3s\\" target=\\"_blank\\">Amazon Elasticsearch Service extends encryption at rest and node-to-node encryption to existing domains</a></ins></p>\n<h4><a id=\\"Stay_in_touch_with_open_source_at_Amazon_Web_Services_190\\"></a><strong>Stay in touch with open source at Amazon Web Services</strong></h4>\\n<p>I hope this summary has been useful. Remember to check out the <ins><a href=\\"https://aws.amazon.com/opensource/?opensource-all.sort-by=item.additionalFields.startDate&amp;opensource-all.sort-order=asc\\" target=\\"_blank\\">Open Source homepage</a></ins> to keep up to date with all our activity in open source by following us on <ins><a href=\\"https://twitter.com/AWSOpen\\" target=\\"_blank\\">@AWSOpen</a></ins>.</p>\n"}