Open source news and updates, #70

{"value":"### **1st June, 2021 - Instalment #70**\nDue to it being a bank holiday in the UK, Newsletter #70 is a day later than usual.\n\nMake sure you check out this weeks \"Quick Updates\" as there are some updates that I know a lot of people will be happy about, and I always share great nuggets that sometimes folk miss. More great projects this week, from iamzero which helps you simplify your identity and access management, PMapper that you will find helpful for identifying risks in your IAM configurations, an Amazon Web Services CDK construct to help you deploy a VSCode server, Orkestra, a super nice project that lets you create workflows using Lambda functions a la Apache Airflow and many more. You will find blog posts from Amazon Web Services and the community covering Apache Airflow, Apache Spark, Apache Zeppelin, TensorFlow, Packer, lots of containers goodies and more. Finally, do not forget the events section, which is a great seg-way into the biggest news last week...\n\n#### **Open@Amazon**\n\nLast week we announced Open@Amazon, a celebration of open source on Amazon Web Services with a fabulous cast of speakers, a fantastic broad set of topics and the event will be fully live so you can get your chance to interact with the speakers and the broader open source and Amazon Web Services community. Check out the blog post, ++[What’s up with open source at Amazon Web Services? Attend Open@Amazon live on Twitch June 16](https://aws-oss.beachgeek.co.uk/jo)++ or just register to get a handy calendar invite ++[HERE](https://aws-oss.beachgeek.co.uk/jn)++\n\n\n#### **DevAx connect**\n\nAlso, later this week we have a new regular open source series by developers for developers that will be streamed live via Twitch, hosted by the awesome Heitor Lessa. The first Thursday of every month you will be able to join Heitor and invited guests as they take you through a variety of interesting open source topics, projects and more. This week we have Lars Jacobsson, and I am looking forward to it. Check out the events section for more details.\n\n#### **Please help me improve this newsletter**\n\nThank you to those who have provided such great feedback already, I am currently reviewing and thinking how to incorporate the suggestions. I would still love to hear from more of you, and I have more Amazon Web Services credit vouchers as a thank you for taking the time to complete the survey. The first 20 will get an Amazon Web Services credit voucher for \$25, so thank you again. It is only 5 questions, and will probably only take you a minute or two to complete.\\n\\n++[Take me to the survey!](https://aws-oss.beachgeek.co.uk/jp)++\\n\\n#### **Celebrating open source contributors**\\n\\nThe articles posted in this series are only possible thanks to contributors and project maintainers and so I would like to shout out and thank those folks who really do power open source and enable us all to build on top of what they have created.\\n\\nSo thank you to the following open source heroes: Stephan Fitzpatrick, Daniel Burkhardt, Malte Luecken, Mrudhula Balasubramanyan, Imaya Kumar Jagannathan, Joerg Woehrle, Brad Kashani, Carl Youngblood, Danilo Poccia, Christian Weber, Alolita Sharma, Nizar Tyrewalla, Josiah Davis, Yin Song, Anne Hu, Joshua McKiddy, Olivier Cruchant, Arunprasath Shankar, Mark Roy, Eli Fisher, Luke Kysow, Devarshi Shah, Horace Nelson, Brian Schiller, Mike Apted, Serhii Vasylenko, Ibrahim Cesar, Niko Virtala, Troy Ameigh and Andrew Glenn.\\n\\nMake sure you find and follow these builders and keep up to date with their open source projects and contributions.\\n\\n#### **Latest from open source projects**\\n#### **orkestra**\\n\\n++[orkestra](https://aws-oss.beachgeek.co.uk/kw)++ this new open source project from Stephan Fitzpatrick looks very promising, allowing you to create workflow tasks via Lambda functions and then orchestrate these similar to how Apache Airflow works. The ++[documentation](https://aws-oss.beachgeek.co.uk/kx)++ says \\"a seamless way of building observable (scheduled or event-driven) cloud-native workflows. It aims to bring a similar development experience to that of Airflow while leveraging the full power of Amazon Web Services.\\"\\n\\nThis is a project that is going to get some closer inspection from myself for sure.\\n\\n\\n\\n#### **iamzero**\\n\\n++[iamzero](https://aws-oss.beachgeek.co.uk/kk)++ is an open source tool that helps simplify your identity and access management on Amazon Web Services. IAM Zero detects identity and access management issues and automatically suggests least-privilege policies. Check out the ++[newly formed website](https://aws-oss.beachgeek.co.uk/km)++ and business that has started to help customers who want support from this tool, and you can read more about the backstory in this ++[post from Reddit](https://aws-oss.beachgeek.co.uk/kl)++. As per the Reddit post:\\n\\nThe tool is in a similar space to iamlive, policy_sentry, and consoleme (all of which are worth checking out too if you're interested in making Amazon Web Services security easier) but the main points of difference I see are:\\n\\n- iam-zero can run transparently on any or all of your roles just by swapping your Amazon Web Services SDK import to the iam-zero instrumented version or using the instrumented CLI\\n\\n- iam-zero can run continuously as a service (deployed into a isolated Amazon Web Services account in an organization behind an SSO proxy) and could send notifications through Slack, email etc\\n\\n- iam-zero uses TLS to dispatch events and doesn't include any session tokens in the dispatched event (Amazon Web Services Client Side Monitoring, which iamlive utilises, includes authentication header details in the event - however iamlive is awesome for local policy development)\\n\\n#### **PMapper**\\n\\n++[PMapper](https://aws-oss.beachgeek.co.uk/kn)++ Principal Mapper (PMapper) is a script and library for identifying risks in the configuration of Amazon Web Services Identity and Access Management (IAM) for an Amazon Web Services account or an Amazon Web Services organization. It models the different IAM Users and Roles in an account as a directed graph, which enables checks for privilege escalation and for alternate paths an attacker could take to gain access to a resource or action in Amazon Web Services. PMapper includes a querying mechanism that uses a local simulation of Amazon Web Services's authorization behavior. When running a query to determine if a principal has access to a certain action/resource, PMapper also checks if the user or role could access other users or roles that have access to that action/resource. This catches scenarios such as when a user doesn't have permission to read an S3 object, but could launch an EC2 instance that can read the S3 object.\\n\\n#### **aurora-serverless-to-s3**\\n\\n++[aurora-serverless-to-s3](https://aws-oss.beachgeek.co.uk/ki)++ this open source project from Devetry simplifies the automation necessary to export Amazon RDS Aurora Serverless snapshots to S3 for a specific database whenever an automated snapshot is created. The project outlines how it achieves its task, provides clear examples and an Amazon Web Services CDK app to help easily deploy this project. Brian Schiller has put together a short blog post, ++[Aurora Serverless DB Export](https://aws-oss.beachgeek.co.uk/kj)++ that provides some additional context. If you want to see a demo, check this short video\\n\\n<video src=\\"https://dev-media.amazoncloud.cn/e528885a85a647a0884f0fd47b5c0e92_Amazon%20RDS%20Snapshot%20Export%20to%20S3.mp4\\" class=\\"manvaVedio\\" controls=\\"controls\\" style=\\"width:160px;height:160px\\"></video>\\n\\n#### **Image-Flex**\\n\\n++[Image-Flex](https://aws-oss.beachgeek.co.uk/kg)++ is an open sourced solution from Horace Nelson that provides a robust image resizing service built on Amazon Web Services Serverless technologies and used to resize, optimise, and cache images on \\"the edge,\\" on the fly. If you want to know more about how you might use or deploy it, he has helpfully put together a detailed blog post, ++[An Open Source and Completely Serverless Image Resizing Service in Amazon Web Services](https://aws-oss.beachgeek.co.uk/kh)++\\n\\n\\n\\n\\n#### **ecs-anywhere-tutorial**\\n\\n++[ecs-anywhere-tutorial](https://aws-oss.beachgeek.co.uk/ke)++ last week one of the big announcements was the general availability of ++[Amazon ECS Anywhere](https://aws-oss.beachgeek.co.uk/kf)++, and this project/tutorial is intended to walk you through an opinionated demonstration of how ECS Anywhere works. The initial steps will show you how to deploy a (somewhat) sophisticated multi services application in an Amazon Web Services region as an ECS service running on Amazon Web Services Fargate. Further in the tutorial, the steps will guide you through how to deploy parts of this application on ECS Anywhere managed instances in a customer managed infrastructure outside of the Amazon Web Services region. If you saw Massimo's presentation and demo, you will know how nice this was and I am looking forward to dusting off my Raspberry Pi's and trying this out for myself.\\n\\n\\n\\n#### **Amazon Web Services-do-docker**\\n\\n++[Amazon Web Services-do-docker](https://aws-oss.beachgeek.co.uk/kd)++ this project provides a simple pattern for building and deployment of Docker containers to Amazon Web Services. It can be used \\"as is\\" to build and run a sample container, or can be customized and used to build, test, and deploy containerized microservices, jobs, or serverless functions. Containers built using this project can be run locally or on a number of Amazon Web Services compute environments by a simple change to the configuration files.\\n\\n\\n\\n#### **serverless-rules**\\n\\n++[serverless-rules](https://aws-oss.beachgeek.co.uk/kv)++ is new project that provides a compilation of rules to validate infrastructure as code template against recommended practices. This currently provides a module for cfn-lint and a plugin for tflint. This is currently in public preview, so if you do try it out please feedback anything you find.\\n\\n#### **cdk-vscode-fargate**\\n\\n++[cdk-vscode-fargate](https://aws-oss.beachgeek.co.uk/ko)++ this was one of my favourite projects that I came across last week, although I have not had a chance to test it out yet. cdk-vscode-fargate is a JSII construct library for Amazon Web Services CDK from Mike Apted that allows you to deploy Code-server running VS Code remotely, on a Amazon Web Services Fargate container. The project provides you with a quick how-to guide to get you started, so hopefully something I can try over the weekend. You can find the source code in the GitHub repository ++[here](https://aws-oss.beachgeek.co.uk/kp)++\\n\\n\\n\\n#### **mwaa-local-runner**\\n\\n++[mwaa-local-runner](https://aws-oss.beachgeek.co.uk/kc)++ I shared this project a few weeks ago (#66) but that was for Apache Airflow 1.x, and the project has been quickly updated to align with the launch last week of Apache Airflow version 2.0.2 support in Amazon Managed Workflows for Apache Airflow (MWAA). Everything is the same, just now using the latest version.\\n\\n#### **S3AssetDeploy**\\n\\n++[S3AssetDeploy](https://aws-oss.beachgeek.co.uk/kb)++ if you are a Ruby developer then you are going to find this open source tool from Loomly very nice indeed. The folks at Loomly use this to safely package up and deploy their web assets to Amazon S3 that are being served via Cloudfront. This gem is designed to upload and clean unneeded assets from S3 in a safe manner such that older versions or recently removed assets are kept on S3 during the rolling deploy process. It also maintains a version limit and TTL (time-to-live) on assets to avoid deleting recent and outdated versions (up to a limit) or those that have been recently removed. Nice detailed docs and clear examples too.\\n\\n#### **amazonlinux-dind**\\n\\n++[amazonlinux-dind](https://aws-oss.beachgeek.co.uk/kt)++ this repo from Niko Virtala is an Amazon Linux 2 container image with systemd and Docker Engine, and as Niko said in his tweet:\\n\\nI built a #Docker in Docker image that meets the #ECS Agent systemd requirement, so you don't have to – you can find it from all well-stocked container registries and #GitHub\\n\\n### **Tweet of the week**\\nWe have not had a Tweet of the week for a while, but last week there were a couple that caught my eye and worth sharing.\\n\\n#### **Blender**\\n\\nThis ++[tweet](https://aws-oss.beachgeek.co.uk/k3)++ shared by John Mark, linked to an example of how the Amazon Web Services open source teams are working with open source projects and foundations to strengthen upstream communities.\\n\\n#### **Amazon Web Services Copilot vs**\\n\\nThis ++[tweet](https://aws-oss.beachgeek.co.uk/k4)++ from Jonathan Matthews asking about using Amazon Web Services Copilot vs the Amazon Web Services cli started a great discussion and it was good to hear from the likes of Massimo, Romain and Efe. Massimo shared a ++[response](https://aws-oss.beachgeek.co.uk/k5)++ but make sure you check out the thread.\\n\\n#### **Amazon Web Services CDK**\\n\\nHenrique Lima had ++[an ask](https://aws-oss.beachgeek.co.uk/k6)++ this week for folks who are using Amazon Web Services CDK. He is specifically looking for developers interested in checking or automating compliance of infrastructure changes, to provide 30 minutes to ++[complete a survey](https://aws-oss.beachgeek.co.uk/k7)++ to help him as he works on a Change Analysis tool for his Master's thesis to do just that.\\n\\n### **Community open source posts**\\n#### **GraakVM**\\n\\n++[Converting Java Lambda functions to GraalVM native-image](https://aws-oss.beachgeek.co.uk/ks)++ Mark Sailes with a quick guide on how to create binary files that you can use in your Lambda functions, showing you how to configure your maven files to do this.\\n\\n#### **OpenSearch**\\n\\nEli Fisher shared the new public roadmap for the OpenSearch project, hosted in GitHub. The roadmap shows the next several months of planned features and releases. Going forward, this roadmap will show what the maintainers and contributors of the various components of the project intend to work on and when that work aims to launch. Read more in the post, ++[The OpenSearch roadmap is now available on GitHub](https://aws-oss.beachgeek.co.uk/k8)++\\n\\n\\n\\n#### **Consul Service Mesh**\\n\\n++[Announcing Tech Preview of Consul Service Mesh for Amazon ECS](https://aws-oss.beachgeek.co.uk/k9)++ Luke Kysow and Devarshi Shah wrote last week about the availability (at this stage it is a tech preview) of HashiCorp Consul service mesh for Amazon ECS, with this tech preview letting you deploy non-production versions of Consul for testing on the Fargate launch type for ECS. The post covers how it works and provides a simple Web and API example to demonstrate this further.\\n\\n\\n\\n#### **Anisible**\\n\\nIn the past week (and indeed this week) there have been a number of posts around how you can use open source tools as part of your CI/CD pipeline to build your applications for the Mac operating systems. In this post, ++[Run Ansible playbook on mac1.metal instances fleet with Amazon Web Services Systems Manager](https://aws-oss.beachgeek.co.uk/ka)++ Amazon Web Services Community Builder Serhii Vasylenko shows you how you can use Ansible in conjunction with Amazon Web Services Services.\\n\\n#### **Reddis**\\n\\nAnother Amazon Web Services Commuinity Builder, Ibrahim Cesar has put this post, ++[Redis: Exploring Redis as Serverless Database to solve idempotence in APIs](https://aws-oss.beachgeek.co.uk/kr)++ which is part of a series, but in this he explores what Reddis is and how to get started with it.\\n\\n### **Amazon Web Services and Amazon open source posts**\\n#### **Assisted Log Enabler**\\n\\nIn my #67 newsletter I introduced you to this open source project from Amazon Web Services, ++[assisted-log-enabler-for-Amazon Web Services](https://aws-oss.beachgeek.co.uk/h7)++ an interesting tool that helps customers manage logging for various services. Joshua McKiddy has put together this blog post, ++[Introducing Assisted Log Enabler for Amazon Web Services](https://aws-oss.beachgeek.co.uk/k0)++ showing you how to use Assisted Log Enabler for Amazon Web Services to ensure logging is turned on within an Amazon Web Services environment. Make sure you check this out.\\n\\n\\n\\n#### **Amazon Web Services Distro for OpenTelemetry**\\n\\n++[Amazon Web Services Distro for OpenTelemetry 0.10.0 is now available with Amazon Web Services Lambda layers for .Net](https://aws-oss.beachgeek.co.uk/jy)++ Alolita Sharma and Nizar Tyrewalla provide a quick update on the availability of version 0.10.0 of Amazon Web Services Distro for OpenTelemetry (ADOT). This is now available in Amazon Web Services Lambda layers for Amazon Web Services X-Ray support in .Net. The latest versions of Amazon Web Services Lambda layers with Amazon Web Services X-Ray support are now available for the OpenTelemetry Collector, Java, Java instrumentation, JavaScript, and Python. The post also covers news of some updated sample applications as well as news that StatsD is now supported within the ADOT Collector. Read the post for the full details.\\n\\n#### **Vim, tmux and Zsh**\\n\\nAs an extensive user of these tools, this post brought a huge smile to my face. In, ++[Enhancing data science environments with Vim, tmux, and Zsh on Amazon EC2](https://aws-oss.beachgeek.co.uk/jz)++ Josiah Davis, Yin Song, and Anne Hu share how you can use these tools to enable data scientists to be more productive in their day to day working environment. From providing Cloudformation templates that enable you to get this up and running yourself, to providing some great examples, this post is your must read post this week. Whether you are a data scientist or not, you can take some of the tips shared here to help you work smarter. Very nice indeed.\\n\\n\\n\\n#### **Apache Zeppelin**\\n\\nLaunched last week, Amazon Kinesis Data Analytics Studio uses a number of open source projects to help make it easier for customers to analyse streaming data. You can now easily launch Apache Zeppelin notebooks, and interact and work with streaming data using SQL, Python or Scala. In, ++[Introducing Amazon Kinesis Data Analytics Studio – Quickly Interact with Streaming Data Using SQL, Python, or Scala](https://aws-oss.beachgeek.co.uk/jw)++ Danilo Poccia provides you an overview of the features as well as a follow along example to help get you orientated.\\n\\n\\n\\nMy favourite part of this blog post is where Danilo writes:\\n\\n\\"In Kinesis Data Analytics Studio, we run the open-source versions of Apache Zeppelin and Apache Flink, and we contribute changes upstream. For example, we have contributed bug fixes for Apache Zeppelin, and we have contributed to Amazon Web Services connectors for Apache Flink, such as those for Kinesis Data Streams and Kinesis Data Firehose. Also, we are working with the Apache Flink community to contribute availability improvements, including automatic classification of errors at runtime to understand whether errors are in user code or in application infrastructure.\\"\\n\\n#### **Apache Spark**\\n\\nBigstream is an Amazon Web Services ISV Partner and pioneer in the field of hardware and software level acceleration. Brad Kashani, CEO at Bigstream, shares in this post, ++[Bigstream Provides Big Data Acceleration with Apache Spark and Amazon EMR](https://aws-oss.beachgeek.co.uk/ju)++ why acceleration has better potential than CPU scaling, the historic challenges organisations have had in implementing accelerators, and how Bigstream are able to address those for a seamless, accelerated Spark experience.\\n\\n\\n\\n#### **Packer**\\n\\nPacker is an open source tool from HashiCorp for building identical machine images for multiple platforms from a single source configuration. In this post, ++[Building Amazon Machine Images (AMIs) for EC2 Mac instances with Packer](https://aws-oss.beachgeek.co.uk/jt)++ Joerg Woehrle shares best practices for building custom AMIs for EC2 Mac instances using HashiCorp Packer. Joerg covers how to create an AMI of an EC2 Mac instance, understake administrative tasks such as resizing the hard drive, resetting ec2-macos-init and installing some base software (golang) onto the image as well as demonstrating how to facilitate SSM to connect to an EC2 instance without the need to open any ports or a public IP. Read on to dive deeper...\\n\\n\\n\\n\\n#### **git**\\n\\n++[Use Git pre-commit hooks to avoid Amazon Web Services CloudFormation errors](https://aws-oss.beachgeek.co.uk/ku)++ Troy Ameigh and Andrew Glenn with a post that walks you through how you can use GitHub pre-commit hooks with a number of open source tools to help address a number of common issues when using Amazon Web Services Cloudformation. The post takes a look at some of those open source tools and how you can use them to help you with things like checking shell scripts, validating end of line, dealing with white space, enforcing style and much more.\\n\\n\\n\\n#### **Amazon Web Services CDK**\\n\\nChristian Weber provides another one of his regular monthly updates covering all things Amazon Web Services CDK related, in his ++[CDK Corner – May 2021](https://aws-oss.beachgeek.co.uk/jx)++. Plenty of highlights, including updates from the recent CDK Day, news on the CDK v2 developer preview, Amazon Web Services CDK for Go developer preview and a nice list of updated/new constructs you should check out. This is essential reading in my opinion, and hats off to all those listed as having their first PRs merged. Please get in touch, I have an Amazon Web Services Credit voucher waiting for you..\\n\\n#### **TensorFlow**\\n\\nA couple of posts this week on TensorFlow. The first post covers how to optimise the inference latency of TensorFlow Serving in the post, ++[How Contentsquare reduced TensorFlow inference latency with TensorFlow Serving on Amazon SageMaker](https://aws-oss.beachgeek.co.uk/k1)++. Olivier Cruchant shares how one customer was able to experiment with a number of approaches to find an optimum solution for hosting their inference endpoints.\\n\\n\\n\\nFollowing that we have the post, ++[Host multiple TensorFlow computer vision models using Amazon SageMaker multi-model endpoints](https://aws-oss.beachgeek.co.uk/k2)++, where Arunprasath Shankar and Mark Roy demonstrate how to use SageMaker multi-model endpoints to host two computer TensorFlow vision models with different model architectures and datasets for image classification. They provide the sample notebook so you can try this for yourself.\\n\\n\\n\\n#### **Open Problems**\\n\\nOpen Problems provides a framework for researchers to define formalized tasks in single-cell genomics and benchmark model performance on those tasks using Amazon Web Services. In the blog post, ++[Driving innovation in single-cell analysis on Amazon Web Services](https://aws-oss.beachgeek.co.uk/jr)++ Daniel Burkhardt, Malte Luecken, and Mrudhula Balasubramanyan take a look in more detail how they are running this on Amazon Web Services, looking at the architecture of the platform, benchmarking computational methods for single-cell analysis and more.\\n\\n\\n\\n#### **Ethereum**\\n\\nAmazon Managed Blockchain support for managed Ethereum nodes makes it possible to build your own decentralized applications (dapps) without having to worry about maintaining reliable Ethereum nodes for relaying blockchain transactions or querying the state of blockchain data. In this post, ++[Deploy an Ethereum node on Amazon Managed Blockchain](https://aws-oss.beachgeek.co.uk/jv)++ Carl Youngblood walks you through setting up and connecting to an Ethereum node with Managed Blockchain\\n\\n### **Quick updates**\\n#### **Apache Airflow**\\n\\nYou can now launch Apache Airflow 2.0 environments on Amazon Managed Workflows for Apache Airflow (MWAA). Apache Airflow 2.0 is the latest version of the popular open-source tool that helps customers author, schedule, and monitor workflows.\\n\\n#### **PostgreSQL**\\n\\nAmazon Web Services Database Migration Service (Amazon Web Services DMS) has expanded functionality by adding support for PostgreSQL version 13 in preview mode. Using Amazon Web Services DMS, you can now perform live migrations from any Amazon Web Services DMS supported sources to PostgreSQL 13 databases, and from PostgreSQL 13 databases to any Amazon Web Services DMS supported targets.\\n\\n#### **Apache Kakfa**\\n\\nAmazon Managed Streaming for Apache Kafka (Amazon MSK) now supports Apache Kafka version 2.7.1 for new and existing clusters. Apache Kafka 2.7.1 includes several bug fixes. For a complete list of fixes, see the Apache Kafka ++[release notes for 2.7.1](https://aws-oss.beachgeek.co.uk/jq)++.\\n\\n#### **Apache Cassandra**\\n\\nA couple of quick updates this week.\\n\\nFirst up is news that Amazon Keyspaces (for Apache Cassandra), a fully managed Apache Cassandra–compatible database service, is now Health Insurance Portability and Accountability Act of 1996 (HIPAA) eligible to help you run healthcare workloads more easily. Amazon Web Services enables covered entities and their business associates subject to HIPAA to use the secure Amazon Web Services environment to process, maintain, and store protected health information. Now, you can run Cassandra workloads that are subject to HIPAA compliance more easily by using a fully managed and serverless database service. Amazon Keyspaces helps you secure PHI through features such as encryption, audit logging via Amazon Web Services CloudTrail, and access management through Amazon Web Services Identity and Access Management.\\n\\nFollowing that is news you can manage service quotas for Amazon Keyspaces (for Apache Cassandra), through the Service Quotas console. Using the Service Quotas console, you can view and manage quotas (formerly referred to as limits) easily and at scale as your Amazon Web Services workloads grow. For example, you can manage Amazon Keyspaces quotas such as account-level throughput, keyspaces and tables per Amazon Web Services Region, and concurrent Data Definition Language (DDL) operations. Service Quotas makes requesting quota increases easier: search for a quota, enter your desired value, and submit a quota increase request. You can manage your quotas proactively by configuring Amazon CloudWatch alarms that monitor usage and alert you to approaching quotas.\\n\\n#### **Apache Spark 3.1.1**\\n\\nAmazon EMR 6.3 release version now supports Apache Spark 3.1.1 and provides runtime performance improvements with EMR Runtime for Spark. Amazon EMR 6.3 also supports Apache Hudi 0.7.0, Flink 1.12.1, PrestoDB 0.245.1, PrestoSQL 350, Hue 4.9, JupyterHub 1.2, Oozie 5.2.1 and TensorFlow 2.4.1.\\n\\n#### **JetBrains**\\n\\nThe Amazon Web Services Toolkit for JetBrains now provides developers with convenient IDE functionality to create and manage deployments from their code or image repositories using Amazon Web Services AppRunner. Amazon Web Services App Runner that was recently announced is a new service that makes it easy for customers without any prior containers or infrastructure experience to build, deploy, and run containerised web applications and APIs in just a few clicks.\\n\\nWith new App Runner features in the Amazon Web Services Toolkit for JetBrains, users of any JetBrains IDE can configure and deploy web services, track deployment status (i.e. view deployment notification logs) and monitor state of deployed services (i.e., services active, running, or paused), all without leaving their IDEs.\\n\\n#### **Lustre**\\n\\nAmazon FSx for Lustre, a service that provides cost-effective, high-performance, scalable file storage for compute workloads, now supports data compression. Data compression enables you to reduce storage consumption of both your file system storage and your file system backups. The data compression feature is designed to deliver high levels of compression without adversely impacting file system performance. Once data compression is enabled, newly written files are automatically compressed by FSx for Lustre before they are written to disk and automatically uncompressed when they are read. In addition to compression, Amazon FSx for Lustre offers a range of storage options to help you reduce costs, including a variety of SSD and HDD storage options, along with an option to store data on unreplicated storage for short-term processing of data.\\n\\n### **Videos of the week**\\n#### **Observability**\\n\\nImplementing observability in applications a requirement for achieving Operational Excellence, and a well implemented observability plan allows customers to react to operational events, run workloads effectively and gain insights into their applications. In this video, Imaya Kumar Jagannathan will walk you through how to use Amazon managed open source tools for observability, how to use Amazon Managed Prometheus to monitor the performance of containerised workloads and how Amazon Managed Service for Prometheus and Amazon Managed Service for Grafana work with other Amazon Web Services services\\n\\n<video src=\\"https://dev-media.amazoncloud.cn/3a231de5f68941eeb85410e7ab2c0acf_Implementing%20Observability%20with%20Amazon%20Managed%20Open%20Source%20Services%20-%20AWS%20Online%20Tech%20Talk.mp4\\" class=\\"manvaVedio\\" controls=\\"controls\\" style=\\"width:160px;height:160px\\"></video>\\n\\n#### **Events for your diary**\\n#### **DevAx**\\n#### **June 3rd, 2pm CET**\\n\\nI am excited to let you know about DevAx, a new workshop series by developers for developers that will be streamed live via Twitch. The first Thursday of every month you will bel able to join our Amazon Web Services experts and invited guests as they take you through a variety of interesting open source topics, projects and more.\\n\\nMake sure you check out Twitch and join in on the fun! ++[https://www.twitch.tv/aws](https://www.twitch.tv/aws)++\\n\\n#### **Data Hack for Good 2021**\\n#### **June 5th, Belfast/Virtual**\\n\\nThe Allstate Data Hack for Good, is taking place virtually Saturday and Sunday 5th and 6th June 2021. It is being held in partnership with Belfast City Council and supported by the Department of Finance Open Data Innovation and Outreach Fund. This year this event is collaborating with Amazon Web Services.\\n\\nThe hackathon is open to technologists of all experience levels and backgrounds, with the goal to use data and technology to make life better in Belfast. During the hackathon, on demand technical training will be available, including data visualisation, data manipulation, model building, and Alexa Skills. Experts will be on hand throughout the event to answer questions, troubleshoot and support a collaborative approach to using data to solve problems.\\n\\nFind out more and register via ++[this link](https://www.meetup.com/Everything-Data/events/277828428/)++.\\n\\n#### **Maintainer Week**\\n#### **week of June 7th**\\n\\nMake sure you check out this week long event starting on the 7th of June, for open source maintainers to gather, share, and be celebrated. You can find details of ++[all the weeks sessions on their GitHub page](https://aws-oss.beachgeek.co.uk/kq)++, with Upstream on June 7th, Global Maintainer Summit on the 8/9th, The Changelog: Maintainer Spotlight on the 10th and FundOSS on the 11th.\\n\\n#### **Open@Amazon**\\n#### **June 16th, 9:00am - 5:00PM EDT**\\n\\nOf course I am biased, but this is a must attend event. A celebration of open source on Amazon Web Services with a fabulous cast of speakers, a fantastic broad set of topics and the event will be fully live so you can get your chance to interact with the speakers and the broader open source and Amazon Web Services community.\\n\\nCheck out the blog post, ++[What’s up with open source at Amazon Web Services? Attend Open@Amazon live on Twitch June 16](https://aws-oss.beachgeek.co.uk/jo)++\\n\\nOr just register to get a handy calendar invite ++[HERE](https://aws-oss.beachgeek.co.uk/jn)++\\n\\n#### **Cloud Native Day**\\n#### **23rd September, Bern Switzerland**\\n\\nWhat is this, an in person event returning? A stellar line up including our own Michael Hausenblas, an event looking at CNCF projects and the future of IT. Find out more and to view prices/register, by clicking ++[here](https://cloudnativeday.ch/en/#speakers)++.\\n\\n### **Stay in touch with open source at Amazon Web Services**\\nI hope this summary has been useful. Remember to check out the ++[Open Source homepage](https://aws.amazon.com/opensource/?opensource-all.sort-by=item.additionalFields.startDate&opensource-all.sort-order=asc)++ to keep up to date with all our activity in open source by following us on ++[@Amazon Web ServicesOpen](https://twitter.com/AWSOpen)++.\\n\\n\\n\\n\\n\\n\\n","render":"<h3><a id=\\"1st_June_2021__Instalment_70_0\\"></a><strong>1st June, 2021 - Instalment #70</strong></h3>\\n<p>Due to it being a bank holiday in the UK, Newsletter #70 is a day later than usual.</p>\\n<p>Make sure you check out this weeks “Quick Updates” as there are some updates that I know a lot of people will be happy about, and I always share great nuggets that sometimes folk miss. More great projects this week, from iamzero which helps you simplify your identity and access management, PMapper that you will find helpful for identifying risks in your IAM configurations, an Amazon Web Services CDK construct to help you deploy a VSCode server, Orkestra, a super nice project that lets you create workflows using Lambda functions a la Apache Airflow and many more. You will find blog posts from Amazon Web Services and the community covering Apache Airflow, Apache Spark, Apache Zeppelin, TensorFlow, Packer, lots of containers goodies and more. Finally, do not forget the events section, which is a great seg-way into the biggest news last week…</p>\\n<h4><a id=\\"OpenAmazon_5\\"></a><strong>Open@Amazon</strong></h4>\\n<p>Last week we announced Open@Amazon, a celebration of open source on Amazon Web Services with a fabulous cast of speakers, a fantastic broad set of topics and the event will be fully live so you can get your chance to interact with the speakers and the broader open source and Amazon Web Services community. Check out the blog post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/jo\\" target=\\"_blank\\">What’s up with open source at Amazon Web Services? Attend Open@Amazon live on Twitch June 16</a></ins> or just register to get a handy calendar invite <ins><a href=\\"https://aws-oss.beachgeek.co.uk/jn\\" target=\\"_blank\\">HERE</a></ins></p>\\n<h4><a id=\\"DevAx_connect_10\\"></a><strong>DevAx connect</strong></h4>\\n<p>Also, later this week we have a new regular open source series by developers for developers that will be streamed live via Twitch, hosted by the awesome Heitor Lessa. The first Thursday of every month you will be able to join Heitor and invited guests as they take you through a variety of interesting open source topics, projects and more. This week we have Lars Jacobsson, and I am looking forward to it. Check out the events section for more details.</p>\\n<h4><a id=\\"Please_help_me_improve_this_newsletter_14\\"></a><strong>Please help me improve this newsletter</strong></h4>\\n<p>Thank you to those who have provided such great feedback already, I am currently reviewing and thinking how to incorporate the suggestions. I would still love to hear from more of you, and I have more Amazon Web Services credit vouchers as a thank you for taking the time to complete the survey. The first 20 will get an Amazon Web Services credit voucher for \$25, so thank you again. It is only 5 questions, and will probably only take you a minute or two to complete.</p>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/jp\\" target=\\"_blank\\">Take me to the survey!</a></ins></p>\n<h4><a id=\\"Celebrating_open_source_contributors_20\\"></a><strong>Celebrating open source contributors</strong></h4>\\n<p>The articles posted in this series are only possible thanks to contributors and project maintainers and so I would like to shout out and thank those folks who really do power open source and enable us all to build on top of what they have created.</p>\n<p>So thank you to the following open source heroes: Stephan Fitzpatrick, Daniel Burkhardt, Malte Luecken, Mrudhula Balasubramanyan, Imaya Kumar Jagannathan, Joerg Woehrle, Brad Kashani, Carl Youngblood, Danilo Poccia, Christian Weber, Alolita Sharma, Nizar Tyrewalla, Josiah Davis, Yin Song, Anne Hu, Joshua McKiddy, Olivier Cruchant, Arunprasath Shankar, Mark Roy, Eli Fisher, Luke Kysow, Devarshi Shah, Horace Nelson, Brian Schiller, Mike Apted, Serhii Vasylenko, Ibrahim Cesar, Niko Virtala, Troy Ameigh and Andrew Glenn.</p>\n<p>Make sure you find and follow these builders and keep up to date with their open source projects and contributions.</p>\n<h4><a id=\\"Latest_from_open_source_projects_28\\"></a><strong>Latest from open source projects</strong></h4>\\n<h4><a id=\\"orkestra_29\\"></a><strong>orkestra</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/kw\\" target=\\"_blank\\">orkestra</a></ins> this new open source project from Stephan Fitzpatrick looks very promising, allowing you to create workflow tasks via Lambda functions and then orchestrate these similar to how Apache Airflow works. The <ins><a href=\\"https://aws-oss.beachgeek.co.uk/kx\\" target=\\"_blank\\">documentation</a></ins> says “a seamless way of building observable (scheduled or event-driven) cloud-native workflows. It aims to bring a similar development experience to that of Airflow while leveraging the full power of Amazon Web Services.”</p>\n<p>This is a project that is going to get some closer inspection from myself for sure.</p>\n<p><img src=\\"1\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"iamzero_37\\"></a><strong>iamzero</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/kk\\" target=\\"_blank\\">iamzero</a></ins> is an open source tool that helps simplify your identity and access management on Amazon Web Services. IAM Zero detects identity and access management issues and automatically suggests least-privilege policies. Check out the <ins><a href=\\"https://aws-oss.beachgeek.co.uk/km\\" target=\\"_blank\\">newly formed website</a></ins> and business that has started to help customers who want support from this tool, and you can read more about the backstory in this <ins><a href=\\"https://aws-oss.beachgeek.co.uk/kl\\" target=\\"_blank\\">post from Reddit</a></ins>. As per the Reddit post:</p>\n<p>The tool is in a similar space to iamlive, policy_sentry, and consoleme (all of which are worth checking out too if you’re interested in making Amazon Web Services security easier) but the main points of difference I see are:</p>\n<ul>\\n<li>\\n<p>iam-zero can run transparently on any or all of your roles just by swapping your Amazon Web Services SDK import to the iam-zero instrumented version or using the instrumented CLI</p>\n</li>\\n<li>\\n<p>iam-zero can run continuously as a service (deployed into a isolated Amazon Web Services account in an organization behind an SSO proxy) and could send notifications through Slack, email etc</p>\n</li>\\n<li>\\n<p>iam-zero uses TLS to dispatch events and doesn’t include any session tokens in the dispatched event (Amazon Web Services Client Side Monitoring, which iamlive utilises, includes authentication header details in the event - however iamlive is awesome for local policy development)</p>\n</li>\\n</ul>\n<h4><a id=\\"PMapper_49\\"></a><strong>PMapper</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/kn\\" target=\\"_blank\\">PMapper</a></ins> Principal Mapper (PMapper) is a script and library for identifying risks in the configuration of Amazon Web Services Identity and Access Management (IAM) for an Amazon Web Services account or an Amazon Web Services organization. It models the different IAM Users and Roles in an account as a directed graph, which enables checks for privilege escalation and for alternate paths an attacker could take to gain access to a resource or action in Amazon Web Services. PMapper includes a querying mechanism that uses a local simulation of Amazon Web Services’s authorization behavior. When running a query to determine if a principal has access to a certain action/resource, PMapper also checks if the user or role could access other users or roles that have access to that action/resource. This catches scenarios such as when a user doesn’t have permission to read an S3 object, but could launch an EC2 instance that can read the S3 object.</p>\n<h4><a id=\\"auroraserverlesstos3_53\\"></a><strong>aurora-serverless-to-s3</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/ki\\" target=\\"_blank\\">aurora-serverless-to-s3</a></ins> this open source project from Devetry simplifies the automation necessary to export Amazon RDS Aurora Serverless snapshots to S3 for a specific database whenever an automated snapshot is created. The project outlines how it achieves its task, provides clear examples and an Amazon Web Services CDK app to help easily deploy this project. Brian Schiller has put together a short blog post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/kj\\" target=\\"_blank\\">Aurora Serverless DB Export</a></ins> that provides some additional context. If you want to see a demo, check this short video</p>\n<p><video src=\\"https://dev-media.amazoncloud.cn/e528885a85a647a0884f0fd47b5c0e92_Amazon%20RDS%20Snapshot%20Export%20to%20S3.mp4\\" controls=\\"controls\\"></video></p>\\n<h4><a id=\\"ImageFlex_59\\"></a><strong>Image-Flex</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/kg\\" target=\\"_blank\\">Image-Flex</a></ins> is an open sourced solution from Horace Nelson that provides a robust image resizing service built on Amazon Web Services Serverless technologies and used to resize, optimise, and cache images on “the edge,” on the fly. If you want to know more about how you might use or deploy it, he has helpfully put together a detailed blog post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/kh\\" target=\\"_blank\\">An Open Source and Completely Serverless Image Resizing Service in Amazon Web Services</a></ins></p>\n<p><img src=\\"2\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"ecsanywheretutorial_66\\"></a><strong>ecs-anywhere-tutorial</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/ke\\" target=\\"_blank\\">ecs-anywhere-tutorial</a></ins> last week one of the big announcements was the general availability of <ins><a href=\\"https://aws-oss.beachgeek.co.uk/kf\\" target=\\"_blank\\">Amazon ECS Anywhere</a></ins>, and this project/tutorial is intended to walk you through an opinionated demonstration of how ECS Anywhere works. The initial steps will show you how to deploy a (somewhat) sophisticated multi services application in an Amazon Web Services region as an ECS service running on Amazon Web Services Fargate. Further in the tutorial, the steps will guide you through how to deploy parts of this application on ECS Anywhere managed instances in a customer managed infrastructure outside of the Amazon Web Services region. If you saw Massimo’s presentation and demo, you will know how nice this was and I am looking forward to dusting off my Raspberry Pi’s and trying this out for myself.</p>\n<p><img src=\\"3\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"Amazon_Web_Servicesdodocker_72\\"></a><strong>Amazon Web Services-do-docker</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/kd\\" target=\\"_blank\\">Amazon Web Services-do-docker</a></ins> this project provides a simple pattern for building and deployment of Docker containers to Amazon Web Services. It can be used “as is” to build and run a sample container, or can be customized and used to build, test, and deploy containerized microservices, jobs, or serverless functions. Containers built using this project can be run locally or on a number of Amazon Web Services compute environments by a simple change to the configuration files.</p>\n<p><img src=\\"4\\" alt=\\"Amazon Web Servicesdodockerdemo.gif\\" /></p>\n<h4><a id=\\"serverlessrules_78\\"></a><strong>serverless-rules</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/kv\\" target=\\"_blank\\">serverless-rules</a></ins> is new project that provides a compilation of rules to validate infrastructure as code template against recommended practices. This currently provides a module for cfn-lint and a plugin for tflint. This is currently in public preview, so if you do try it out please feedback anything you find.</p>\n<h4><a id=\\"cdkvscodefargate_82\\"></a><strong>cdk-vscode-fargate</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/ko\\" target=\\"_blank\\">cdk-vscode-fargate</a></ins> this was one of my favourite projects that I came across last week, although I have not had a chance to test it out yet. cdk-vscode-fargate is a JSII construct library for Amazon Web Services CDK from Mike Apted that allows you to deploy Code-server running VS Code remotely, on a Amazon Web Services Fargate container. The project provides you with a quick how-to guide to get you started, so hopefully something I can try over the weekend. You can find the source code in the GitHub repository <ins><a href=\\"https://aws-oss.beachgeek.co.uk/kp\\" target=\\"_blank\\">here</a></ins></p>\n<p><img src=\\"5\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"mwaalocalrunner_88\\"></a><strong>mwaa-local-runner</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/kc\\" target=\\"_blank\\">mwaa-local-runner</a></ins> I shared this project a few weeks ago (#66) but that was for Apache Airflow 1.x, and the project has been quickly updated to align with the launch last week of Apache Airflow version 2.0.2 support in Amazon Managed Workflows for Apache Airflow (MWAA). Everything is the same, just now using the latest version.</p>\n<h4><a id=\\"S3AssetDeploy_92\\"></a><strong>S3AssetDeploy</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/kb\\" target=\\"_blank\\">S3AssetDeploy</a></ins> if you are a Ruby developer then you are going to find this open source tool from Loomly very nice indeed. The folks at Loomly use this to safely package up and deploy their web assets to Amazon S3 that are being served via Cloudfront. This gem is designed to upload and clean unneeded assets from S3 in a safe manner such that older versions or recently removed assets are kept on S3 during the rolling deploy process. It also maintains a version limit and TTL (time-to-live) on assets to avoid deleting recent and outdated versions (up to a limit) or those that have been recently removed. Nice detailed docs and clear examples too.</p>\n<h4><a id=\\"amazonlinuxdind_96\\"></a><strong>amazonlinux-dind</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/kt\\" target=\\"_blank\\">amazonlinux-dind</a></ins> this repo from Niko Virtala is an Amazon Linux 2 container image with systemd and Docker Engine, and as Niko said in his tweet:</p>\n<p>I built a #Docker in Docker image that meets the #ECS Agent systemd requirement, so you don’t have to – you can find it from all well-stocked container registries and #GitHub</p>\n<h3><a id=\\"Tweet_of_the_week_102\\"></a><strong>Tweet of the week</strong></h3>\\n<p>We have not had a Tweet of the week for a while, but last week there were a couple that caught my eye and worth sharing.</p>\n<h4><a id=\\"Blender_105\\"></a><strong>Blender</strong></h4>\\n<p>This <ins><a href=\\"https://aws-oss.beachgeek.co.uk/k3\\" target=\\"_blank\\">tweet</a></ins> shared by John Mark, linked to an example of how the Amazon Web Services open source teams are working with open source projects and foundations to strengthen upstream communities.</p>\n<h4><a id=\\"Amazon_Web_Services_Copilot_vs_109\\"></a><strong>Amazon Web Services Copilot vs</strong></h4>\\n<p>This <ins><a href=\\"https://aws-oss.beachgeek.co.uk/k4\\" target=\\"_blank\\">tweet</a></ins> from Jonathan Matthews asking about using Amazon Web Services Copilot vs the Amazon Web Services cli started a great discussion and it was good to hear from the likes of Massimo, Romain and Efe. Massimo shared a <ins><a href=\\"https://aws-oss.beachgeek.co.uk/k5\\" target=\\"_blank\\">response</a></ins> but make sure you check out the thread.</p>\n<h4><a id=\\"Amazon_Web_Services_CDK_113\\"></a><strong>Amazon Web Services CDK</strong></h4>\\n<p>Henrique Lima had <ins><a href=\\"https://aws-oss.beachgeek.co.uk/k6\\" target=\\"_blank\\">an ask</a></ins> this week for folks who are using Amazon Web Services CDK. He is specifically looking for developers interested in checking or automating compliance of infrastructure changes, to provide 30 minutes to <ins><a href=\\"https://aws-oss.beachgeek.co.uk/k7\\" target=\\"_blank\\">complete a survey</a></ins> to help him as he works on a Change Analysis tool for his Master’s thesis to do just that.</p>\n<h3><a id=\\"Community_open_source_posts_117\\"></a><strong>Community open source posts</strong></h3>\\n<h4><a id=\\"GraakVM_118\\"></a><strong>GraakVM</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/ks\\" target=\\"_blank\\">Converting Java Lambda functions to GraalVM native-image</a></ins> Mark Sailes with a quick guide on how to create binary files that you can use in your Lambda functions, showing you how to configure your maven files to do this.</p>\n<h4><a id=\\"OpenSearch_122\\"></a><strong>OpenSearch</strong></h4>\\n<p>Eli Fisher shared the new public roadmap for the OpenSearch project, hosted in GitHub. The roadmap shows the next several months of planned features and releases. Going forward, this roadmap will show what the maintainers and contributors of the various components of the project intend to work on and when that work aims to launch. Read more in the post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/k8\\" target=\\"_blank\\">The OpenSearch roadmap is now available on GitHub</a></ins></p>\n<p><img src=\\"6\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"Consul_Service_Mesh_128\\"></a><strong>Consul Service Mesh</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/k9\\" target=\\"_blank\\">Announcing Tech Preview of Consul Service Mesh for Amazon ECS</a></ins> Luke Kysow and Devarshi Shah wrote last week about the availability (at this stage it is a tech preview) of HashiCorp Consul service mesh for Amazon ECS, with this tech preview letting you deploy non-production versions of Consul for testing on the Fargate launch type for ECS. The post covers how it works and provides a simple Web and API example to demonstrate this further.</p>\n<p><img src=\\"7\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"Anisible_134\\"></a><strong>Anisible</strong></h4>\\n<p>In the past week (and indeed this week) there have been a number of posts around how you can use open source tools as part of your CI/CD pipeline to build your applications for the Mac operating systems. In this post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/ka\\" target=\\"_blank\\">Run Ansible playbook on mac1.metal instances fleet with Amazon Web Services Systems Manager</a></ins> Amazon Web Services Community Builder Serhii Vasylenko shows you how you can use Ansible in conjunction with Amazon Web Services Services.</p>\n<h4><a id=\\"Reddis_138\\"></a><strong>Reddis</strong></h4>\\n<p>Another Amazon Web Services Commuinity Builder, Ibrahim Cesar has put this post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/kr\\" target=\\"_blank\\">Redis: Exploring Redis as Serverless Database to solve idempotence in APIs</a></ins> which is part of a series, but in this he explores what Reddis is and how to get started with it.</p>\n<h3><a id=\\"Amazon_Web_Services_and_Amazon_open_source_posts_142\\"></a><strong>Amazon Web Services and Amazon open source posts</strong></h3>\\n<h4><a id=\\"Assisted_Log_Enabler_143\\"></a><strong>Assisted Log Enabler</strong></h4>\\n<p>In my #67 newsletter I introduced you to this open source project from Amazon Web Services, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/h7\\" target=\\"_blank\\">assisted-log-enabler-for-Amazon Web Services</a></ins> an interesting tool that helps customers manage logging for various services. Joshua McKiddy has put together this blog post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/k0\\" target=\\"_blank\\">Introducing Assisted Log Enabler for Amazon Web Services</a></ins> showing you how to use Assisted Log Enabler for Amazon Web Services to ensure logging is turned on within an Amazon Web Services environment. Make sure you check this out.</p>\n<p><img src=\\"8\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"Amazon_Web_Services_Distro_for_OpenTelemetry_149\\"></a><strong>Amazon Web Services Distro for OpenTelemetry</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/jy\\" target=\\"_blank\\">Amazon Web Services Distro for OpenTelemetry 0.10.0 is now available with Amazon Web Services Lambda layers for .Net</a></ins> Alolita Sharma and Nizar Tyrewalla provide a quick update on the availability of version 0.10.0 of Amazon Web Services Distro for OpenTelemetry (ADOT). This is now available in Amazon Web Services Lambda layers for Amazon Web Services X-Ray support in .Net. The latest versions of Amazon Web Services Lambda layers with Amazon Web Services X-Ray support are now available for the OpenTelemetry Collector, Java, Java instrumentation, JavaScript, and Python. The post also covers news of some updated sample applications as well as news that StatsD is now supported within the ADOT Collector. Read the post for the full details.</p>\n<h4><a id=\\"Vim_tmux_and_Zsh_153\\"></a><strong>Vim, tmux and Zsh</strong></h4>\\n<p>As an extensive user of these tools, this post brought a huge smile to my face. In, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/jz\\" target=\\"_blank\\">Enhancing data science environments with Vim, tmux, and Zsh on Amazon EC2</a></ins> Josiah Davis, Yin Song, and Anne Hu share how you can use these tools to enable data scientists to be more productive in their day to day working environment. From providing Cloudformation templates that enable you to get this up and running yourself, to providing some great examples, this post is your must read post this week. Whether you are a data scientist or not, you can take some of the tips shared here to help you work smarter. Very nice indeed.</p>\n<p><img src=\\"9\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"Apache_Zeppelin_159\\"></a><strong>Apache Zeppelin</strong></h4>\\n<p>Launched last week, Amazon Kinesis Data Analytics Studio uses a number of open source projects to help make it easier for customers to analyse streaming data. You can now easily launch Apache Zeppelin notebooks, and interact and work with streaming data using SQL, Python or Scala. In, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/jw\\" target=\\"_blank\\">Introducing Amazon Kinesis Data Analytics Studio – Quickly Interact with Streaming Data Using SQL, Python, or Scala</a></ins> Danilo Poccia provides you an overview of the features as well as a follow along example to help get you orientated.</p>\n<p><img src=\\"10\\" alt=\\"image.png\\" /></p>\n<p>My favourite part of this blog post is where Danilo writes:</p>\n<p>“In Kinesis Data Analytics Studio, we run the open-source versions of Apache Zeppelin and Apache Flink, and we contribute changes upstream. For example, we have contributed bug fixes for Apache Zeppelin, and we have contributed to Amazon Web Services connectors for Apache Flink, such as those for Kinesis Data Streams and Kinesis Data Firehose. Also, we are working with the Apache Flink community to contribute availability improvements, including automatic classification of errors at runtime to understand whether errors are in user code or in application infrastructure.”</p>\n<h4><a id=\\"Apache_Spark_169\\"></a><strong>Apache Spark</strong></h4>\\n<p>Bigstream is an Amazon Web Services ISV Partner and pioneer in the field of hardware and software level acceleration. Brad Kashani, CEO at Bigstream, shares in this post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/ju\\" target=\\"_blank\\">Bigstream Provides Big Data Acceleration with Apache Spark and Amazon EMR</a></ins> why acceleration has better potential than CPU scaling, the historic challenges organisations have had in implementing accelerators, and how Bigstream are able to address those for a seamless, accelerated Spark experience.</p>\n<p><img src=\\"11\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"Packer_175\\"></a><strong>Packer</strong></h4>\\n<p>Packer is an open source tool from HashiCorp for building identical machine images for multiple platforms from a single source configuration. In this post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/jt\\" target=\\"_blank\\">Building Amazon Machine Images (AMIs) for EC2 Mac instances with Packer</a></ins> Joerg Woehrle shares best practices for building custom AMIs for EC2 Mac instances using HashiCorp Packer. Joerg covers how to create an AMI of an EC2 Mac instance, understake administrative tasks such as resizing the hard drive, resetting ec2-macos-init and installing some base software (golang) onto the image as well as demonstrating how to facilitate SSM to connect to an EC2 instance without the need to open any ports or a public IP. Read on to dive deeper…</p>\n<p><img src=\\"12\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"git_182\\"></a><strong>git</strong></h4>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/ku\\" target=\\"_blank\\">Use Git pre-commit hooks to avoid Amazon Web Services CloudFormation errors</a></ins> Troy Ameigh and Andrew Glenn with a post that walks you through how you can use GitHub pre-commit hooks with a number of open source tools to help address a number of common issues when using Amazon Web Services Cloudformation. The post takes a look at some of those open source tools and how you can use them to help you with things like checking shell scripts, validating end of line, dealing with white space, enforcing style and much more.</p>\n<p><img src=\\"13\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"Amazon_Web_Services_CDK_188\\"></a><strong>Amazon Web Services CDK</strong></h4>\\n<p>Christian Weber provides another one of his regular monthly updates covering all things Amazon Web Services CDK related, in his <ins><a href=\\"https://aws-oss.beachgeek.co.uk/jx\\" target=\\"_blank\\">CDK Corner – May 2021</a></ins>. Plenty of highlights, including updates from the recent CDK Day, news on the CDK v2 developer preview, Amazon Web Services CDK for Go developer preview and a nice list of updated/new constructs you should check out. This is essential reading in my opinion, and hats off to all those listed as having their first PRs merged. Please get in touch, I have an Amazon Web Services Credit voucher waiting for you…</p>\n<h4><a id=\\"TensorFlow_192\\"></a><strong>TensorFlow</strong></h4>\\n<p>A couple of posts this week on TensorFlow. The first post covers how to optimise the inference latency of TensorFlow Serving in the post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/k1\\" target=\\"_blank\\">How Contentsquare reduced TensorFlow inference latency with TensorFlow Serving on Amazon SageMaker</a></ins>. Olivier Cruchant shares how one customer was able to experiment with a number of approaches to find an optimum solution for hosting their inference endpoints.</p>\n<p><img src=\\"14\\" alt=\\"image.png\\" /></p>\n<p>Following that we have the post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/k2\\" target=\\"_blank\\">Host multiple TensorFlow computer vision models using Amazon SageMaker multi-model endpoints</a></ins>, where Arunprasath Shankar and Mark Roy demonstrate how to use SageMaker multi-model endpoints to host two computer TensorFlow vision models with different model architectures and datasets for image classification. They provide the sample notebook so you can try this for yourself.</p>\n<p><img src=\\"15\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"Open_Problems_202\\"></a><strong>Open Problems</strong></h4>\\n<p>Open Problems provides a framework for researchers to define formalized tasks in single-cell genomics and benchmark model performance on those tasks using Amazon Web Services. In the blog post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/jr\\" target=\\"_blank\\">Driving innovation in single-cell analysis on Amazon Web Services</a></ins> Daniel Burkhardt, Malte Luecken, and Mrudhula Balasubramanyan take a look in more detail how they are running this on Amazon Web Services, looking at the architecture of the platform, benchmarking computational methods for single-cell analysis and more.</p>\n<p><img src=\\"16\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"Ethereum_208\\"></a><strong>Ethereum</strong></h4>\\n<p>Amazon Managed Blockchain support for managed Ethereum nodes makes it possible to build your own decentralized applications (dapps) without having to worry about maintaining reliable Ethereum nodes for relaying blockchain transactions or querying the state of blockchain data. In this post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/jv\\" target=\\"_blank\\">Deploy an Ethereum node on Amazon Managed Blockchain</a></ins> Carl Youngblood walks you through setting up and connecting to an Ethereum node with Managed Blockchain</p>\n<h3><a id=\\"Quick_updates_212\\"></a><strong>Quick updates</strong></h3>\\n<h4><a id=\\"Apache_Airflow_213\\"></a><strong>Apache Airflow</strong></h4>\\n<p>You can now launch Apache Airflow 2.0 environments on Amazon Managed Workflows for Apache Airflow (MWAA). Apache Airflow 2.0 is the latest version of the popular open-source tool that helps customers author, schedule, and monitor workflows.</p>\n<h4><a id=\\"PostgreSQL_217\\"></a><strong>PostgreSQL</strong></h4>\\n<p>Amazon Web Services Database Migration Service (Amazon Web Services DMS) has expanded functionality by adding support for PostgreSQL version 13 in preview mode. Using Amazon Web Services DMS, you can now perform live migrations from any Amazon Web Services DMS supported sources to PostgreSQL 13 databases, and from PostgreSQL 13 databases to any Amazon Web Services DMS supported targets.</p>\n<h4><a id=\\"Apache_Kakfa_221\\"></a><strong>Apache Kakfa</strong></h4>\\n<p>Amazon Managed Streaming for Apache Kafka (Amazon MSK) now supports Apache Kafka version 2.7.1 for new and existing clusters. Apache Kafka 2.7.1 includes several bug fixes. For a complete list of fixes, see the Apache Kafka <ins><a href=\\"https://aws-oss.beachgeek.co.uk/jq\\" target=\\"_blank\\">release notes for 2.7.1</a></ins>.</p>\n<h4><a id=\\"Apache_Cassandra_225\\"></a><strong>Apache Cassandra</strong></h4>\\n<p>A couple of quick updates this week.</p>\n<p>First up is news that Amazon Keyspaces (for Apache Cassandra), a fully managed Apache Cassandra–compatible database service, is now Health Insurance Portability and Accountability Act of 1996 (HIPAA) eligible to help you run healthcare workloads more easily. Amazon Web Services enables covered entities and their business associates subject to HIPAA to use the secure Amazon Web Services environment to process, maintain, and store protected health information. Now, you can run Cassandra workloads that are subject to HIPAA compliance more easily by using a fully managed and serverless database service. Amazon Keyspaces helps you secure PHI through features such as encryption, audit logging via Amazon Web Services CloudTrail, and access management through Amazon Web Services Identity and Access Management.</p>\n<p>Following that is news you can manage service quotas for Amazon Keyspaces (for Apache Cassandra), through the Service Quotas console. Using the Service Quotas console, you can view and manage quotas (formerly referred to as limits) easily and at scale as your Amazon Web Services workloads grow. For example, you can manage Amazon Keyspaces quotas such as account-level throughput, keyspaces and tables per Amazon Web Services Region, and concurrent Data Definition Language (DDL) operations. Service Quotas makes requesting quota increases easier: search for a quota, enter your desired value, and submit a quota increase request. You can manage your quotas proactively by configuring Amazon CloudWatch alarms that monitor usage and alert you to approaching quotas.</p>\n<h4><a id=\\"Apache_Spark_311_233\\"></a><strong>Apache Spark 3.1.1</strong></h4>\\n<p>Amazon EMR 6.3 release version now supports Apache Spark 3.1.1 and provides runtime performance improvements with EMR Runtime for Spark. Amazon EMR 6.3 also supports Apache Hudi 0.7.0, Flink 1.12.1, PrestoDB 0.245.1, PrestoSQL 350, Hue 4.9, JupyterHub 1.2, Oozie 5.2.1 and TensorFlow 2.4.1.</p>\n<h4><a id=\\"JetBrains_237\\"></a><strong>JetBrains</strong></h4>\\n<p>The Amazon Web Services Toolkit for JetBrains now provides developers with convenient IDE functionality to create and manage deployments from their code or image repositories using Amazon Web Services AppRunner. Amazon Web Services App Runner that was recently announced is a new service that makes it easy for customers without any prior containers or infrastructure experience to build, deploy, and run containerised web applications and APIs in just a few clicks.</p>\n<p>With new App Runner features in the Amazon Web Services Toolkit for JetBrains, users of any JetBrains IDE can configure and deploy web services, track deployment status (i.e. view deployment notification logs) and monitor state of deployed services (i.e., services active, running, or paused), all without leaving their IDEs.</p>\n<h4><a id=\\"Lustre_243\\"></a><strong>Lustre</strong></h4>\\n<p>Amazon FSx for Lustre, a service that provides cost-effective, high-performance, scalable file storage for compute workloads, now supports data compression. Data compression enables you to reduce storage consumption of both your file system storage and your file system backups. The data compression feature is designed to deliver high levels of compression without adversely impacting file system performance. Once data compression is enabled, newly written files are automatically compressed by FSx for Lustre before they are written to disk and automatically uncompressed when they are read. In addition to compression, Amazon FSx for Lustre offers a range of storage options to help you reduce costs, including a variety of SSD and HDD storage options, along with an option to store data on unreplicated storage for short-term processing of data.</p>\n<h3><a id=\\"Videos_of_the_week_247\\"></a><strong>Videos of the week</strong></h3>\\n<h4><a id=\\"Observability_248\\"></a><strong>Observability</strong></h4>\\n<p>Implementing observability in applications a requirement for achieving Operational Excellence, and a well implemented observability plan allows customers to react to operational events, run workloads effectively and gain insights into their applications. In this video, Imaya Kumar Jagannathan will walk you through how to use Amazon managed open source tools for observability, how to use Amazon Managed Prometheus to monitor the performance of containerised workloads and how Amazon Managed Service for Prometheus and Amazon Managed Service for Grafana work with other Amazon Web Services services</p>\n<p><video src=\\"https://dev-media.amazoncloud.cn/3a231de5f68941eeb85410e7ab2c0acf_Implementing%20Observability%20with%20Amazon%20Managed%20Open%20Source%20Services%20-%20AWS%20Online%20Tech%20Talk.mp4\\" controls=\\"controls\\"></video></p>\\n<h4><a id=\\"Events_for_your_diary_254\\"></a><strong>Events for your diary</strong></h4>\\n<h4><a id=\\"DevAx_255\\"></a><strong>DevAx</strong></h4>\\n<h4><a id=\\"June_3rd_2pm_CET_256\\"></a><strong>June 3rd, 2pm CET</strong></h4>\\n<p>I am excited to let you know about DevAx, a new workshop series by developers for developers that will be streamed live via Twitch. The first Thursday of every month you will bel able to join our Amazon Web Services experts and invited guests as they take you through a variety of interesting open source topics, projects and more.</p>\n<p>Make sure you check out Twitch and join in on the fun! <ins><a href=\\"https://www.twitch.tv/aws\\" target=\\"_blank\\">https://www.twitch.tv/aws</a></ins></p>\n<h4><a id=\\"Data_Hack_for_Good_2021_262\\"></a><strong>Data Hack for Good 2021</strong></h4>\\n<h4><a id=\\"June_5th_BelfastVirtual_263\\"></a><strong>June 5th, Belfast/Virtual</strong></h4>\\n<p>The Allstate Data Hack for Good, is taking place virtually Saturday and Sunday 5th and 6th June 2021. It is being held in partnership with Belfast City Council and supported by the Department of Finance Open Data Innovation and Outreach Fund. This year this event is collaborating with Amazon Web Services.</p>\n<p>The hackathon is open to technologists of all experience levels and backgrounds, with the goal to use data and technology to make life better in Belfast. During the hackathon, on demand technical training will be available, including data visualisation, data manipulation, model building, and Alexa Skills. Experts will be on hand throughout the event to answer questions, troubleshoot and support a collaborative approach to using data to solve problems.</p>\n<p>Find out more and register via <ins><a href=\\"https://www.meetup.com/Everything-Data/events/277828428/\\" target=\\"_blank\\">this link</a></ins>.</p>\n<h4><a id=\\"Maintainer_Week_271\\"></a><strong>Maintainer Week</strong></h4>\\n<h4><a id=\\"week_of_June_7th_272\\"></a><strong>week of June 7th</strong></h4>\\n<p>Make sure you check out this week long event starting on the 7th of June, for open source maintainers to gather, share, and be celebrated. You can find details of <ins><a href=\\"https://aws-oss.beachgeek.co.uk/kq\\" target=\\"_blank\\">all the weeks sessions on their GitHub page</a></ins>, with Upstream on June 7th, Global Maintainer Summit on the 8/9th, The Changelog: Maintainer Spotlight on the 10th and FundOSS on the 11th.</p>\n<h4><a id=\\"OpenAmazon_276\\"></a><strong>Open@Amazon</strong></h4>\\n<h4><a id=\\"June_16th_900am__500PM_EDT_277\\"></a><strong>June 16th, 9:00am - 5:00PM EDT</strong></h4>\\n<p>Of course I am biased, but this is a must attend event. A celebration of open source on Amazon Web Services with a fabulous cast of speakers, a fantastic broad set of topics and the event will be fully live so you can get your chance to interact with the speakers and the broader open source and Amazon Web Services community.</p>\n<p>Check out the blog post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/jo\\" target=\\"_blank\\">What’s up with open source at Amazon Web Services? Attend Open@Amazon live on Twitch June 16</a></ins></p>\n<p>Or just register to get a handy calendar invite <ins><a href=\\"https://aws-oss.beachgeek.co.uk/jn\\" target=\\"_blank\\">HERE</a></ins></p>\n<h4><a id=\\"Cloud_Native_Day_285\\"></a><strong>Cloud Native Day</strong></h4>\\n<h4><a id=\\"23rd_September_Bern_Switzerland_286\\"></a><strong>23rd September, Bern Switzerland</strong></h4>\\n<p>What is this, an in person event returning? A stellar line up including our own Michael Hausenblas, an event looking at CNCF projects and the future of IT. Find out more and to view prices/register, by clicking <ins><a href=\\"https://cloudnativeday.ch/en/#speakers\\" target=\\"_blank\\">here</a></ins>.</p>\n<h3><a id=\\"Stay_in_touch_with_open_source_at_Amazon_Web_Services_290\\"></a><strong>Stay in touch with open source at Amazon Web Services</strong></h3>\\n<p>I hope this summary has been useful. Remember to check out the <ins><a href=\\"https://aws.amazon.com/opensource/?opensource-all.sort-by=item.additionalFields.startDate&amp;opensource-all.sort-order=asc\\" target=\\"_blank\\">Open Source homepage</a></ins> to keep up to date with all our activity in open source by following us on <ins><a href=\\"https://twitter.com/AWSOpen\\" target=\\"_blank\\">@Amazon Web ServicesOpen</a></ins>.</p>\n"}