Open source news and updates #86

{"value":"#### **October 18th, 2021 - Instalment #86**\nNewsletter #86.\n\nThis week we have a very broad variety of topics. Starting off with some fresh open source projects such as cfn-diagram, Amazon Web Services-jwt-verify, damo, Amazon Web Services-dotnet-deploy, automated-account-configuration, BayerCLAW and more. For mains, we have new Amazon Web Services and community authored posts on MySQL, OpenSearch, .NET, miniwdl, OpenMRS, Apache Hudi, Apache Spark, ROS, cdk8s, Jax, Deep Graph Library and more. Finally, for pudding we have this weeks video that provides an insight into contributing to Apache Airflow and we have some new events added so check those out and add them to your diary.\n\n##### **Celebrating open source contributors**\n\nThe articles posted in this series are only possible thanks to contributors and project maintainers and so I would like to shout out and thank those folks who really do power open source and enable us all to build on top of what they have created.\n\nSo thank you to the following open source heroes: Ian Avilez, Tim Pavlick, Jarek Potiuk, Ash Berlin-Taylor, Norm Johanson, Trivikram Kamat, Madhavan Sriram, Diego Menin, Gabriele Cacciola, Kunal Gautam, Mike Lin, Lee Pang, Krishna Singh, Razvan Ionasec, Archis Joglekar, Sean Morgan, Eddie Zaneski, Rohin Bhargava, Gopala Krishna, Saikumar Karanam, François Bouteruche and Nik Krichko.\n\nMake sure you find and follow these builders and keep up to date with their open source projects and contributions.\n\n##### **Latest open source projects**\n###### **cfn-diagram**\n\n++[cfn-diagram](https://aws-oss.beachgeek.co.uk/4r)++ whilst I did cover this project back in Feb, I was ++[notified](https://aws-oss.beachgeek.co.uk/10b)++ by Lars from Mathem that there was now a new capability, ascii graphics.\n\n\n\n##### **Amazon Web Services-jwt-verify**\n\n++[Amazon Web Services-jwt-verify](https://aws-oss.beachgeek.co.uk/10c)++ this project provides a NodeJS library for verifying JWTs signed by [Amazon Cognito](https://aws.amazon.com/cn/cognito/?trk=cndc-detail), and any OIDC-compatible IDP that signs JWTs with RS256 (currently, only signature algorithm RS256 is supported)\n\n##### **Amazon Web Services-appconfig-codepipeline-cdk**\n\n++[amazon-appconfig-codepipeline-cdk](https://aws-oss.beachgeek.co.uk/10d)++ this project provides a sample application demos setup of Amazon Web Services AppConfig using Amazon CDK. Amazon Web Services AppConfig application is set up to use Amazon Web Services CodePipeline as configuration store. It also sets up Amazon Web Services Lambda validator to validate the configuration.\n\n\n\n##### **Amazon Web Services-cloud-map-mcs-controller-for-k8s**\n\n++[amazon-cloud-map-mcs-controller-for-k8s](https://aws-oss.beachgeek.co.uk/10e)++ this project that snuck out late last week, Amazon Web Services Cloud Map multi-cluster service discovery for Kubernetes (K8s) is a controller that implements existing multi-cluster services API that allows services to communicate across multiple clusters. The implementation relies on Amazon Web Services Cloud Map for enabling cross-cluster service discovery.\n\n##### **BayerCLAW**\n\n++[BayerCLAW](https://aws-oss.beachgeek.co.uk/10f)++ this project from the Bayer Group is a workflow orchestration system for Amazon Web Services, targeted at bioinformatics pipelines. A workflow consists of a sequence of computational steps, each of which is captured in a Docker container. Some steps may parallelise work across many executions of the same container (scatter/gather pattern).\n\n##### **damo**\n\n++[damo](https://aws-oss.beachgeek.co.uk/10g)++ this is a cool project, damo is a user space tool for ++[DAMON](https://aws-oss.beachgeek.co.uk/10h)++. Using this, you can monitor the data access patterns of your system or workloads and make data access-aware memory management optimisations.\n\n\n\n\n##### **Amazon Web Services-dotnet-deploy**\n\n++[amazon-dotnet-deploy](https://aws-oss.beachgeek.co.uk/b7)++ this repository contains the Amazon Web Services .NET deployment tool for .NET CLI - the opinionated tooling that simplifies deployment of .NET applications with minimum Amazon knowledge. The tool suggests the right Amazon Web Services compute service to deploy your application to. It then builds and packages your application as required by the chosen compute service, generates the deployment infrastructure, deploys your application by using the Cloud Development Kit (CDK), and displays the endpoint. Norm Johanson has put together a couple of blog posts. First we have ++[Update on our new Amazon Web Services .NET Deployment Experience](https://aws-oss.beachgeek.co.uk/102)++ which provides you with an update of the recent updates to this project. Following that we have ++[Deployment Projects with the new Amazon Web Services .NET Deployment Experience](https://aws-oss.beachgeek.co.uk/zz)++ that walks you through a new feature of this project, deployment projects[hands on]\n\n##### **automated-account-configuration**\n\n++[automated-account-configuration](https://aws-oss.beachgeek.co.uk/zy)++ this project is the Automated Account Configuration, a sample solution to enable operational scale for Amazon Web Services customers by automating repeatable steps required before Amazon Web Services accounts are used for customer workloads. Steps include setting up backups and patching for the resources within the account.\n\n\n\n#### **Amazon Web Services and Community blog posts**\n###### **MySQL**\n\nNik Krichko has put together this post, ++[Comparing Graviton (ARM) Performance to Intel and AMD for MySQL](https://aws-oss.beachgeek.co.uk/10i)++ that takes a closer look at running MySQL against different instance types on Amazon Web Services to see what the different performance characteristics are. If you are running MySQL workloads you should check this out - no spoilers, but very interesting indeed.\n\n###### **headless-recorder**\n\n[Amazon CloudWatch](https://aws.amazon.com/cn/cloudwatch/?trk=cndc-detail) Synthetics to create canaries, configurable scripts that run on a schedule, to monitor your endpoints and APIs. Canaries offer programmatic access to a headless Google Chrome Browser via Puppeteer or Selenium Webdriver. Amazon provides a CloudWatch Synthetics Recorder to help you create canaries more easily. The recorder is a Google Chrome extension and based on the open source project, ++[Headless recorder](https://aws-oss.beachgeek.co.uk/10a)++. In this post, ++[Visual monitoring of applications with Amazon CloudWatch Synthetics](https://aws-oss.beachgeek.co.uk/109)++ Mahanth Jayadeva and Yesh Ravindra walk through how the visual monitoring blueprint for [Amazon CloudWatch](https://aws.amazon.com/cn/cloudwatch/?trk=cndc-detail) Synthetics can be utilised to monitor your applications for visual defects. [hands on]\n\n\n\n\n###### **.NET**\n\nMy colleague François Bouteruche is back with another instalment of the story of My Local Farmer, a fictional company inspired by customer interactions with Amazon Web Services Solutions Architects. This episode, ++[Migrating our trusty ol’ .NET Framework applications to Amazon Web Services, I couldn’t believe it!](https://aws-oss.beachgeek.co.uk/108)++ takes a look at a number of open source projects you can use to help move your .NET workloads over to Amazon Web Services. Tres bien!\n\n###### **cdk8s**\n\nPerfect timing with Kubecon last week, was Eddie Zaneski in his post, ++[Announcing the general availability of cdk8s and support for Go](https://aws-oss.beachgeek.co.uk/106)++. For those who may not be familiar with cdk8s, this is a CNCF Sandbox project, an open-source software development framework for defining Kubernetes applications using general-purpose programming languages. The output of a CDK8s program is Kubernetes YAML that you can apply directly to any cluster.\n\n###### **OpenSearch**\n\nRohin Bhargava, Gopala Krishna, and Saikumar Karanam provide a brief overview of the cross-cluster replication feature within OpenSearch, and the thought process behind the design and implementation. Read on in ++[Introduction to Cross-Cluster Replication](https://aws-oss.beachgeek.co.uk/107)++\n\n###### **Jax**\n\nJAX is NumPy on the CPU, GPU, and TPU, with great automatic differentiation for high-performance machine learning research. In this post, ++[Train and deploy deep learning models using JAX with Amazon SageMaker](https://aws-oss.beachgeek.co.uk/105)++ Archis Joglekar and Sean Morgan show how you can utilise the Bring Your Own Container (BYOC) paradigm to train ML models on GPUs using the increasingly popular JAX library from Google. As a bonus, the post shows how to serialise trained models into the TensorFlow SavedModel format so that we can use the existing TensorFlow Serving infrastructure provided by SageMaker. [hands on]\n\n\n###### **Deep Graph Library**\n\nIan Avilez and Tim Pavlick from HawkEye 360 provide some insights in how they are using the open source Deep Graph Library and [Amazon Neptune](https://aws.amazon.com/cn/neptune/?trk=cndc-detail) to find risks for nearby ships in the post ++[HawkEye 360 predicts vessel risk using the Deep Graph Library and Amazon Neptune](https://aws-oss.beachgeek.co.uk/10j)++. The graph networks in Neptune and GNN models enable HawkEye 360 to reveal hidden relationships among vessels that would otherwise be lost in the vast sea of complexity. Read on to find out more.\n\n\n\n\n###### **OpenMRS**\n\nOpenMRS is a collaborative open-source project to develop software to support the delivery of health care. In the post ++[Deploying OpenMRS Electronic Health Record (EHR) system on Amazon Web Services](https://aws-oss.beachgeek.co.uk/104)++, Krishna Singh and Razvan Ionasec explore how to deploy a OpenMRS EHR system on Amazon Web Services so you can support hundreds of concurrent HCP users and tens of thousands of patients. [hands on]\n\n\n\n###### **miniwdl**\n\nThe Workflow Description Language (WDL) is a way to specify data processing workflows with a human-readable and -writeable syntax. WDL makes it straightforward to define analysis tasks, chain them together in workflows, and parallelise their execution. miniwdl is a local runner and developer toolkit for the bioinformatics-focused Workflow Description Language (WDL). In the blog post, ++[Using miniwdl, GWFCore, and SageMaker Studio as a cloud IDE for genomics workflows](https://aws-oss.beachgeek.co.uk/103)++ written by Mike Lin and Lee Pang, will show you how you can combine these open source tools together with Amazon Web Services to build workflows and manage ad hoc analysis runs.\n\n\n\n###### **Apache Hudi**\n\nApache Hudi is an open-source data management framework used to simplify incremental data processing and data pipeline development. In this case study from Amazon's Transportation service, they share how they enabled near-real-time event analytics at petabyte scale using Apache Hudi tables created by Amazon Web Services Glue Spark jobs. For data analytics afficiondos, read on in ++[How Amazon Transportation Service enabled near-real-time event analytics at petabyte scale using Amazon Web Services Glue with Apache Hudi](https://aws-oss.beachgeek.co.uk/101)++ [warning, may contain gratuitous use of graphs!]\n\n\n\n###### **Amazon Web Services SDKs**\n\nTrivikram Kamat shares how they were able to reduce the publish size of v3 SDK for Javascript modular packages by ~50% in the post, ++[How we halved the publish size of modular Amazon Web Services SDK for JavaScript clients](https://aws-oss.beachgeek.co.uk/100)++. A must read this week.\n\n\n\n##### **Open Data**\n\nWhilst not open source, open data sets help democratise access to data by making it available to the public for analysis on Amazon Web Services; develop new cloud-native techniques, formats, and tools that lower the cost of working with data; and encourage the development of communities that benefit from access to shared datasets. In the post, ++[Climate data, koala genomes, analysis ready radar data, and highly-queryable genomic data: The latest open data on Amazon Web Services](https://aws-oss.beachgeek.co.uk/10k)++ Joe Flasher shares 26 new data sets released.\n\n##### **Quick updates**\n###### **Apache Spark**\n\n[Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) 6.4 release version now supports Apache Spark 3.1.2 and provides runtime improvements with [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) Runtime for Apache Spark. [Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) 6.4 provides Presto runtime improvements for PrestoDB 0.254, and runtime improvements for Apache Hive 3.1.2 when you use Amazon Web Services Glue Data Catalog for your metastore.\n\n[Amazon EMR](https://aws.amazon.com/cn/emr/?trk=cndc-detail) 6.4 supports Apache Hudi 0.8.0, Trino 359, PrestoDB 0.254, Apache HBase 2.4.4, Apache Phoenix 5.1.2, Apache Flink 1.13.1, Apache Livy 0.7.1, JupyterHub 1.4.1, Apache Zookeeper 3.5.7 and Apache MXNet 1.8.0. Please see our release guide to learn more.\n\n###### **ActiveMQ**\n\nYou can now launch Apache ActiveMQ 5.16.3 brokers on [Amazon MQ](https://aws.amazon.com/cn/amazon-mq/?trk=cndc-detail). This version update to ActiveMQ contains several fixes and improvements compared to the previously supported version, ActiveMQ 5.16.2.\n\n###### **Amazon Web Services CDK**\n\nDuring September, 2021, 5 new versions of the Amazon Web Services Cloud Development Kit (CDK) for JavaScript, TypeScript, Java, Python, .NET and Go were released (v1.121.0 through v.125.0). With these releases, the CDK CLI now has support for hotswap deployments for faster inner-loop development iterations on the application code in your CDK project. Hotswap initially supports Amazon Web Services Lambda handler code, but support is planned for additional resource types and a “watch” mode which continually watches for changes and deploys any updates. Additionally, users can preserve successfully provisioned resources by disabling automatic stack rollbacks, further reducing deployment and iteration time. These releases also resolve 21 issues and introduce 40 new features that span over 30 different modules across the library. Many of these changes were contributed by the developer community.\n\nRead the full release notes for ++[1.121.0](https://aws-oss.beachgeek.co.uk/zt)++, ++[1.122.0](https://aws-oss.beachgeek.co.uk/zu)++, ++[1.123.0](https://aws-oss.beachgeek.co.uk/zv)++, ++[1.124.0](https://aws-oss.beachgeek.co.uk/zw)++, ++[1.125.0](https://aws-oss.beachgeek.co.uk/zx)++\n\n###### **ROS**\n\nAmazon Web Services RoboMaker, a service that allows customers to simulate robotics applications at cloud scale, now supports GPU based simulation jobs for compute-intensive simulation workloads such as high fidelity simulation, vision processing, and machine learning (ML). Previously, Amazon Web Services RoboMaker simulation jobs ran only on central processing unit (CPU) instances; now you can choose between a CPU based or GPU based simulation job. Developers can run, scale, and automate GPU based simulations. GPU based simulations support higher frames-per-second, higher resolutions, lower sensor latencies, and faster simulation job completion times than CPU based simulation jobs. These capabilities enable improved sensing by cameras and realistic rendering needed for use cases such as ML model training, reinforcement learning, and testing use cases that require high fidelity simulations. When running a GPU based simulation job, the Amazon Web Services RoboMaker GUI tool viewer now supports higher resolutions, enabling you to see simulated objects in greater detail.\n\nAlso new, Amazon Web Services RoboMaker now supports expanded configuration for any robot and simulation software. Previously Robot Operating System (ROS) and Gazebo are the only supported robot and simulation software configuration in RoboMaker. This new feature enables customers to use and configure any robot and simulation software of their choice while running simulations in RoboMaker. To use this feature, you select General software suite for Robot application and Simulation runtime for Simulation Application. By choosing the Simulation Runtime configuration, RoboMaker bypasses validations for any specific robot or simulation software, and provides generic simulation features such as sourcing files to the simulation environment, logging, launching simulation tools, and streaming tool GUIs.\n\n##### **Video of the week**\n\nWant to contribute to Apache Airflow? Find out more as to how this works by checking out this video, where Jarek Potiuk and Ash Berlin-Taylor, Apache Airflow maintainers walk you through why developer and contributor experience matters for Apache Community projects such as Apache Airflow and what they have done together with other members of the community to improve it.\n\n<video src=\\"https://dev-media.amazoncloud.cn/c9125262aeb14beca9422ed61ec855db_Contributor%20Experience%20matters%20-%20Jarek%20Potiuk%2C%20Ash%20Berlin-Taylor.mp4\\" class=\\"manvaVedio\\" controls=\\"controls\\" style=\\"width:160px;height:160px\\"></video>\n\n##### **Events for your diary**\n##### **Enterprise Scale NLP with Hugging Face & SageMaker**\n##### **October 26th 2021 - 5:00 PM (BST)**\n\nIn this workshop, ++[Getting Started with Amazon SageMaker: Training your first NLP Transformer model with Hugging Face](https://aws-oss.beachgeek.co.uk/10l)++ you will learn how to use [Amazon SageMaker](https://aws.amazon.com/cn/sagemaker/?trk=cndc-detail) to train a Hugging Face Transformer model and deploy it afterwards. The workshop covers preparing and uploading test dataset to S3, fine-tuning a script to be used with [Amazon SageMaker](https://aws.amazon.com/cn/sagemaker/?trk=cndc-detail) Training jobs, launching a training job and storing the trained model into S3 and then deploying the model after successful training.\n\nEverything you need will be provided, so ++[read more and register using this link](https://aws-oss.beachgeek.co.uk/10l)++.\n\n\n##### **Flink Forwards Global 2021**\n##### **October 26th/27th**\n\nFlink Forward Global 2021 is a 2-day virtual conference for the Apache Flink and stream processing communities. Apache Flink is an open-source distributed engine for processing data streams that can support both streaming and batch workloads. Flink Forward has keynote presentations and talks on production Flink use cases, technical deep dive sessions, and the growth of the Flink ecosystem. You can meet core Flink committers, new and experienced users, and thought leaders who share experiences and best practices in stream processing, real-time analytics, and the management of mission-critical Flink deployments in production.\n\n++[Read more and sign up here](https://aws-oss.beachgeek.co.uk/wh)++.\n\n##### **Databricks | Amazon Web Services Lakehouse Dev Day Live Workshop**\n##### **November 16th 9:00 AM PT**\n\nDelta Lake is an open source storage layer that provides ACID transactions, scalable metadata handling, and unifies streaming and batch data processing. You can use Delta Lake on top of your existing data lake. During this workshop you will learn how to:\n\n- Make your existing [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail) data lakes into a lakehouse with Delta Lake.\n- Provide an easy-to-use platform for analysts to directly query data on your data lake using SQL Analytics\n- Simplify and automate data pipelines for streaming and batch data to lower costs and boost productivity for your data teams\n\n++[Read more and sign up here](https://aws-oss.beachgeek.co.uk/zs)++.\n\n##### **Stay in touch with open source at Amazon Web Services**\nI hope this summary has been useful. Remember to check out the ++[Open Source homepage](https://aws.amazon.com/opensource/?opensource-all.sort-by=item.additionalFields.startDate&opensource-all.sort-order=asc)++ to keep up to date with all our activity in open source by following us on ++[@Amazon Web ServicesOpen](https://twitter.com/AWSOpen)++\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","render":"<h4><a id=\\"October_18th_2021__Instalment_86_0\\"></a><strong>October 18th, 2021 - Instalment #86</strong></h4>\\n<p>Newsletter #86.</p>\n<p>This week we have a very broad variety of topics. Starting off with some fresh open source projects such as cfn-diagram, Amazon Web Services-jwt-verify, damo, Amazon Web Services-dotnet-deploy, automated-account-configuration, BayerCLAW and more. For mains, we have new Amazon Web Services and community authored posts on MySQL, OpenSearch, .NET, miniwdl, OpenMRS, Apache Hudi, Apache Spark, ROS, cdk8s, Jax, Deep Graph Library and more. Finally, for pudding we have this weeks video that provides an insight into contributing to Apache Airflow and we have some new events added so check those out and add them to your diary.</p>\n<h5><a id=\\"Celebrating_open_source_contributors_5\\"></a><strong>Celebrating open source contributors</strong></h5>\\n<p>The articles posted in this series are only possible thanks to contributors and project maintainers and so I would like to shout out and thank those folks who really do power open source and enable us all to build on top of what they have created.</p>\n<p>So thank you to the following open source heroes: Ian Avilez, Tim Pavlick, Jarek Potiuk, Ash Berlin-Taylor, Norm Johanson, Trivikram Kamat, Madhavan Sriram, Diego Menin, Gabriele Cacciola, Kunal Gautam, Mike Lin, Lee Pang, Krishna Singh, Razvan Ionasec, Archis Joglekar, Sean Morgan, Eddie Zaneski, Rohin Bhargava, Gopala Krishna, Saikumar Karanam, François Bouteruche and Nik Krichko.</p>\n<p>Make sure you find and follow these builders and keep up to date with their open source projects and contributions.</p>\n<h5><a id=\\"Latest_open_source_projects_13\\"></a><strong>Latest open source projects</strong></h5>\\n<h6><a id=\\"cfndiagram_14\\"></a><strong>cfn-diagram</strong></h6>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/4r\\" target=\\"_blank\\">cfn-diagram</a></ins> whilst I did cover this project back in Feb, I was <ins><a href=\\"https://aws-oss.beachgeek.co.uk/10b\\" target=\\"_blank\\">notified</a></ins> by Lars from Mathem that there was now a new capability, ascii graphics.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/7f69f1d184e847e98603f383d1e6f882_demo-ascii.gif\\" alt=\\"demoascii.gif\\" /></p>\n<h5><a id=\\"Amazon_Web_Servicesjwtverify_20\\"></a><strong>Amazon Web Services-jwt-verify</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/10c\\" target=\\"_blank\\">Amazon Web Services-jwt-verify</a></ins> this project provides a NodeJS library for verifying JWTs signed by Amazon Cognito, and any OIDC-compatible IDP that signs JWTs with RS256 (currently, only signature algorithm RS256 is supported)</p>\n<h5><a id=\\"Amazon_Web_Servicesappconfigcodepipelinecdk_24\\"></a><strong>Amazon Web Services-appconfig-codepipeline-cdk</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/10d\\" target=\\"_blank\\">amazon-appconfig-codepipeline-cdk</a></ins> this project provides a sample application demos setup of Amazon Web Services AppConfig using Amazon CDK. Amazon Web Services AppConfig application is set up to use Amazon Web Services CodePipeline as configuration store. It also sets up Amazon Web Services Lambda validator to validate the configuration.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/d936303809ef4c59955dafe6ed11bcef_image.png\\" alt=\\"image.png\\" /></p>\n<h5><a id=\\"Amazon_Web_Servicescloudmapmcscontrollerfork8s_30\\"></a><strong>Amazon Web Services-cloud-map-mcs-controller-for-k8s</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/10e\\" target=\\"_blank\\">amazon-cloud-map-mcs-controller-for-k8s</a></ins> this project that snuck out late last week, Amazon Web Services Cloud Map multi-cluster service discovery for Kubernetes (K8s) is a controller that implements existing multi-cluster services API that allows services to communicate across multiple clusters. The implementation relies on Amazon Web Services Cloud Map for enabling cross-cluster service discovery.</p>\n<h5><a id=\\"BayerCLAW_34\\"></a><strong>BayerCLAW</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/10f\\" target=\\"_blank\\">BayerCLAW</a></ins> this project from the Bayer Group is a workflow orchestration system for Amazon Web Services, targeted at bioinformatics pipelines. A workflow consists of a sequence of computational steps, each of which is captured in a Docker container. Some steps may parallelise work across many executions of the same container (scatter/gather pattern).</p>\n<h5><a id=\\"damo_38\\"></a><strong>damo</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/10g\\" target=\\"_blank\\">damo</a></ins> this is a cool project, damo is a user space tool for <ins><a href=\\"https://aws-oss.beachgeek.co.uk/10h\\" target=\\"_blank\\">DAMON</a></ins>. Using this, you can monitor the data access patterns of your system or workloads and make data access-aware memory management optimisations.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/0e37bde5b10943ccaeea8a0b714e9480_damo_monitor_water_nsquared.gif\\" alt=\\"damo_monitor_water_nsquared.gif\\" /></p>\n<h5><a id=\\"Amazon_Web_Servicesdotnetdeploy_45\\"></a><strong>Amazon Web Services-dotnet-deploy</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/b7\\" target=\\"_blank\\">amazon-dotnet-deploy</a></ins> this repository contains the Amazon Web Services .NET deployment tool for .NET CLI - the opinionated tooling that simplifies deployment of .NET applications with minimum Amazon knowledge. The tool suggests the right Amazon Web Services compute service to deploy your application to. It then builds and packages your application as required by the chosen compute service, generates the deployment infrastructure, deploys your application by using the Cloud Development Kit (CDK), and displays the endpoint. Norm Johanson has put together a couple of blog posts. First we have <ins><a href=\\"https://aws-oss.beachgeek.co.uk/102\\" target=\\"_blank\\">Update on our new Amazon Web Services .NET Deployment Experience</a></ins> which provides you with an update of the recent updates to this project. Following that we have <ins><a href=\\"https://aws-oss.beachgeek.co.uk/zz\\" target=\\"_blank\\">Deployment Projects with the new Amazon Web Services .NET Deployment Experience</a></ins> that walks you through a new feature of this project, deployment projects[hands on]</p>\n<h5><a id=\\"automatedaccountconfiguration_49\\"></a><strong>automated-account-configuration</strong></h5>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/zy\\" target=\\"_blank\\">automated-account-configuration</a></ins> this project is the Automated Account Configuration, a sample solution to enable operational scale for Amazon Web Services customers by automating repeatable steps required before Amazon Web Services accounts are used for customer workloads. Steps include setting up backups and patching for the resources within the account.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/7324415a212c446d9a6ccae6c8738263_image.png\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"Amazon_Web_Services_and_Community_blog_posts_55\\"></a><strong>Amazon Web Services and Community blog posts</strong></h4>\\n<h6><a id=\\"MySQL_56\\"></a><strong>MySQL</strong></h6>\\n<p>Nik Krichko has put together this post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/10i\\" target=\\"_blank\\">Comparing Graviton (ARM) Performance to Intel and AMD for MySQL</a></ins> that takes a closer look at running MySQL against different instance types on Amazon Web Services to see what the different performance characteristics are. If you are running MySQL workloads you should check this out - no spoilers, but very interesting indeed.</p>\n<h6><a id=\\"headlessrecorder_60\\"></a><strong>headless-recorder</strong></h6>\\n<p>Amazon CloudWatch Synthetics to create canaries, configurable scripts that run on a schedule, to monitor your endpoints and APIs. Canaries offer programmatic access to a headless Google Chrome Browser via Puppeteer or Selenium Webdriver. Amazon provides a CloudWatch Synthetics Recorder to help you create canaries more easily. The recorder is a Google Chrome extension and based on the open source project, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/10a\\" target=\\"_blank\\">Headless recorder</a></ins>. In this post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/109\\" target=\\"_blank\\">Visual monitoring of applications with Amazon CloudWatch Synthetics</a></ins> Mahanth Jayadeva and Yesh Ravindra walk through how the visual monitoring blueprint for Amazon CloudWatch Synthetics can be utilised to monitor your applications for visual defects. [hands on]</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/e49ab61719664baead4f63809f43439b_image.png\\" alt=\\"image.png\\" /></p>\n<h6><a id=\\"NET_67\\"></a><strong>.NET</strong></h6>\\n<p>My colleague François Bouteruche is back with another instalment of the story of My Local Farmer, a fictional company inspired by customer interactions with Amazon Web Services Solutions Architects. This episode, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/108\\" target=\\"_blank\\">Migrating our trusty ol’ .NET Framework applications to Amazon Web Services, I couldn’t believe it!</a></ins> takes a look at a number of open source projects you can use to help move your .NET workloads over to Amazon Web Services. Tres bien!</p>\n<h6><a id=\\"cdk8s_71\\"></a><strong>cdk8s</strong></h6>\\n<p>Perfect timing with Kubecon last week, was Eddie Zaneski in his post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/106\\" target=\\"_blank\\">Announcing the general availability of cdk8s and support for Go</a></ins>. For those who may not be familiar with cdk8s, this is a CNCF Sandbox project, an open-source software development framework for defining Kubernetes applications using general-purpose programming languages. The output of a CDK8s program is Kubernetes YAML that you can apply directly to any cluster.</p>\n<h6><a id=\\"OpenSearch_75\\"></a><strong>OpenSearch</strong></h6>\\n<p>Rohin Bhargava, Gopala Krishna, and Saikumar Karanam provide a brief overview of the cross-cluster replication feature within OpenSearch, and the thought process behind the design and implementation. Read on in <ins><a href=\\"https://aws-oss.beachgeek.co.uk/107\\" target=\\"_blank\\">Introduction to Cross-Cluster Replication</a></ins></p>\n<h6><a id=\\"Jax_79\\"></a><strong>Jax</strong></h6>\\n<p>JAX is NumPy on the CPU, GPU, and TPU, with great automatic differentiation for high-performance machine learning research. In this post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/105\\" target=\\"_blank\\">Train and deploy deep learning models using JAX with Amazon SageMaker</a></ins> Archis Joglekar and Sean Morgan show how you can utilise the Bring Your Own Container (BYOC) paradigm to train ML models on GPUs using the increasingly popular JAX library from Google. As a bonus, the post shows how to serialise trained models into the TensorFlow SavedModel format so that we can use the existing TensorFlow Serving infrastructure provided by SageMaker. [hands on]</p>\n<h6><a id=\\"Deep_Graph_Library_84\\"></a><strong>Deep Graph Library</strong></h6>\\n<p>Ian Avilez and Tim Pavlick from HawkEye 360 provide some insights in how they are using the open source Deep Graph Library and Amazon Neptune to find risks for nearby ships in the post <ins><a href=\\"https://aws-oss.beachgeek.co.uk/10j\\" target=\\"_blank\\">HawkEye 360 predicts vessel risk using the Deep Graph Library and Amazon Neptune</a></ins>. The graph networks in Neptune and GNN models enable HawkEye 360 to reveal hidden relationships among vessels that would otherwise be lost in the vast sea of complexity. Read on to find out more.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/a51903b93aee4fdba1275fbb505f3fb9_image.png\\" alt=\\"image.png\\" /></p>\n<h6><a id=\\"OpenMRS_91\\"></a><strong>OpenMRS</strong></h6>\\n<p>OpenMRS is a collaborative open-source project to develop software to support the delivery of health care. In the post <ins><a href=\\"https://aws-oss.beachgeek.co.uk/104\\" target=\\"_blank\\">Deploying OpenMRS Electronic Health Record (EHR) system on Amazon Web Services</a></ins>, Krishna Singh and Razvan Ionasec explore how to deploy a OpenMRS EHR system on Amazon Web Services so you can support hundreds of concurrent HCP users and tens of thousands of patients. [hands on]</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/cc5fddd971f949d390e22b639c4e56f5_image.png\\" alt=\\"image.png\\" /></p>\n<h6><a id=\\"miniwdl_97\\"></a><strong>miniwdl</strong></h6>\\n<p>The Workflow Description Language (WDL) is a way to specify data processing workflows with a human-readable and -writeable syntax. WDL makes it straightforward to define analysis tasks, chain them together in workflows, and parallelise their execution. miniwdl is a local runner and developer toolkit for the bioinformatics-focused Workflow Description Language (WDL). In the blog post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/103\\" target=\\"_blank\\">Using miniwdl, GWFCore, and SageMaker Studio as a cloud IDE for genomics workflows</a></ins> written by Mike Lin and Lee Pang, will show you how you can combine these open source tools together with Amazon Web Services to build workflows and manage ad hoc analysis runs.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/039b236a55ea46df81f1861963ddd992_image.png\\" alt=\\"image.png\\" /></p>\n<h6><a id=\\"Apache_Hudi_103\\"></a><strong>Apache Hudi</strong></h6>\\n<p>Apache Hudi is an open-source data management framework used to simplify incremental data processing and data pipeline development. In this case study from Amazon’s Transportation service, they share how they enabled near-real-time event analytics at petabyte scale using Apache Hudi tables created by Amazon Web Services Glue Spark jobs. For data analytics afficiondos, read on in <ins><a href=\\"https://aws-oss.beachgeek.co.uk/101\\" target=\\"_blank\\">How Amazon Transportation Service enabled near-real-time event analytics at petabyte scale using Amazon Web Services Glue with Apache Hudi</a></ins> [warning, may contain gratuitous use of graphs!]</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/6c61f4fc50d34ebaad2e4761bf3a12a2_image.png\\" alt=\\"image.png\\" /></p>\n<h6><a id=\\"Amazon_Web_Services_SDKs_109\\"></a><strong>Amazon Web Services SDKs</strong></h6>\\n<p>Trivikram Kamat shares how they were able to reduce the publish size of v3 SDK for Javascript modular packages by ~50% in the post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/100\\" target=\\"_blank\\">How we halved the publish size of modular Amazon Web Services SDK for JavaScript clients</a></ins>. A must read this week.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/314ab6a0db7b43bab8902c23c21e9716_image.png\\" alt=\\"image.png\\" /></p>\n<h5><a id=\\"Open_Data_115\\"></a><strong>Open Data</strong></h5>\\n<p>Whilst not open source, open data sets help democratise access to data by making it available to the public for analysis on Amazon Web Services; develop new cloud-native techniques, formats, and tools that lower the cost of working with data; and encourage the development of communities that benefit from access to shared datasets. In the post, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/10k\\" target=\\"_blank\\">Climate data, koala genomes, analysis ready radar data, and highly-queryable genomic data: The latest open data on Amazon Web Services</a></ins> Joe Flasher shares 26 new data sets released.</p>\n<h5><a id=\\"Quick_updates_119\\"></a><strong>Quick updates</strong></h5>\\n<h6><a id=\\"Apache_Spark_120\\"></a><strong>Apache Spark</strong></h6>\\n<p>Amazon EMR 6.4 release version now supports Apache Spark 3.1.2 and provides runtime improvements with Amazon EMR Runtime for Apache Spark. Amazon EMR 6.4 provides Presto runtime improvements for PrestoDB 0.254, and runtime improvements for Apache Hive 3.1.2 when you use Amazon Web Services Glue Data Catalog for your metastore.</p>\n<p>Amazon EMR 6.4 supports Apache Hudi 0.8.0, Trino 359, PrestoDB 0.254, Apache HBase 2.4.4, Apache Phoenix 5.1.2, Apache Flink 1.13.1, Apache Livy 0.7.1, JupyterHub 1.4.1, Apache Zookeeper 3.5.7 and Apache MXNet 1.8.0. Please see our release guide to learn more.</p>\n<h6><a id=\\"ActiveMQ_126\\"></a><strong>ActiveMQ</strong></h6>\\n<p>You can now launch Apache ActiveMQ 5.16.3 brokers on Amazon MQ. This version update to ActiveMQ contains several fixes and improvements compared to the previously supported version, ActiveMQ 5.16.2.</p>\n<h6><a id=\\"Amazon_Web_Services_CDK_130\\"></a><strong>Amazon Web Services CDK</strong></h6>\\n<p>During September, 2021, 5 new versions of the Amazon Web Services Cloud Development Kit (CDK) for JavaScript, TypeScript, Java, Python, .NET and Go were released (v1.121.0 through v.125.0). With these releases, the CDK CLI now has support for hotswap deployments for faster inner-loop development iterations on the application code in your CDK project. Hotswap initially supports Amazon Web Services Lambda handler code, but support is planned for additional resource types and a “watch” mode which continually watches for changes and deploys any updates. Additionally, users can preserve successfully provisioned resources by disabling automatic stack rollbacks, further reducing deployment and iteration time. These releases also resolve 21 issues and introduce 40 new features that span over 30 different modules across the library. Many of these changes were contributed by the developer community.</p>\n<p>Read the full release notes for <ins><a href=\\"https://aws-oss.beachgeek.co.uk/zt\\" target=\\"_blank\\">1.121.0</a></ins>, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/zu\\" target=\\"_blank\\">1.122.0</a></ins>, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/zv\\" target=\\"_blank\\">1.123.0</a></ins>, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/zw\\" target=\\"_blank\\">1.124.0</a></ins>, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/zx\\" target=\\"_blank\\">1.125.0</a></ins></p>\n<h6><a id=\\"ROS_136\\"></a><strong>ROS</strong></h6>\\n<p>Amazon Web Services RoboMaker, a service that allows customers to simulate robotics applications at cloud scale, now supports GPU based simulation jobs for compute-intensive simulation workloads such as high fidelity simulation, vision processing, and machine learning (ML). Previously, Amazon Web Services RoboMaker simulation jobs ran only on central processing unit (CPU) instances; now you can choose between a CPU based or GPU based simulation job. Developers can run, scale, and automate GPU based simulations. GPU based simulations support higher frames-per-second, higher resolutions, lower sensor latencies, and faster simulation job completion times than CPU based simulation jobs. These capabilities enable improved sensing by cameras and realistic rendering needed for use cases such as ML model training, reinforcement learning, and testing use cases that require high fidelity simulations. When running a GPU based simulation job, the Amazon Web Services RoboMaker GUI tool viewer now supports higher resolutions, enabling you to see simulated objects in greater detail.</p>\n<p>Also new, Amazon Web Services RoboMaker now supports expanded configuration for any robot and simulation software. Previously Robot Operating System (ROS) and Gazebo are the only supported robot and simulation software configuration in RoboMaker. This new feature enables customers to use and configure any robot and simulation software of their choice while running simulations in RoboMaker. To use this feature, you select General software suite for Robot application and Simulation runtime for Simulation Application. By choosing the Simulation Runtime configuration, RoboMaker bypasses validations for any specific robot or simulation software, and provides generic simulation features such as sourcing files to the simulation environment, logging, launching simulation tools, and streaming tool GUIs.</p>\n<h5><a id=\\"Video_of_the_week_142\\"></a><strong>Video of the week</strong></h5>\\n<p>Want to contribute to Apache Airflow? Find out more as to how this works by checking out this video, where Jarek Potiuk and Ash Berlin-Taylor, Apache Airflow maintainers walk you through why developer and contributor experience matters for Apache Community projects such as Apache Airflow and what they have done together with other members of the community to improve it.</p>\n<p><video src=\\"https://dev-media.amazoncloud.cn/c9125262aeb14beca9422ed61ec855db_Contributor%20Experience%20matters%20-%20Jarek%20Potiuk%2C%20Ash%20Berlin-Taylor.mp4\\" controls=\\"controls\\"></video></p>\\n<h5><a id=\\"Events_for_your_diary_148\\"></a><strong>Events for your diary</strong></h5>\\n<h5><a id=\\"Enterprise_Scale_NLP_with_Hugging_Face__SageMaker_149\\"></a><strong>Enterprise Scale NLP with Hugging Face &amp; SageMaker</strong></h5>\\n<h5><a id=\\"October_26th_2021__500_PM_BST_150\\"></a><strong>October 26th 2021 - 5:00 PM (BST)</strong></h5>\\n<p>In this workshop, <ins><a href=\\"https://aws-oss.beachgeek.co.uk/10l\\" target=\\"_blank\\">Getting Started with Amazon SageMaker: Training your first NLP Transformer model with Hugging Face</a></ins> you will learn how to use Amazon SageMaker to train a Hugging Face Transformer model and deploy it afterwards. The workshop covers preparing and uploading test dataset to S3, fine-tuning a script to be used with Amazon SageMaker Training jobs, launching a training job and storing the trained model into S3 and then deploying the model after successful training.</p>\n<p>Everything you need will be provided, so <ins><a href=\\"https://aws-oss.beachgeek.co.uk/10l\\" target=\\"_blank\\">read more and register using this link</a></ins>.</p>\n<h5><a id=\\"Flink_Forwards_Global_2021_157\\"></a><strong>Flink Forwards Global 2021</strong></h5>\\n<h5><a id=\\"October_26th27th_158\\"></a><strong>October 26th/27th</strong></h5>\\n<p>Flink Forward Global 2021 is a 2-day virtual conference for the Apache Flink and stream processing communities. Apache Flink is an open-source distributed engine for processing data streams that can support both streaming and batch workloads. Flink Forward has keynote presentations and talks on production Flink use cases, technical deep dive sessions, and the growth of the Flink ecosystem. You can meet core Flink committers, new and experienced users, and thought leaders who share experiences and best practices in stream processing, real-time analytics, and the management of mission-critical Flink deployments in production.</p>\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/wh\\" target=\\"_blank\\">Read more and sign up here</a></ins>.</p>\n<h5><a id=\\"Databricks__Amazon_Web_Services_Lakehouse_Dev_Day_Live_Workshop_164\\"></a><strong>Databricks | Amazon Web Services Lakehouse Dev Day Live Workshop</strong></h5>\\n<h5><a id=\\"November_16th_900_AM_PT_165\\"></a><strong>November 16th 9:00 AM PT</strong></h5>\\n<p>Delta Lake is an open source storage layer that provides ACID transactions, scalable metadata handling, and unifies streaming and batch data processing. You can use Delta Lake on top of your existing data lake. During this workshop you will learn how to:</p>\n<ul>\\n<li>Make your existing Amazon S3 data lakes into a lakehouse with Delta Lake.</li>\n<li>Provide an easy-to-use platform for analysts to directly query data on your data lake using SQL Analytics</li>\n<li>Simplify and automate data pipelines for streaming and batch data to lower costs and boost productivity for your data teams</li>\n</ul>\\n<p><ins><a href=\\"https://aws-oss.beachgeek.co.uk/zs\\" target=\\"_blank\\">Read more and sign up here</a></ins>.</p>\n<h5><a id=\\"Stay_in_touch_with_open_source_at_Amazon_Web_Services_175\\"></a><strong>Stay in touch with open source at Amazon Web Services</strong></h5>\\n<p>I hope this summary has been useful. Remember to check out the <ins><a href=\\"https://aws.amazon.com/opensource/?opensource-all.sort-by=item.additionalFields.startDate&amp;opensource-all.sort-order=asc\\" target=\\"_blank\\">Open Source homepage</a></ins> to keep up to date with all our activity in open source by following us on <ins><a href=\\"https://twitter.com/AWSOpen\\" target=\\"_blank\\">@Amazon Web ServicesOpen</a></ins></p>\n"}