New features in Amazon DMS 3.4.7

{"value":"<p>We are excited to announce the availability of <a href=\\"https://aws.amazon.com/dms/\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">AWS Database Migration Service</a> (AWS DMS) replication engine version 3.4.7. This version provides improvements covering security, performance, and connectivity that were requested by many of our customers. In this post, we highlight a few key features. For the entire list of improvements, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReleaseNotes.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">AWS DMS release notes</a>.</p>\\n\\n<p>We dive deep into the following improvements in AWS DMS v3.4.7:</p>\n\n<ul> \\n <li>Use IBM Db2 for z/OS as source</li> \n <li>Use AWS Babelfish as a target for AWS DMS</li> \n <li>Configure VPC endpoints as source and target</li> \n <li>Use a Microsoft SQL Server secondary availability group replica as a source</li> \n </ul>\\n\\n### **Use IBM Db2 for z/OS as source**\\n\\n<p>Customers are increasingly interested in migrating from mainframe database systems to the cloud. Before AWS DMS v3.4.7, you had limited automation to migrate IBM Db2 for z/OS workloads into AWS. Now, we’re happy to share that AWS DMS will migrate your IBM Db2 for z/OS workloads to any of the target engines AWS DMS <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Endpoints.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">supports</a>, including cloud-native databases like <a href=\\"https://aws.amazon.com/rds/aurora/\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Amazon Aurora</a> or managed service, open-source engines like <a href=\\"https://aws.amazon.com/rds/postgresql/\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Amazon Relational Database Service (Amazon RDS) for PostgreSQL</a>.</p>\\n\\n<p>You can use AWS DMS to migrate a full load of your IBM Db2 for z/OS source to your target of choice. You can also use the <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Validating.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">data validation</a> feature of AWS DMS for supported <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">target endpoints</a>.</p>\\n\\n<p>Use the steps in this section to migrate from IBM Db2 for z/OS to <a href=\\"https://aws.amazon.com/rds/aurora/postgresql-features/\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Amazon Aurora PostgreSQL-Compatible Edition</a>. To demonstrate this migration, we use an existing AWS DMS v3.4.7 replication instance and an existing Aurora PostgreSQL endpoint. To learn more about creating replication instances and endpoints, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReplicationInstance.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Working with an AWS DMS replication instance</a> and <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.PostgreSQL.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Using a PostgreSQL database as a target for AWS Database Migration Service</a>.</p>\\n\\n<p>Before you start your migration, you need to complete the <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_GettingStarted.Prerequisites.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Prerequisites for AWS Database Migration Service</a>.</p>\\n\\n#### **Use AWS SCT for schema conversion**\\n\\n<p>You can use <a href=\\"https://aws.amazon.com/dms/schema-conversion-tool/\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">AWS Schema Conversion Tool</a> (AWS SCT) to migrate your <a href=\\"https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_Source.DB2zOS.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">IBM Db2 for z/OS schema</a> to any supported target database. AWS SCT is free to download for both Windows and Linux Operating Systems. If you don’t want to convert your database objects and only migrate the data, you can skip this step.</p>\\n\\n<h3>Create an IBM Db2 for z/OS source endpoint in AWS DMS</h3>\n\n<p>In the following example, we select <strong>Source endpoint</strong> as our endpoint type. We name this endpoint <code>db2zos</code> and choose <strong>IBM Db2 for z/OS</strong> for <strong>Source engine</strong>. Enter the connection details and choose <strong>Create endpoint</strong>.</p>\\n\\n\\n\\n<h3>Create an AWS DMS migration task</h3>\n\n<p>To migrate the data from the source to target database, you need to create an <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">AWS DMS task</a>.</p>\\n\\n<p>For <strong>Task identifier</strong>, enter a name of your choosing. Choose your replication instance, the newly created Db2 for z/OS source database endpoint, and the existing PostgreSQL target endpoint. For <strong>Migration type</strong>, choose <strong>Migrate existing data</strong> at present AWS DMS v3.4.7 only supports full load for Db2 z/OS.</p>\\n\\n\\n\\n<p>Please select the tables/data, define any transformations, and enable <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Validating.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">data validation</a> as you wish while you continue with this migration task.</p>\\n\\n<p>We’ll be enhancing support for this endpoint in future releases. To learn more about using Db2 endpoint, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.DB2zOS.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Using IBM Db2 for z/OS databases as a source for AWS DMS</a>.</p>\\n\\n### **Use Babelfish as a target for AWS DMS**\\n\\n<p><a href=\\"https://aws.amazon.com/rds/aurora/babelfish/\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Babelfish for Aurora PostgreSQL</a> is a capability for Aurora PostgreSQL that enables Aurora to understand commands from applications written for Microsoft SQL Server.</p>\\n\\n<p>With Babelfish, Aurora PostgreSQL can understand T-SQL, SQL Server’s proprietary SQL dialect, and supports the same communications protocol, so your apps that were originally written for SQL Server can work with Aurora with fewer code changes. As a result, the effort required to modify and move applications running on SQL Server 2005 or newer to Aurora is reduced, leading to faster, lower-risk, and more cost-effective migrations.</p>\n\n<p>AWS DMS v3.4.7 introduces a new Babelfish target endpoint, which allows you to migrate your SQL Server workloads directly to PostgreSQL. As of this writing, AWS DMS supports full load migration to Babelfish support, and CDC is planned for future releases. If you want to create a Babelfish for Aurora PostgreSQL cluster, refer to <a href=\\"https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/babelfish-create.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Creating a Babelfish for Aurora PostgreSQL DB cluster</a>.</p>\\n\\n<p>You can use the following steps to complete a migration from SQL Server to Babelfish for Aurora PostgreSQL. Before you start your migration, you need to complete the <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_GettingStarted.Prerequisites.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Prerequisites for AWS Database Migration Service</a>. In this example, we use an existing AWS DMS v3.4.7 replication instance and an existing SQL Server source endpoint. For more information, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReplicationInstance.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Working with an AWS DMS replication instance</a> and <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.PostgreSQL.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Using a PostgreSQL database as a target for AWS Database Migration Service</a>.</p>\\n\\n<h3>Use AWS SCT to generate an assessment report</h3>\n\n<p>You can use AWS SCT to <a href=\\"https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_AssessmentReport.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">generate an assessment report</a>. Babelfish is supported as a <a href=\\"https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_Mapping.VirtualTargets.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">virtual target </a>in AWS SCT. In the following example, we use the SQL Server <code>AdventureWorksDW</code> database. To generate an assessment report for migration, we need to create a new project and for <strong>Add Source</strong>, choose <strong>Microsoft SQL Server</strong>. Select the <code>Sales</code> schema in the left pane as your source and <code>Babelfish (virtual)</code> in the right pane as your target. We need to create at least one <a href=\\"https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_Mapping.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">mapping rule</a> in order to <a href=\\"https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_AssessmentReport.Create.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">create a migration assessment report</a>.</p>\\n\\n\\n\\n<h3>Create a Babelfish endpoint</h3>\n\n<p>In the following example, we select <strong>Target endpoint</strong> as the endpoint type. We name this endpoint <code>babelfish</code> and choose <strong>Babelfish </strong>for <strong>Target</strong> <strong>engine</strong>. Enter the connection details and choose <strong>Create endpoint</strong>.</p>\\n\\n\\n\\n<h3>Create an AWS DMS migration task</h3>\n\n<p>AWS DMS uses an <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">AWS DMS task</a> for data migration. For <strong>Task identifier</strong>, enter a name. Select the existing 3.4.7 replication instance, existing SQL Server endpoint from the source database endpoint, and the newly created Babelfish target endpoint. For <strong>Migration type</strong>, enter a name of your choosing <strong>Migrate existing data</strong> as AWS DMS v3.4.7 does not support CDC for Babelfish at present.</p>\\n\\n\\n\\n<p>Please select the tables/data, define any transformations, and enable <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Validating.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">data validation</a> as you wish while you continue with this migration task.</p>\\n\\n<p>To learn more, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.Babelfish.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Using Babelfish as a target for AWS Database Migration Service</a>.</p>\\n\\n### **Configure VPC endpoints as source and target**\\n\\n<p>An <a href=\\"http://aws.amazon.com/vpc\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Amazon Virtual Private Cloud</a> ([Amazon VPC](https://aws.amazon.com/cn/vpc/?trk=cndc-detail)) <a href=\\"https://docs.aws.amazon.com/vpc/latest/privatelink/create-interface-endpoint.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">endpoint</a> is an interface powered by <a href=\\"https://aws.amazon.com/privatelink/\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">AWS PrivateLink</a> without requiring an internet gateway, NAT device, VPC connection, or <a href=\\"https://aws.amazon.com/directconnect/\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">AWS Direct Connect</a> connection. Starting with v3.4.7, AWS DMS supports VPC endpoints. Your AWS DMS replication instance VPC doesn’t require a public IP address to communicate with AWS services such as <a href=\\"http://aws.amazon.com/s3\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Amazon Simple Storage Service</a> ([Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail)), <a href=\\"https://aws.amazon.com/kinesis/\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Amazon Kinesis</a>, <a href=\\"https://aws.amazon.com/secrets-manager/\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">AWS Secrets Manager</a>, <a href=\\"https://aws.amazon.com/dynamodb/\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Amazon DynamoDB</a>, <a href=\\"http://aws.amazon.com/redshift\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Amazon Redshift</a>, and <a href=\\"https://aws.amazon.com/opensearch-service/\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Amazon OpenSearch Service</a>. AWS DMS database endpoints such as Oracle are reachable using native TCP routing and don’t require a VPC endpoint under 3.4.7 when the replication instance is configured with no accessible public IP address.</p>\\n\\n<p>The following diagram illustrates this architecture.</p>\n\n\n\n<p>Prior to 3.4.7, AWS DMS didn’t allow you to properly use VPC endpoints associated with other AWS services. That limitation resulted in AWS DMS replication traffic to be routed through the Amazon network backbone, which may not have been in line with your networking plan. With VPC endpoints, traffic between your replication instance with no accessible public IP and your other AWS services doesn’t leave the Amazon network.</p>\n\n<p>Starting with 3.4.7, you need to configure your VPC endpoint if your AWS DMS replication has no publicly accessible IP address and interacts with the services listed earlier. The AWS DMS task will fail when communication with AWS services from the 3.4.7 replication instance has no public accessibility from the internet or VPC endpoint.</p>\n\n<p>Starting with 3.4.7, AWS DMS can connect to any AWS source or target database with VPC endpoints as long as explicitly defined routes to these endpoints are defined in the AWS DMS endpoint and VPC endpoint.</p>\n\n<p>With VPC endpoint support, all traffic involved in moving your data remains inside your VPC network. It reduces replication interruptions and improves the quality of the data transfer. To learn more about VPC endpoints, refer to <a href=\\"https://docs.aws.amazon.com/vpc/latest/userguide/endpoint-services-overview.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Connect your VPC to services using AWS PrivateLink</a>.</p>\\n\\n<p>In the following example, we show you steps to create an AWS DMS replication instance without a publicly accessible IP, create a VPC endpoint, and set up an AWS DMS endpoint.</p>\n\n<h3>Create an AWS DMS replication instance without a publicly accessible IP</h3>\n\n<p>On the <strong>Create replication instance</strong> page of the AWS DMS console, specify your replication instance information and deselect the default <strong>Publicly accessible</strong>. To learn more about creating replication instances, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReplicationInstance.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Working with an AWS DMS replication instance</a>.</p>\\n\\n\\n\\n<h3>Create a VPC endpoint</h3>\n\n<p>In the following example, we show you how to create a VPC endpoint.</p>\n\n<ol> \\n <li>On the Amazon VPC console, choose the same AWS Region as your AWS DMS replication instance.</li> \n <li>In the navigation pane, choose <strong>Endpoints</strong>.<br> \\n\\n\\n</li> \n <li>Choose <strong>Create endpoint</strong>.</li> \\n <li>You can optionally specify a name tag. For example, <code>my-endpoint-DynamoDB-01</code>.</li> \\n <li>For <strong>Service category</strong>, select <strong>AWS services</strong>.<br> \\n\\n</li> \n <li>Under <strong>Services</strong>, for [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail) or DynamoDB only, choose a service name with its type set to <strong>Gateway</strong>.<br>\\n\\n</li> \n <li>For <strong>VPC</strong>, choose the same VPC as our AWS DMS replication instance to create the endpoint.</li> \\n <li>For <strong>Route Tables</strong>, choose all available <strong>Route Table ID</strong> values.</li> \\n <li>To specify access control, for <strong>Policy</strong>, choose <strong>Full access</strong>. If you want to use a policy creation tool to specify your own access control, choose <strong>Custom</strong>. In any case, use a trust policy that conforms with the XML policy document <code>dms-vpc-role</code>.For more information on this XML policy document, see <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Security.html#CHAP_Security.APIRole\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Creating the IAM roles to use with the AWS CLI and AWS DMS API</a>.</li> \\n <li>Under <strong>Endpoints</strong>, verify that your newly created VPC endpoint status is <strong>Available</strong>.For more information on creating VPC endpoints for accessing AWS services generally, see <a href=\\"https://docs.aws.amazon.com/vpc/latest/privatelink/create-interface-endpoint.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Access an AWS service using an interface VPC endpoint.</a></li> \\n </ol>\n\n<h3>Create the AWS DMS source and target endpoints</h3>\n\n<p>After you create your VPC endpoint, you need to create source and target database endpoints in AWS DMS. For more information, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Endpoints.Creating.html\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Creating source and target endpoints</a>. Specify your connection information for the source or target AWS services.</p>\\n\\n### **Use a SQL Server secondary availability group replica as a source**\\n\\n<p>The SQL Server Always On availability group feature provides both high availability and read-scale. A read-scale availability group is a set of replicated databases that you can use for read-only workloads. Before AWS DMS v3.4.7, you could use only a primary replica as the source endpoint for migration, which added extra overhead due to replication. Now, we’re delighted to share that AWS DMS supports secondary availability group replicas as a source endpoint.</p>\n\n<p>AWS DMS v3.4.7 introduces support for self managed Always On secondary replicas as a source. RDS SQL Server Multi-AZ read replicas are not supported. This new feature takes advantage of the Always On availability group read-scale feature of SQL Server and sets aside the primary replica for mission-critical production workloads.</p>\n\n<h3>Configure networking</h3>\n\n<p>You can use the following steps to configure your networking and database for AWS DMS migration from a SQL Server secondary replica:</p>\n\n<ol> \\n <li>Make sure that the AWS DMS replication instance can resolve DNS names of all your availability group replicas. If you’re using an on-premises DNS server, make sure you configure <a href=\\"https://aws.amazon.com/route53/\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Amazon Route 53 Resolver</a>.</li> \\n <li>Enable MS publishing and distribution on your primary replica using the following query: \\n <div class=\\"hide-language\\"> \\n <div class=\\"code-toolbar\\"><pre class=\\" language-sql\\"><code class=\\" language-sql\\">sp_replicationdboption <span class=\\"token variable\\">@dbname</span> <span class=\\"token operator\\">=</span> N<span class=\\"token string\\">'<span style=\\"color: #ff0000\\">&lt;source DB name&gt;</span>'</span><span class=\\"token punctuation\\">,</span> <span class=\\"token variable\\">@optname</span> <span class=\\"token operator\\">=</span> N<span class=\\"token string\\">'publish'</span><span class=\\"token punctuation\\">,</span> <span class=\\"token variable\\">@value</span> <span class=\\"token operator\\">=</span> N<span class=\\"token string\\">'true'</span><span class=\\"token punctuation\\">;</span></code></pre><div class=\\"toolbar\\"><div class=\\"toolbar-item\\"><span>SQL</span></div></div></div> \\n </div> </li> \\n <li>Enable the distribution option on all replicas in your availability group. For more information, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.SQLServer.html#CHAP_Source.SQLServer.Configuration\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Setting up ongoing replication using the sysadmin role with self-managed SQL Server</a>.</li> \\n </ol>\n\n<h3>Create a SQL Server endpoint</h3>\n\n<p>In the following example, we select <strong>Source endpoint</strong> as the endpoint type. Enter a name for the endpoint and choose <strong>Microsoft SQL Server</strong> for <strong>Source engine</strong>. Enter the connection details, select <strong>Use endpoint connection attributes</strong>, and add the following <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.SQLServer.html#CHAP_Source.SQLServer.ConnectionAttrib\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">extra connection attributes</a>:</p>\\n\\n<div class=\\"hide-language\\"> \\n <div class=\\"code-toolbar\\"><pre class=\\" language-bash\\"><code class=\\" language-bash\\"><span class=\\"token assign-left variable\\">applicationIntent</span><span class=\\"token operator\\">=</span>ReadOnly<span class=\\"token punctuation\\">;</span><span class=\\"token assign-left variable\\">multiSubnetFailover</span><span class=\\"token operator\\">=</span>yes<span class=\\"token punctuation\\">;</span><span class=\\"token assign-left variable\\">alwaysOnSharedSynchedBackupIsEnabled</span><span class=\\"token operator\\">=</span>false<span class=\\"token punctuation\\">;</span><span class=\\"token assign-left variable\\">activateSafeguard</span><span class=\\"token operator\\">=</span>false<span class=\\"token punctuation\\">;</span><span class=\\"token assign-left variable\\">setUpMsCdcForTables</span><span class=\\"token operator\\">=</span>false </code></pre><div class=\\"toolbar\\"><div class=\\"toolbar-item\\"><span>Bash</span></div></div></div> \\n </div>\n\n\n\n<p>For more information about secondary availability group replicas, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.SQLServer.html#CHAP_Source.SQLServer.AlwaysOn\\" target=\\"_blank\\" rel=\\"noopener noreferrer\\">Working with a secondary availability group replica</a>.</p>\\n\\n### **Summary**\\n\\n<p>In this post, we introduced some of the new features in AWS DMS v3.4.7 requested by our customers. With Db2 for z/OS as a supported endpoint, you can move your workload to any supported DMS target including Aurora or Amazon RDS platforms. You can use Babelfish for Aurora PostgreSQL as your application database without having to rewrite your SQL Server based application. Using a SQL Server Always On secondary replica as a source will save your primary database for mission-critical production workloads and move the replication workload to the secondary replica. VPC endpoints can help you stay within your network and avoid network interruptions.</p>\n\n<p>Please share your feedback in the comments section.</p>\n\n<h3>About the authors</h3>\n\n\n\n<p><strong>Sumit Singh </strong>is a database engineer with Database Migration Service team at Amazon Web Services. He works closely with customers and provide technical assistance to migrate their on-premises workload to AWS cloud. He also assist in continuously improving the quality and functionality of AWS Data migration products.</p>\\n\\n\\n\\n<p style=\\"clear: both\\"><b>Don Tam </b>is a Database Engineer with the [AWS Database Migration Service](https://aws.amazon.com/cn/dms/?trk=cndc-detail) team. He works with engineers to migrate customer’s database platforms to the AWS cloud. He also assist developers in continuously improving the functionality of AWS cloud services.</p>","render":"<p>We are excited to announce the availability of <a href=\\"https://aws.amazon.com/dms/\\" target=\\"_blank\\">AWS Database Migration Service</a> (AWS DMS) replication engine version 3.4.7. This version provides improvements covering security, performance, and connectivity that were requested by many of our customers. In this post, we highlight a few key features. For the entire list of improvements, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReleaseNotes.html\\" target=\\"_blank\\">AWS DMS release notes</a>.</p>\\n<p>We dive deep into the following improvements in AWS DMS v3.4.7:</p>\n<ul> \\n <li>Use IBM Db2 for z/OS as source</li> \n <li>Use AWS Babelfish as a target for AWS DMS</li> \n <li>Configure VPC endpoints as source and target</li> \n <li>Use a Microsoft SQL Server secondary availability group replica as a source</li> \n </ul>\\n<h3><a id=\\"Use_IBM_Db2_for_zOS_as_source_11\\"></a><strong>Use IBM Db2 for z/OS as source</strong></h3>\\n<p>Customers are increasingly interested in migrating from mainframe database systems to the cloud. Before AWS DMS v3.4.7, you had limited automation to migrate IBM Db2 for z/OS workloads into AWS. Now, we’re happy to share that AWS DMS will migrate your IBM Db2 for z/OS workloads to any of the target engines AWS DMS <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Endpoints.html\\" target=\\"_blank\\">supports</a>, including cloud-native databases like <a href=\\"https://aws.amazon.com/rds/aurora/\\" target=\\"_blank\\">Amazon Aurora</a> or managed service, open-source engines like <a href=\\"https://aws.amazon.com/rds/postgresql/\\" target=\\"_blank\\">Amazon Relational Database Service (Amazon RDS) for PostgreSQL</a>.</p>\\n<p>You can use AWS DMS to migrate a full load of your IBM Db2 for z/OS source to your target of choice. You can also use the <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Validating.html\\" target=\\"_blank\\">data validation</a> feature of AWS DMS for supported <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.html\\" target=\\"_blank\\">target endpoints</a>.</p>\\n<p>Use the steps in this section to migrate from IBM Db2 for z/OS to <a href=\\"https://aws.amazon.com/rds/aurora/postgresql-features/\\" target=\\"_blank\\">Amazon Aurora PostgreSQL-Compatible Edition</a>. To demonstrate this migration, we use an existing AWS DMS v3.4.7 replication instance and an existing Aurora PostgreSQL endpoint. To learn more about creating replication instances and endpoints, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReplicationInstance.html\\" target=\\"_blank\\">Working with an AWS DMS replication instance</a> and <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.PostgreSQL.html\\" target=\\"_blank\\">Using a PostgreSQL database as a target for AWS Database Migration Service</a>.</p>\\n<p>Before you start your migration, you need to complete the <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_GettingStarted.Prerequisites.html\\" target=\\"_blank\\">Prerequisites for AWS Database Migration Service</a>.</p>\\n<h4><a id=\\"Use_AWS_SCT_for_schema_conversion_21\\"></a><strong>Use AWS SCT for schema conversion</strong></h4>\\n<p>You can use <a href=\\"https://aws.amazon.com/dms/schema-conversion-tool/\\" target=\\"_blank\\">AWS Schema Conversion Tool</a> (AWS SCT) to migrate your <a href=\\"https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_Source.DB2zOS.html\\" target=\\"_blank\\">IBM Db2 for z/OS schema</a> to any supported target database. AWS SCT is free to download for both Windows and Linux Operating Systems. If you don’t want to convert your database objects and only migrate the data, you can skip this step.</p>\\n<h3>Create an IBM Db2 for z/OS source endpoint in AWS DMS</h3>\n<p>In the following example, we select <strong>Source endpoint</strong> as our endpoint type. We name this endpoint <code>db2zos</code> and choose <strong>IBM Db2 for z/OS</strong> for <strong>Source engine</strong>. Enter the connection details and choose <strong>Create endpoint</strong>.</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/c6df1ef6ab1a4a3bab0a346847c29040_image.png\\" alt=\\"image.png\\" /></p>\n<h3>Create an AWS DMS migration task</h3>\n<p>To migrate the data from the source to target database, you need to create an <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.html\\" target=\\"_blank\\">AWS DMS task</a>.</p>\\n<p>For <strong>Task identifier</strong>, enter a name of your choosing. Choose your replication instance, the newly created Db2 for z/OS source database endpoint, and the existing PostgreSQL target endpoint. For <strong>Migration type</strong>, choose <strong>Migrate existing data</strong> at present AWS DMS v3.4.7 only supports full load for Db2 z/OS.</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/701b07555ba04515bbeafce1c379fb35_image.png\\" alt=\\"image.png\\" /></p>\n<p>Please select the tables/data, define any transformations, and enable <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Validating.html\\" target=\\"_blank\\">data validation</a> as you wish while you continue with this migration task.</p>\\n<p>We’ll be enhancing support for this endpoint in future releases. To learn more about using Db2 endpoint, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.DB2zOS.html\\" target=\\"_blank\\">Using IBM Db2 for z/OS databases as a source for AWS DMS</a>.</p>\\n<h3><a id=\\"Use_Babelfish_as_a_target_for_AWS_DMS_43\\"></a><strong>Use Babelfish as a target for AWS DMS</strong></h3>\\n<p><a href=\\"https://aws.amazon.com/rds/aurora/babelfish/\\" target=\\"_blank\\">Babelfish for Aurora PostgreSQL</a> is a capability for Aurora PostgreSQL that enables Aurora to understand commands from applications written for Microsoft SQL Server.</p>\\n<p>With Babelfish, Aurora PostgreSQL can understand T-SQL, SQL Server’s proprietary SQL dialect, and supports the same communications protocol, so your apps that were originally written for SQL Server can work with Aurora with fewer code changes. As a result, the effort required to modify and move applications running on SQL Server 2005 or newer to Aurora is reduced, leading to faster, lower-risk, and more cost-effective migrations.</p>\n<p>AWS DMS v3.4.7 introduces a new Babelfish target endpoint, which allows you to migrate your SQL Server workloads directly to PostgreSQL. As of this writing, AWS DMS supports full load migration to Babelfish support, and CDC is planned for future releases. If you want to create a Babelfish for Aurora PostgreSQL cluster, refer to <a href=\\"https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/babelfish-create.html\\" target=\\"_blank\\">Creating a Babelfish for Aurora PostgreSQL DB cluster</a>.</p>\\n<p>You can use the following steps to complete a migration from SQL Server to Babelfish for Aurora PostgreSQL. Before you start your migration, you need to complete the <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_GettingStarted.Prerequisites.html\\" target=\\"_blank\\">Prerequisites for AWS Database Migration Service</a>. In this example, we use an existing AWS DMS v3.4.7 replication instance and an existing SQL Server source endpoint. For more information, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReplicationInstance.html\\" target=\\"_blank\\">Working with an AWS DMS replication instance</a> and <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.PostgreSQL.html\\" target=\\"_blank\\">Using a PostgreSQL database as a target for AWS Database Migration Service</a>.</p>\\n<h3>Use AWS SCT to generate an assessment report</h3>\n<p>You can use AWS SCT to <a href=\\"https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_AssessmentReport.html\\" target=\\"_blank\\">generate an assessment report</a>. Babelfish is supported as a <a href=\\"https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_Mapping.VirtualTargets.html\\" target=\\"_blank\\">virtual target </a>in AWS SCT. In the following example, we use the SQL Server <code>AdventureWorksDW</code> database. To generate an assessment report for migration, we need to create a new project and for <strong>Add Source</strong>, choose <strong>Microsoft SQL Server</strong>. Select the <code>Sales</code> schema in the left pane as your source and <code>Babelfish (virtual)</code> in the right pane as your target. We need to create at least one <a href=\\"https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_Mapping.html\\" target=\\"_blank\\">mapping rule</a> in order to <a href=\\"https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_AssessmentReport.Create.html\\" target=\\"_blank\\">create a migration assessment report</a>.</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/bac1ae3a219747419acbb30288f6abd3_image.png\\" alt=\\"image.png\\" /></p>\n<h3>Create a Babelfish endpoint</h3>\n<p>In the following example, we select <strong>Target endpoint</strong> as the endpoint type. We name this endpoint <code>babelfish</code> and choose <strong>Babelfish </strong>for <strong>Target</strong> <strong>engine</strong>. Enter the connection details and choose <strong>Create endpoint</strong>.</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/0be51c71474143a49d4d8efa03214189_image.png\\" alt=\\"image.png\\" /></p>\n<h3>Create an AWS DMS migration task</h3>\n<p>AWS DMS uses an <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.html\\" target=\\"_blank\\">AWS DMS task</a> for data migration. For <strong>Task identifier</strong>, enter a name. Select the existing 3.4.7 replication instance, existing SQL Server endpoint from the source database endpoint, and the newly created Babelfish target endpoint. For <strong>Migration type</strong>, enter a name of your choosing <strong>Migrate existing data</strong> as AWS DMS v3.4.7 does not support CDC for Babelfish at present.</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/cb6ea635483a4b64955d16086c52cfbf_image.png\\" alt=\\"image.png\\" /></p>\n<p>Please select the tables/data, define any transformations, and enable <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Validating.html\\" target=\\"_blank\\">data validation</a> as you wish while you continue with this migration task.</p>\\n<p>To learn more, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.Babelfish.html\\" target=\\"_blank\\">Using Babelfish as a target for AWS Database Migration Service</a>.</p>\\n<h3><a id=\\"Configure_VPC_endpoints_as_source_and_target_75\\"></a><strong>Configure VPC endpoints as source and target</strong></h3>\\n<p>An <a href=\\"http://aws.amazon.com/vpc\\" target=\\"_blank\\">Amazon Virtual Private Cloud</a> ([Amazon VPC](https://aws.amazon.com/cn/vpc/?trk=cndc-detail)) <a href=\\"https://docs.aws.amazon.com/vpc/latest/privatelink/create-interface-endpoint.html\\" target=\\"_blank\\">endpoint</a> is an interface powered by <a href=\\"https://aws.amazon.com/privatelink/\\" target=\\"_blank\\">AWS PrivateLink</a> without requiring an internet gateway, NAT device, VPC connection, or <a href=\\"https://aws.amazon.com/directconnect/\\" target=\\"_blank\\">AWS Direct Connect</a> connection. Starting with v3.4.7, AWS DMS supports VPC endpoints. Your AWS DMS replication instance VPC doesn’t require a public IP address to communicate with AWS services such as <a href=\\"http://aws.amazon.com/s3\\" target=\\"_blank\\">Amazon Simple Storage Service</a> ([Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail)), <a href=\\"https://aws.amazon.com/kinesis/\\" target=\\"_blank\\">Amazon Kinesis</a>, <a href=\\"https://aws.amazon.com/secrets-manager/\\" target=\\"_blank\\">AWS Secrets Manager</a>, <a href=\\"https://aws.amazon.com/dynamodb/\\" target=\\"_blank\\">Amazon DynamoDB</a>, <a href=\\"http://aws.amazon.com/redshift\\" target=\\"_blank\\">Amazon Redshift</a>, and <a href=\\"https://aws.amazon.com/opensearch-service/\\" target=\\"_blank\\">Amazon OpenSearch Service</a>. AWS DMS database endpoints such as Oracle are reachable using native TCP routing and don’t require a VPC endpoint under 3.4.7 when the replication instance is configured with no accessible public IP address.</p>\\n<p>The following diagram illustrates this architecture.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/36091952cd6e42aab00731d2cde6f3d9_image.png\\" alt=\\"image.png\\" /></p>\n<p>Prior to 3.4.7, AWS DMS didn’t allow you to properly use VPC endpoints associated with other AWS services. That limitation resulted in AWS DMS replication traffic to be routed through the Amazon network backbone, which may not have been in line with your networking plan. With VPC endpoints, traffic between your replication instance with no accessible public IP and your other AWS services doesn’t leave the Amazon network.</p>\n<p>Starting with 3.4.7, you need to configure your VPC endpoint if your AWS DMS replication has no publicly accessible IP address and interacts with the services listed earlier. The AWS DMS task will fail when communication with AWS services from the 3.4.7 replication instance has no public accessibility from the internet or VPC endpoint.</p>\n<p>Starting with 3.4.7, AWS DMS can connect to any AWS source or target database with VPC endpoints as long as explicitly defined routes to these endpoints are defined in the AWS DMS endpoint and VPC endpoint.</p>\n<p>With VPC endpoint support, all traffic involved in moving your data remains inside your VPC network. It reduces replication interruptions and improves the quality of the data transfer. To learn more about VPC endpoints, refer to <a href=\\"https://docs.aws.amazon.com/vpc/latest/userguide/endpoint-services-overview.html\\" target=\\"_blank\\">Connect your VPC to services using AWS PrivateLink</a>.</p>\\n<p>In the following example, we show you steps to create an AWS DMS replication instance without a publicly accessible IP, create a VPC endpoint, and set up an AWS DMS endpoint.</p>\n<h3>Create an AWS DMS replication instance without a publicly accessible IP</h3>\n<p>On the <strong>Create replication instance</strong> page of the AWS DMS console, specify your replication instance information and deselect the default <strong>Publicly accessible</strong>. To learn more about creating replication instances, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReplicationInstance.html\\" target=\\"_blank\\">Working with an AWS DMS replication instance</a>.</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/b25d426ea4744064be34235e33a48914_image.png\\" alt=\\"image.png\\" /></p>\n<h3>Create a VPC endpoint</h3>\n<p>In the following example, we show you how to create a VPC endpoint.</p>\n<ol> \\n <li>On the Amazon VPC console, choose the same AWS Region as your AWS DMS replication instance.</li> \n <li>In the navigation pane, choose <strong>Endpoints</strong>.<br> \\n<p><img src=\\"https://dev-media.amazoncloud.cn/d77e073558f84b46b1c37c012945dbd1_image.png\\" alt=\\"image.png\\" /></p>\n</li> \\n <li>Choose <strong>Create endpoint</strong>.</li> \\n <li>You can optionally specify a name tag. For example, <code>my-endpoint-DynamoDB-01</code>.</li> \\n <li>For <strong>Service category</strong>, select <strong>AWS services</strong>.<br> \\n<p><img src=\\"https://dev-media.amazoncloud.cn/227a0254055545a69f1e4f6b96e1e79e_image.png\\" alt=\\"image.png\\" /></li><br />\\n<li>Under <strong>Services</strong>, for [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail) or DynamoDB only, choose a service name with its type set to <strong>Gateway</strong>.<br></p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/656816192be442acb12a39edac0f4d84_image.png\\" alt=\\"image.png\\" /></li><br />\\n<li>For <strong>VPC</strong>, choose the same VPC as our AWS DMS replication instance to create the endpoint.</li><br />\\n<li>For <strong>Route Tables</strong>, choose all available <strong>Route Table ID</strong> values.</li><br />\\n<li>To specify access control, for <strong>Policy</strong>, choose <strong>Full access</strong>. If you want to use a policy creation tool to specify your own access control, choose <strong>Custom</strong>. In any case, use a trust policy that conforms with the XML policy document <code>dms-vpc-role</code>.For more information on this XML policy document, see <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Security.html#CHAP_Security.APIRole\\" target=\\"_blank\\">Creating the IAM roles to use with the AWS CLI and AWS DMS API</a>.</li><br />\\n<li>Under <strong>Endpoints</strong>, verify that your newly created VPC endpoint status is <strong>Available</strong>.For more information on creating VPC endpoints for accessing AWS services generally, see <a href=\\"https://docs.aws.amazon.com/vpc/latest/privatelink/create-interface-endpoint.html\\" target=\\"_blank\\">Access an AWS service using an interface VPC endpoint.</a></li><br />\\n</ol></p>\\n<h3>Create the AWS DMS source and target endpoints</h3>\n<p>After you create your VPC endpoint, you need to create source and target database endpoints in AWS DMS. For more information, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Endpoints.Creating.html\\" target=\\"_blank\\">Creating source and target endpoints</a>. Specify your connection information for the source or target AWS services.</p>\\n<h3><a id=\\"Use_a_SQL_Server_secondary_availability_group_replica_as_a_source_127\\"></a><strong>Use a SQL Server secondary availability group replica as a source</strong></h3>\\n<p>The SQL Server Always On availability group feature provides both high availability and read-scale. A read-scale availability group is a set of replicated databases that you can use for read-only workloads. Before AWS DMS v3.4.7, you could use only a primary replica as the source endpoint for migration, which added extra overhead due to replication. Now, we’re delighted to share that AWS DMS supports secondary availability group replicas as a source endpoint.</p>\n<p>AWS DMS v3.4.7 introduces support for self managed Always On secondary replicas as a source. RDS SQL Server Multi-AZ read replicas are not supported. This new feature takes advantage of the Always On availability group read-scale feature of SQL Server and sets aside the primary replica for mission-critical production workloads.</p>\n<h3>Configure networking</h3>\n<p>You can use the following steps to configure your networking and database for AWS DMS migration from a SQL Server secondary replica:</p>\n<ol> \\n <li>Make sure that the AWS DMS replication instance can resolve DNS names of all your availability group replicas. If you’re using an on-premises DNS server, make sure you configure <a href=\\"https://aws.amazon.com/route53/\\" target=\\"_blank\\">Amazon Route 53 Resolver</a>.</li> \\n <li>Enable MS publishing and distribution on your primary replica using the following query: \\n <div> \\n <div><pre><code>sp_replicationdboption <span>@dbname</span> <span>=</span> N<span>'<span>&lt;source DB name&gt;</span>'</span><span>,</span> <span>@optname</span> <span>=</span> N<span>'publish'</span><span>,</span> <span>@value</span> <span>=</span> N<span>'true'</span><span>;</span></code></pre><div><div><span>SQL</span></div></div></div> \\n </div> </li> \\n <li>Enable the distribution option on all replicas in your availability group. For more information, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.SQLServer.html#CHAP_Source.SQLServer.Configuration\\" target=\\"_blank\\">Setting up ongoing replication using the sysadmin role with self-managed SQL Server</a>.</li> \\n </ol>\n<h3>Create a SQL Server endpoint</h3>\n<p>In the following example, we select <strong>Source endpoint</strong> as the endpoint type. Enter a name for the endpoint and choose <strong>Microsoft SQL Server</strong> for <strong>Source engine</strong>. Enter the connection details, select <strong>Use endpoint connection attributes</strong>, and add the following <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.SQLServer.html#CHAP_Source.SQLServer.ConnectionAttrib\\" target=\\"_blank\\">extra connection attributes</a>:</p>\\n<div> \\n <div><pre><code><span>applicationIntent</span><span>=</span>ReadOnly<span>;</span><span>multiSubnetFailover</span><span>=</span>yes<span>;</span><span>alwaysOnSharedSynchedBackupIsEnabled</span><span>=</span>false<span>;</span><span>activateSafeguard</span><span>=</span>false<span>;</span><span>setUpMsCdcForTables</span><span>=</span>false </code></pre><div><div><span>Bash</span></div></div></div> \\n </div>\n<p><img src=\\"https://dev-media.amazoncloud.cn/28e701ef940e4a99ba13317d92664ae1_image.png\\" alt=\\"image.png\\" /></p>\n<p>For more information about secondary availability group replicas, refer to <a href=\\"https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.SQLServer.html#CHAP_Source.SQLServer.AlwaysOn\\" target=\\"_blank\\">Working with a secondary availability group replica</a>.</p>\\n<h3><a id=\\"Summary_158\\"></a><strong>Summary</strong></h3>\\n<p>In this post, we introduced some of the new features in AWS DMS v3.4.7 requested by our customers. With Db2 for z/OS as a supported endpoint, you can move your workload to any supported DMS target including Aurora or Amazon RDS platforms. You can use Babelfish for Aurora PostgreSQL as your application database without having to rewrite your SQL Server based application. Using a SQL Server Always On secondary replica as a source will save your primary database for mission-critical production workloads and move the replication workload to the secondary replica. VPC endpoints can help you stay within your network and avoid network interruptions.</p>\n<p>Please share your feedback in the comments section.</p>\n<h3>About the authors</h3>\n<p><img src=\\"https://dev-media.amazoncloud.cn/aaec2c42b5c3410b92a21043ec03f576_image.png\\" alt=\\"image.png\\" /></p>\n<p><strong>Sumit Singh </strong>is a database engineer with Database Migration Service team at Amazon Web Services. He works closely with customers and provide technical assistance to migrate their on-premises workload to AWS cloud. He also assist in continuously improving the quality and functionality of AWS Data migration products.</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/1710cfe6eebb4294ad044b92d9180143_image.png\\" alt=\\"image.png\\" /></p>\n<p><b>Don Tam </b>is a Database Engineer with the [AWS Database Migration Service](https://aws.amazon.com/cn/dms/?trk=cndc-detail) team. He works with engineers to migrate customer’s database platforms to the AWS cloud. He also assist developers in continuously improving the functionality of AWS cloud services.</p>"}