Automated Data Discovery for Amazon Macie

{"value":"Today, we announce **automated data discovery** for [Amazon Macie](https://aws.amazon.com/macie/). This new capability allows you to gain visibility into where your sensitive data resides on [Amazon Simple Storage Service (Amazon S3) ](https://aws.amazon.com/s3/)at a fraction of the cost of running a full data inspection across all your S3 buckets.\n\nAt Amazon Web Services, [security is our first priority](https://aws.amazon.com/security/). The security of the infrastructure itself, but also the security of your data. We give you access to [services](https://aws.amazon.com/products/security/) to manage identities and access, to protect the network and your applications, to detect suspicious activities, to protect your data, and to report on and monitor your compliance status.\n\n[Amazon Macie](https://aws.amazon.com/cn/macie/?trk=cndc-detail) is a data security service that discovers sensitive data using machine learning and pattern matching and enables visibility and automated protection against data security risks. You use [Amazon Macie](https://aws.amazon.com/cn/macie/?trk=cndc-detail) to protect your data in S3 by scanning for the presence of sensitive data, such as names, addresses, and credit card numbers, and continually monitoring for properly configured preventative controls, such as encryption and access policies. [Amazon Macie](https://aws.amazon.com/cn/macie/?trk=cndc-detail) generates alerts when it detects publicly accessible buckets, unencrypted buckets, or buckets shared with an Amazon Web Services account outside of your organization. You may also configure [Amazon Macie](https://aws.amazon.com/cn/macie/?trk=cndc-detail) to scan your S3 to run full sensitive data discovery scans on your S3 buckets to provide visibility into where sensitive data resides.\n\nBut customers operating at scale told us it is difficult to know where to start. When employees and applications add new buckets and generate petabytes of data on a daily basis, what should be scanned first?\n\nAutomated data discovery automates the continual discovery of sensitive data and potential data security risks across your entire set of buckets aggregated at [Amazon Web Services Organizations](https://aws.amazon.com/organizations/) level.\n\nWhen you enable automated discovery in the console, Macie starts to evaluate the level of sensitivity of each of your buckets and highlights any data security risks. Automated data discovery introduces intelligent and fully managed data sampling to provide an optimized sample rate that meaningfully reduces the amount of data that needs to be analyzed. This reduces the cost of discovering S3 buckets containing sensitive data compared to the cost of full data inspection.\n\nYou can tune automated data discovery to only identify the types of sensitive data that are relevant for your use case by choosing from over 100 managed sensitive data types, such as personally identifiable information (PII) and financial records with specific formats for multiple countries. For example, you can enable detection of Spanish or Swedish driving license numbers and choose to ignore US Social Security numbers, depending on your use cases. When the specific type of data you manage is not on our list, you can create custom data types that may be unique to your business, such as employee or patient identification numbers.\n\n**++Let’s See It in Action++**\nAutomated data discovery is on by default for all new [Amazon Macie](https://aws.amazon.com/cn/macie/?trk=cndc-detail) customers, and existing Macie customers can enable it with one click in the [Amazon Web Services Management Console](https://console.aws.amazon.com/) of the [Amazon Macie](https://aws.amazon.com/cn/macie/?trk=cndc-detail) administrator account. There is a 30-day free trial, and you can always opt out at the administrator level.\n\nI can enable or disable the capability from the **Automated discovery** entry–under **Settings**–on the left side navigation menu. The **Status** section reveals the current status.\n\n\n\nOn the same page, I can configure the list of managed data identifiers. I can turn on or off individual types of data among more than one hundred managed data identifier types. I can also configure new ones. I select **Edit** on the **Managed data identifiers** section to include or exclude additional data identifiers.\n\n\n\nIf I have some buckets with lots of objects and others with a few, Macie won’t spend all its time inspecting one really large bucket at the expense of other smaller ones. Macie also prioritizes buckets that it knows the least about. For example, if it looked at the majority of objects in a small bucket, that bucket will be deprioritized compared to larger buckets where it has seen proportionally fewer objects.\n\nAutomated data discovery can provide an interactive data map of sensitive data distribution in S3 buckets within days of the feature being enabled. This data map refreshes daily as it intelligently picks and scans S3 objects in buckets and spreads the scan effort across the entire S3 estate in a given month.\n\nHere is the Summary section of the [Amazon Macie](https://aws.amazon.com/cn/macie/?trk=cndc-detail) page. It looks like my set of buckets is secured. I have no bucket with public access, and 31 of my buckets might contain sensitive data.\n\nhttps://d2908q01vomqb2.cloudfront.net/da4b9237bacccdf19c0760cab7aec4a8359010b0/2022/11/24/2022-11-24_11-50-50.png\n\nWhen selecting the **S3 buckets** section of the navigation menu on the left side, I can see a data map of my buckets. The more red the squares are, the more sensitive data are detected in the buckets. The squares in blue represent buckets with no sensitive data detected so far. From there, I can drill down at bucket level to investigate the details.\n\n\n\n**++Pricing and Availability++**\nWhen you are new to [Amazon Macie](https://aws.amazon.com/macie/), automated data discovery is enabled by default. When you already use [Amazon Macie](https://aws.amazon.com/cn/macie/?trk=cndc-detail) in your organization, you can enable automatic data discovery with one click in the Management Console of the [Amazon Macie](https://aws.amazon.com/cn/macie/?trk=cndc-detail) administrator account.\n\nThere is a 30-day free trial period when you enable automatic data discovery on your Amazon Web Services account. After the evaluation period, we charge based on the total quantity of S3 objects in your account as well as the bytes scanned for sensitive content. Charges are prorated per day. You can disable this capability at any time. [The pricing page has all the details](https://aws.amazon.com/macie/pricing/).\n\nThis new capability is now available in [all 21 commercial Amazon Web Services Regions where Macie is available.](https://docs.aws.amazon.com/general/latest/gr/macie.html)\n\n**Go and enable [Amazon Macie](https://aws.amazon.com/macie/) [automated data discovery](https://console.Amazon Web Services .amazon.com/macie) today!**\n\n-- [seb](https://twitter.com/sebsto)\n\n\n\n### [Sébastien Stormacq](https://aws.amazon.com/blogs/aws/author/stormacq/)\nSeb has been writing code since he first touched a Commodore 64 in the mid-eighties. He inspires builders to unlock the value of the Amazon Web Services cloud, using his secret blend of passion, enthusiasm, customer advocacy, curiosity and creativity. His interests are software architecture, developer tools and mobile computing. If you want to sell him something, be sure it has an API. Follow him on Twitter @sebsto.","render":"<p>Today, we announce <strong>automated data discovery</strong> for <a href=\\"https://aws.amazon.com/macie/\\" target=\\"_blank\\">Amazon Macie</a>. This new capability allows you to gain visibility into where your sensitive data resides on <a href=\\"https://aws.amazon.com/s3/\\" target=\\"_blank\\">Amazon Simple Storage Service (Amazon S3) </a>at a fraction of the cost of running a full data inspection across all your S3 buckets.</p>\\n<p>At Amazon Web Services, <a href=\\"https://aws.amazon.com/security/\\" target=\\"_blank\\">security is our first priority</a>. The security of the infrastructure itself, but also the security of your data. We give you access to <a href=\\"https://aws.amazon.com/products/security/\\" target=\\"_blank\\">services</a> to manage identities and access, to protect the network and your applications, to detect suspicious activities, to protect your data, and to report on and monitor your compliance status.</p>\\n<p>Amazon Macie is a data security service that discovers sensitive data using machine learning and pattern matching and enables visibility and automated protection against data security risks. You use Amazon Macie to protect your data in S3 by scanning for the presence of sensitive data, such as names, addresses, and credit card numbers, and continually monitoring for properly configured preventative controls, such as encryption and access policies. Amazon Macie generates alerts when it detects publicly accessible buckets, unencrypted buckets, or buckets shared with an Amazon Web Services account outside of your organization. You may also configure Amazon Macie to scan your S3 to run full sensitive data discovery scans on your S3 buckets to provide visibility into where sensitive data resides.</p>\n<p>But customers operating at scale told us it is difficult to know where to start. When employees and applications add new buckets and generate petabytes of data on a daily basis, what should be scanned first?</p>\n<p>Automated data discovery automates the continual discovery of sensitive data and potential data security risks across your entire set of buckets aggregated at <a href=\\"https://aws.amazon.com/organizations/\\" target=\\"_blank\\">Amazon Web Services Organizations</a> level.</p>\\n<p>When you enable automated discovery in the console, Macie starts to evaluate the level of sensitivity of each of your buckets and highlights any data security risks. Automated data discovery introduces intelligent and fully managed data sampling to provide an optimized sample rate that meaningfully reduces the amount of data that needs to be analyzed. This reduces the cost of discovering S3 buckets containing sensitive data compared to the cost of full data inspection.</p>\n<p>You can tune automated data discovery to only identify the types of sensitive data that are relevant for your use case by choosing from over 100 managed sensitive data types, such as personally identifiable information (PII) and financial records with specific formats for multiple countries. For example, you can enable detection of Spanish or Swedish driving license numbers and choose to ignore US Social Security numbers, depending on your use cases. When the specific type of data you manage is not on our list, you can create custom data types that may be unique to your business, such as employee or patient identification numbers.</p>\n<p><strong><ins>Let’s See It in Action</ins></strong><br />\\nAutomated data discovery is on by default for all new Amazon Macie customers, and existing Macie customers can enable it with one click in the <a href=\\"https://console.aws.amazon.com/\\" target=\\"_blank\\">Amazon Web Services Management Console</a> of the [Amazon Macie](https://aws.amazon.com/cn/macie/?trk=cndc-detail) administrator account. There is a 30-day free trial, and you can always opt out at the administrator level.</p>\\n<p>I can enable or disable the capability from the <strong>Automated discovery</strong> entry–under <strong>Settings</strong>–on the left side navigation menu. The <strong>Status</strong> section reveals the current status.</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/a5fe1ea6cac34f1687ace49efc104a6c_image.png\\" alt=\\"image.png\\" /></p>\n<p>On the same page, I can configure the list of managed data identifiers. I can turn on or off individual types of data among more than one hundred managed data identifier types. I can also configure new ones. I select <strong>Edit</strong> on the <strong>Managed data identifiers</strong> section to include or exclude additional data identifiers.</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/d9cfac6b70584825a22826e68929d026_image.png\\" alt=\\"image.png\\" /></p>\n<p>If I have some buckets with lots of objects and others with a few, Macie won’t spend all its time inspecting one really large bucket at the expense of other smaller ones. Macie also prioritizes buckets that it knows the least about. For example, if it looked at the majority of objects in a small bucket, that bucket will be deprioritized compared to larger buckets where it has seen proportionally fewer objects.</p>\n<p>Automated data discovery can provide an interactive data map of sensitive data distribution in S3 buckets within days of the feature being enabled. This data map refreshes daily as it intelligently picks and scans S3 objects in buckets and spreads the scan effort across the entire S3 estate in a given month.</p>\n<p>Here is the Summary section of the Amazon Macie page. It looks like my set of buckets is secured. I have no bucket with public access, and 31 of my buckets might contain sensitive data.</p>\n<p>https://d2908q01vomqb2.cloudfront.net/da4b9237bacccdf19c0760cab7aec4a8359010b0/2022/11/24/2022-11-24_11-50-50.png</p>\n<p>When selecting the <strong>S3 buckets</strong> section of the navigation menu on the left side, I can see a data map of my buckets. The more red the squares are, the more sensitive data are detected in the buckets. The squares in blue represent buckets with no sensitive data detected so far. From there, I can drill down at bucket level to investigate the details.</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/57c86e9cff7d47a39482d61dd777c084_image.png\\" alt=\\"image.png\\" /></p>\n<p><strong><ins>Pricing and Availability</ins></strong><br />\\nWhen you are new to <a href=\\"https://aws.amazon.com/macie/\\" target=\\"_blank\\">Amazon Macie</a>, automated data discovery is enabled by default. When you already use [Amazon Macie](https://aws.amazon.com/cn/macie/?trk=cndc-detail) in your organization, you can enable automatic data discovery with one click in the Management Console of the [Amazon Macie](https://aws.amazon.com/cn/macie/?trk=cndc-detail) administrator account.</p>\\n<p>There is a 30-day free trial period when you enable automatic data discovery on your Amazon Web Services account. After the evaluation period, we charge based on the total quantity of S3 objects in your account as well as the bytes scanned for sensitive content. Charges are prorated per day. You can disable this capability at any time. <a href=\\"https://aws.amazon.com/macie/pricing/\\" target=\\"_blank\\">The pricing page has all the details</a>.</p>\\n<p>This new capability is now available in <a href=\\"https://docs.aws.amazon.com/general/latest/gr/macie.html\\" target=\\"_blank\\">all 21 commercial Amazon Web Services Regions where Macie is available.</a></p>\\n<p><strong>Go and enable <a href=\\"https://aws.amazon.com/macie/\\" target=\\"_blank\\">Amazon Macie</a> [automated data discovery](https://console.Amazon Web Services .amazon.com/macie) today!</strong></p>\n<p>– <a href=\\"https://twitter.com/sebsto\\" target=\\"_blank\\">seb</a></p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/89a0bae7cd4d468e84368285fb8612f4_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"Sbastien_Stormacqhttpsawsamazoncomblogsawsauthorstormacq_50\\"></a><a href=\\"https://aws.amazon.com/blogs/aws/author/stormacq/\\" target=\\"_blank\\">Sébastien Stormacq</a></h3>\\n<p>Seb has been writing code since he first touched a Commodore 64 in the mid-eighties. He inspires builders to unlock the value of the Amazon Web Services cloud, using his secret blend of passion, enthusiasm, customer advocacy, curiosity and creativity. His interests are software architecture, developer tools and mobile computing. If you want to sell him something, be sure it has an API. Follow him on Twitter @sebsto.</p>\n"}