How we halved the publish size of modular Amazon SDK for JavaScript clients

{"value":"On December 15th, 2020, we announced the general availability of the [AWS SDK for JavaScript, version 3 (v3)](https://aws.amazon.com/blogs/developer/modular-aws-sdk-for-javascript-is-now-generally-available/). In v3, the [modular packages](https://aws.amazon.com/blogs/developer/modular-packages-in-aws-sdk-for-javascript/) reduce the bundle size of your application by **~75%** as compared to that in AWS SDK for JavaScript, version 2 (v2). However, v3 had a large publish/install size for each modular package. In this blog post, we cover how we reduced the publish size of v3 modular packages by **~50%**.\n\n### **Why did we do it?**\n\nThe JavaScript community often jokes about the large size of ```node_modules``` 😉\n\n\n\nAs such, we are always looking for opportunities to reduce the size of our packages. While working with the [AWS Lambda](https://aws.amazon.com/cn/lambda/?trk=cndc-detail) team to provide v3, we focused on reducing the install size of modular packages. There were [improvement](https://github.com/aws/aws-sdk-js-v3/issues/2132) [suggestions](https://github.com/aws/aws-sdk-js-v3/issues/2765) from the community, and we had [backlog](https://github.com/aws/aws-sdk-js-v3/issues/1594) [items](https://github.com/aws/aws-sdk-js-v3/issues/1306) to work on.\n\n### **What did we accomplish?**\n\nWe are happy to report that we reduced the publish size of v3 modular packages by **~50%** in **v3.36.1** as compared to that in **v3.33.0**!. As an effect, the install size for each client is also reduced by **~40%**.\n\nYou can check the cost of installing ```@aws-sdk/*``` modular packages on [packagephobia](https://packagephobia.com/). The below screenshot for ```@aws-sdk/client-sts``` shows publish/install size reductions in versions leading to v3.36.1:\n\n\n\n#### **Install size reductions in top clients**\n\nThe overall unpacked publish size reduction in clients ranges between **40%** to **60%**. For example, the graph below shows the unpacked publish size reduction for top 5 downloaded clients:\n\n```\\n@aws-sdk/client-sso : [███████████████░░░░░░░░░░░░░░░░░░░░░░] 41.39%\\n@aws-sdk/client-sts : [█████████████████████░░░░░░░░░░░░░░░░] 58.09%\\n@aws-sdk/client-s3 : [██████████████████████░░░░░░░░░░░░░░░] 59.09%\\n@aws-sdk/client-cognito-identity: [██████████████████░░░░░░░░░░░░░░░░░░░] 49.94%\\n@aws-sdk/client-firehose : [███████████████████░░░░░░░░░░░░░░░░░░] 51.11%\\n```\n\nFor the largest 5 clients, the unpacked publish size got reduced **>50%**:\n\n```\\n@aws-sdk/client-ec2 : [████████████████████░░░░░░░░░░░░░░░░░] 53.85%\\n@aws-sdk/client-sagemaker : [████████████████████░░░░░░░░░░░░░░░░░] 53.76%\\n@aws-sdk/client-iot : [███████████████████░░░░░░░░░░░░░░░░░░] 51.63%\\n@aws-sdk/client-chime : [███████████████████░░░░░░░░░░░░░░░░░░] 50.47%\\n@aws-sdk/client-rds : [████████████████████░░░░░░░░░░░░░░░░░] 54.22%\\n```\n\n\n#### **node_modules size comparison between different versions**\n\nIf you install a client in a fresh workspace and check the size of the package inside node_modules, you will see the disk-usage reduction in v3.36.1.\n\nFor example, installing ```@aws-sdk/client-sts@3.33.0``` creates a node_modules with size of **8.9 MB**. The client-sts is of size **1.4 MB** and contains **115** files with **10054** lines of code.\n\n```\\n\\n\$ npm install @aws-sdk/client-sts@3.33.0 --save-exact\\n...\\n\\n\$ du -sh --apparent-size node_modules \\n8.9M node_modules\\n\\n\$ du -sh --apparent-size node_modules/@aws-sdk/client-sts\\n1.4M node_modules/@aws-sdk/client-sts\\n\\n\$ npx cloc node_modules/@aws-sdk/client-sts \\n 163 text files.\\n 160 unique files. \\n 48 files ignored.\\n-------------------------------------------------------------------------------\\nLanguage files blank comment code\\n-------------------------------------------------------------------------------\\nJavaScript 45 0 1948 4674\\nTypeScript 64 321 7506 4295\\nMarkdown 2 1679 0 825\\nJSON 4 0 0 260\\n-------------------------------------------------------------------------------\\nSUM: 115 2000 9454 10054\\n-------------------------------------------------------------------------------\\n\\n```\nThe version ```@aws-sdk/client-sts@3.36.1``` creates a node_modules with size of **5.7 MB**. The client-sts is of size **603 KB** and contains **85** files with **6585** lines of code.\n\n```\\n\$ npm install @aws-sdk/client-sts@3.36.1 --save-exact\\n...\\n\\n\$ du -sh --apparent-size node_modules\\n5.7M node_modules\\n\\n\$ du -sh --apparent-size node_modules/@aws-sdk/client-sts\\n603K node_modules/@aws-sdk/client-sts\\n\\n\$ npx cloc node_modules/@aws-sdk/client-sts \\n 88 text files.\\n 86 unique files. \\n 3 files ignored.\\n-------------------------------------------------------------------------------\\nLanguage files blank comment code\\n-------------------------------------------------------------------------------\\nJavaScript 40 0 0 4383\\nTypeScript 42 185 2481 1236\\nMarkdown 2 1711 0 840\\nJSON 1 0 0 126\\n-------------------------------------------------------------------------------\\nSUM: 85 1896 2481 6585\\n-------------------------------------------------------------------------------\\n```\n### **How did we do it?**\n\nWe created a copy of the client-s3 source code in [trivikr/temp-client-s3](https://github.com/trivikr/temp-client-s3). This allowed us to move fast, quickly implement and test ideas, and quantify the publish/install size reductions. We went through each file being published to npm in the client-s3 package and asked ourselves what role it plays. We brainstormed ideas and documented them in GitHub issues. Then, we implemented those ideas in descending order of return on investment.\n\nWe released each version of [@trivikr-test/client-s3 ](https://www.npmjs.com/package/@trivikr-test/client-s3)with a specific change and documented the metrics from npm publish. We tested the debugging experience in [trivikr/debug-temp-client-s3](https://github.com/trivikr/debug-temp-client-s3), and verified functionality in Node.js, browser and react-native environments in [aws-samples/aws-sdk-js-tests#103](https://github.com/aws-samples/aws-sdk-js-tests/pull/103).\n\nWe shared details about these improvements across multiple JavaScript interest channels including Github, Twitter, and even internal Slack rooms to get early feedback from the community.\n\nWe received overall positive feedback from developers:\n\n\n\n\n\n\n\n\n\nThe key learnings are:\n\n- Provide developers with experimental artifacts to play with.\n- Ask specific questions to get responses.\n- Iterate on their feedback!\n\n### **What did we change?**\n\nOnce we quantified npm publish numbers for changes, we shortlisted the four best improvements to implement in v3:\n\n- We removed comments from transpiled ```*.js``` files.\n- We removed comments from downleveled ```*.d.ts files```.\n- We removed TypeScript source code.\n- We removed source map files.\n\n\nThe v3 SDK is written in [TypeScript](https://www.typescriptlang.org/) programming language. TypeScript extends JavaScript by adding types, and saves you time catching errors and providing fixes before you run your code. We’ve covered in detail why we chose TypeScript in the blog post on [first-class TypeScript support](https://aws.amazon.com/blogs/developer/first-class-typescript-support-in-modular-aws-sdk-for-javascript/).\n\n### **We removed comments from transpiled ```* .js``` files**\n\nWe transpile TypeScript code to JavaScript in commonjs target for Node.js and es5 target for browser. We also ship types as a distribution in a different folder.\n\nTo assist customers, the services ship with extensive documentation for service and operations. We add this documentation in JSDoc comments. In our TSConfig setup, we shipped redundant comments in every distribution.\n\nThe JSDoc comments appear when you hover over Symbols in your code. In the below example, you see JSDoc for DynamoDB when hovering over the import.\n\n\n\nThis JSDoc is from ```*.d.ts``` files. We removed the redundant comments from transpiled *.js files and it led to [**~6%** reduction](https://github.com/aws/aws-sdk-js-v3/pull/2893) in unpacked publish size.\n\n```\\n\$ pwd\\n/home/trivikr/workspace/aws-sdk-js-v3/clients/client-sts\\n\\n# Before the change\\n\$ du -sh --apparent-size dist/cjs | cut -f1\\n301K\\n\\n\$ npx cloc dist/cjs\\n...\\nLanguage files blank comment code\\n-------------------------------------------------------------------------------\\nJavaScript 22 0 974 2328\\n...\\n\\n# After the change\\n\$ du -sh --apparent-size dist/cjs | cut -f1 \\n239K\\n\\n\$ npx cloc dist/cjs\\n...\\nLanguage files blank comment code\\n-------------------------------------------------------------------------------\\nJavaScript 22 0 0 1354\\n...\\n```\n### **We removed comments from downleveled ```* .d.ts``` files**\n\nTo support customers who use older versions of TypeScript, we downlevel types using [downlevel-dts](https://www.npmjs.com/package/downlevel-dts) which converts code with new TypeScript features into code that uses equivalent old features. This feature adds duplicate comments in downleveled types which increases publish size.\n\nWe removed comments from downleveled ```*.d.ts``` files which led to [**~10%** reduction](https://github.com/aws/aws-sdk-js-v3/pull/2891) in unpacked publish size.\n\n```\\n\$ pwd\\n/home/trivikr/workspace/aws-sdk-js-v3/clients/client-sts\\n\\n# Before the change\\n\$ du -sh --apparent-size dist-types | cut -f1\\n742K\\n\\n\$ npx cloc dist-types\\n...\\nLanguage files blank comment code\\n-------------------------------------------------------------------------------\\nTypeScript 59 0 4962 1813\\n...\\n\\n# After the change\\n\$ du -sh --apparent-size dist-types | cut -f1\\n396K\\n\\n\$ npx cloc dist-types\\n...\\nLanguage files blank comment code\\n-------------------------------------------------------------------------------\\nTypeScript 59 185 2481 1813\\n...\\n```\n\nAs a consequence, customers using versions of TypeScript older than 4.0 will not see JSDoc comments in their IDE although the downleveled types will work. We went ahead with this change to encourage customers to switch to TypeScript 4.0+ which was released in [August 2020](https://devblogs.microsoft.com/typescript/announcing-typescript-4-0/).\n\nIf you are using TypeScript <4.0, the JSDoc comments will appear for SDK versions < 3.36.0.\\n\\n\\n#### We removed TypeScript source code\\n\\nThe authors of JavaScript libraries decide the language to write a library in based on variety of reasons. For example, we use TypeScript for v3 for reasons explained in the blog post on [first-class TypeScript support](https://aws.amazon.com/blogs/developer/first-class-typescript-support-in-modular-aws-sdk-for-javascript/). Other maintainers may choose a different language for writing their library: JavaScript, ReScript, PureScript, ClosureScript, CoffeeScript, Reason, Elm, Flow, etc. The consumer of their library does not have to know the language a library is written in. At the end, the engine processes JavaScript code.\\n\\nTo provide first-class TypeScript support, a library needs to ship types. If a library is not written in TypeScript, they either manually write types, or use TypeScript to generate type declarations.\\n\\nWe asked the question on Twitter if maintainers ship the source code in npm packages. Here is a quote from one of the replies: “Shipping source code is going against the spirit of module definition”.\\n\\n\\n\\n\\n\\nWe removed the source code from our published packages along with other dev/test configurations, which led to [**~28%** reduction](https://github.com/aws/aws-sdk-js-v3/pull/2873) in unpacked publish size.\\n\\n```\\n\$ pwd\\n/home/trivikr/workspace/aws-sdk-js-v3/clients/client-sts\\n\\n# Before the change\\n\$ npm pack --dry-run\\n...\\npackage size: 141.7 kB \\nunpacked size: 1.2 MB \\ntotal files: 177\\n...\\n\\n# After the change\\n\$ npm pack --dry-run\\n...\\npackage size: 99.9 kB \\nunpacked size: 783.8 kB \\ntotal files: 88 \\n...\\n```\\n\\n#### **We removed source map files**\\n\\n[Source map files](https://developer.mozilla.org/en-US/docs/Tools/Debugger/How_to/Use_a_source_map) allow debuggers and other tools to display the original TypeScript source code when actually working with the emitted JavaScript files. In TypeScript, the source map files are emitted as ```.js.map``` (or ```.jsx.map```) files next to the corresponding .js output file. TypeScript also allows embedding the source map content in the ```.js``` files. TypeScript also allows for inclusion of the original content of the ```.ts``` file as an embedded string in the source map.\\n\\nThe source map files are helpful with debugging the application code. They are not useful for libraries and dependencies who have to meet strict publish/install size restrictions. A better solution is to release debug versions of the SDK. If you have feedback about source maps, or would like to explain your debugging or other use cases, please comment on GitHub issue [aws/aws-sdk-js-v3/#2895](https://github.com/aws/aws-sdk-js-v3/issues/2895).\\n\\nIn the [TypeScript publishing guide](https://www.typescriptlang.org/docs/handbook/declaration-files/publishing.html), there is no recommendation on publishing source maps. In TSConfig, the settings for [sourceMap](https://www.typescriptlang.org/tsconfig#sourceMap), [inlineSourceMap](https://www.typescriptlang.org/tsconfig#inlineSources) and [inlineSources](https://www.typescriptlang.org/tsconfig#inlineSources) are turned off by default.\\n\\nWe removed sourceMaps from v3, which led to [**~20%** reduction](https://github.com/aws/aws-sdk-js-v3/pull/2867) in unpacked publish size.\\n\\n```\\n\$ pwd\\n/home/trivikr/workspace/aws-sdk-js-v3/clients/client-sts\\n\\n# Before the change\\n\$ du -sh --apparent-size dist-cjs | cut -f1\\n246K\\n\\n\$ npx cloc dist-cjs\\n...\\n 42 text files.\\n 42 unique files. \\n 21 files ignored.\\n...\\n\\n# After the change\\n\$ du -sh --apparent-size dist-cjs | cut -f1\\n174K\\n\\n\$ npx cloc dist-cjs\\n...\\n 21 text files.\\n 21 unique files. \\n 0 files ignored.\\n...\\n\\n```\\n\\n\\n### **What was community’s reaction?**\\n\\nWe had an overwhelming response from the Twitter community to these exciting news!\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nJoin [the conversation on Twitter ](https://twitter.com/intent/tweet?text=%23aws%20%23javascript%20%23nodejs)and let us know how you have reduced publish/install/bundle sizes in your npm packages or any other experiences you’ve had with AWS SDK for JavaScript.\\n\\n### **What do we plan to do in the future?**\\n\\nTo ensure that we keep bloat from sneaking back in, [we plan to add CI](https://github.com/aws/aws-sdk-js-v3/issues/2816) to check publish size.\\n\\nWe’ve not reduced the size of the actual source code yet. For example, a [generic function for API calls](https://github.com/trivikr/temp-client-s3/pull/40) will reduce the size of the source code by ~0.5%. If you have any ideas to reduce publish size, do post them in our experimental repo at [trivikr/temp-client-s3/issues](https://github.com/trivikr/temp-client-s3/issues).\\n\\nWe have also not considered using advanced or alternative compilation options like Google Closure Compiler, Babel or SWC. If you have ideas/suggestions or examples of how they can help, please comment on GitHub issue [aws/aws-sdk-js-v3/#2897](https://github.com/aws/aws-sdk-js-v3/issues/2897).\\n\\nWe are also considering shipping Node.js specific distributions and a type definitions distribution in separate prerelease version numbers, which could further reduce npm install size from between ```60%``` and ```75%```. Although the expected improvements are huge, it would take lot of discussion and testing prior to implementation. Please post your ideas/suggestions/concerns on GitHub Issue [aws/aws-sdk-js-v3/#2889](https://github.com/aws/aws-sdk-js-v3/issues/2889).\\n\\nIf you have a feedback about TypeScript source code and source maps, or would like to explain your debugging or other use cases, please comment on GitHub issue [aws/aws-sdk-js-v3/#2895](https://github.com/aws/aws-sdk-js-v3/issues/2895).\\n\\n### **How can you contribute?**\\n\\nWe value your feedback, so please tell us what you like and don’t like by [opening an issue on GitHub](https://github.com/aws/aws-sdk-js-v3/issues/new/choose). Reducing install/publish size is a [two-way doors decision](https://www.inc.com/jeff-haden/amazon-founder-jeff-bezos-this-is-how-successful-people-make-such-smart-decisions.html). Do inform us on GitHub if it breaks your developer experience.\\n\\n\\n\\n### [**Trivikram Kamat**](https://twitter.com/trivikram)\\n\\nTrivikram is maintainer of AWS SDK for JavaScript in Node.js and browser. Trivikram is also a Node.js Core collaborator and have contributed to HTTP, HTTP/2 and HTTP/3 over QUIC implementations in the past. He has been writing JavaScript for over a decade. You can find him on Twitter [@trivikram](https://twitter.com/trivikram) and GitHub [@trivikr](https://github.com/trivikr).","render":"<p>On December 15th, 2020, we announced the general availability of the <a href=\\"https://aws.amazon.com/blogs/developer/modular-aws-sdk-for-javascript-is-now-generally-available/\\" target=\\"_blank\\">AWS SDK for JavaScript, version 3 (v3)</a>. In v3, the <a href=\\"https://aws.amazon.com/blogs/developer/modular-packages-in-aws-sdk-for-javascript/\\" target=\\"_blank\\">modular packages</a> reduce the bundle size of your application by <strong>~75%</strong> as compared to that in AWS SDK for JavaScript, version 2 (v2). However, v3 had a large publish/install size for each modular package. In this blog post, we cover how we reduced the publish size of v3 modular packages by <strong>~50%</strong>.</p>\\n<h3><a id=\\"Why_did_we_do_it_2\\"></a><strong>Why did we do it?</strong></h3>\\n<p>The JavaScript community often jokes about the large size of <code>node_modules</code> 😉</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/6e009eb9a6ff46ee903ca57121e4ec3a_image.png\\" alt=\\"image.png\\" /></p>\n<p>As such, we are always looking for opportunities to reduce the size of our packages. While working with the AWS Lambda team to provide v3, we focused on reducing the install size of modular packages. There were <a href=\\"https://github.com/aws/aws-sdk-js-v3/issues/2132\\" target=\\"_blank\\">improvement</a> <a href=\\"https://github.com/aws/aws-sdk-js-v3/issues/2765\\" target=\\"_blank\\">suggestions</a> from the community, and we had <a href=\\"https://github.com/aws/aws-sdk-js-v3/issues/1594\\" target=\\"_blank\\">backlog</a> <a href=\\"https://github.com/aws/aws-sdk-js-v3/issues/1306\\" target=\\"_blank\\">items</a> to work on.</p>\\n<h3><a id=\\"What_did_we_accomplish_10\\"></a><strong>What did we accomplish?</strong></h3>\\n<p>We are happy to report that we reduced the publish size of v3 modular packages by <strong>~50%</strong> in <strong>v3.36.1</strong> as compared to that in <strong>v3.33.0</strong>!. As an effect, the install size for each client is also reduced by <strong>~40%</strong>.</p>\\n<p>You can check the cost of installing <code>@aws-sdk/*</code> modular packages on <a href=\\"https://packagephobia.com/\\" target=\\"_blank\\">packagephobia</a>. The below screenshot for <code>@aws-sdk/client-sts</code> shows publish/install size reductions in versions leading to v3.36.1:</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/7a67588d9b9e41aa8da0e419a323975d_image.png\\" alt=\\"image.png\\" /></p>\n<h4><a id=\\"Install_size_reductions_in_top_clients_18\\"></a><strong>Install size reductions in top clients</strong></h4>\\n<p>The overall unpacked publish size reduction in clients ranges between <strong>40%</strong> to <strong>60%</strong>. For example, the graph below shows the unpacked publish size reduction for top 5 downloaded clients:</p>\\n<pre><code class=\\"lang-\\">@aws-sdk/client-sso : [███████████████░░░░░░░░░░░░░░░░░░░░░░] 41.39%\\n@aws-sdk/client-sts : [█████████████████████░░░░░░░░░░░░░░░░] 58.09%\\n@aws-sdk/client-s3 : [██████████████████████░░░░░░░░░░░░░░░] 59.09%\\n@aws-sdk/client-cognito-identity: [██████████████████░░░░░░░░░░░░░░░░░░░] 49.94%\\n@aws-sdk/client-firehose : [███████████████████░░░░░░░░░░░░░░░░░░] 51.11%\\n</code></pre>\\n<p>For the largest 5 clients, the unpacked publish size got reduced <strong>&gt;50%</strong>:</p>\\n<pre><code class=\\"lang-\\">@aws-sdk/client-ec2 : [████████████████████░░░░░░░░░░░░░░░░░] 53.85%\\n@aws-sdk/client-sagemaker : [████████████████████░░░░░░░░░░░░░░░░░] 53.76%\\n@aws-sdk/client-iot : [███████████████████░░░░░░░░░░░░░░░░░░] 51.63%\\n@aws-sdk/client-chime : [███████████████████░░░░░░░░░░░░░░░░░░] 50.47%\\n@aws-sdk/client-rds : [████████████████████░░░░░░░░░░░░░░░░░] 54.22%\\n</code></pre>\\n<h4><a id=\\"node_modules_size_comparison_between_different_versions_41\\"></a><strong>node_modules size comparison between different versions</strong></h4>\\n<p>If you install a client in a fresh workspace and check the size of the package inside node_modules, you will see the disk-usage reduction in v3.36.1.</p>\n<p>For example, installing <code>@aws-sdk/client-sts@3.33.0</code> creates a node_modules with size of <strong>8.9 MB</strong>. The client-sts is of size <strong>1.4 MB</strong> and contains <strong>115</strong> files with <strong>10054</strong> lines of code.</p>\\n<pre><code class=\\"lang-\\">\\n\$ npm install @aws-sdk/client-sts@3.33.0 --save-exact\\n...\\n\\n\$ du -sh --apparent-size node_modules \\n8.9M node_modules\\n\\n\$ du -sh --apparent-size node_modules/@aws-sdk/client-sts\\n1.4M node_modules/@aws-sdk/client-sts\\n\\n\$ npx cloc node_modules/@aws-sdk/client-sts \\n 163 text files.\\n 160 unique files. \\n 48 files ignored.\\n-------------------------------------------------------------------------------\\nLanguage files blank comment code\\n-------------------------------------------------------------------------------\\nJavaScript 45 0 1948 4674\\nTypeScript 64 321 7506 4295\\nMarkdown 2 1679 0 825\\nJSON 4 0 0 260\\n-------------------------------------------------------------------------------\\nSUM: 115 2000 9454 10054\\n-------------------------------------------------------------------------------\\n\\n</code></pre>\\n<p>The version <code>@aws-sdk/client-sts@3.36.1</code> creates a node_modules with size of <strong>5.7 MB</strong>. The client-sts is of size <strong>603 KB</strong> and contains <strong>85</strong> files with <strong>6585</strong> lines of code.</p>\\n<pre><code class=\\"lang-\\">\$ npm install @aws-sdk/client-sts@3.36.1 --save-exact\\n...\\n\\n\$ du -sh --apparent-size node_modules\\n5.7M node_modules\\n\\n\$ du -sh --apparent-size node_modules/@aws-sdk/client-sts\\n603K node_modules/@aws-sdk/client-sts\\n\\n\$ npx cloc node_modules/@aws-sdk/client-sts \\n 88 text files.\\n 86 unique files. \\n 3 files ignored.\\n-------------------------------------------------------------------------------\\nLanguage files blank comment code\\n-------------------------------------------------------------------------------\\nJavaScript 40 0 0 4383\\nTypeScript 42 185 2481 1236\\nMarkdown 2 1711 0 840\\nJSON 1 0 0 126\\n-------------------------------------------------------------------------------\\nSUM: 85 1896 2481 6585\\n-------------------------------------------------------------------------------\\n</code></pre>\\n<h3><a id=\\"How_did_we_do_it_101\\"></a><strong>How did we do it?</strong></h3>\\n<p>We created a copy of the client-s3 source code in <a href=\\"https://github.com/trivikr/temp-client-s3\\" target=\\"_blank\\">trivikr/temp-client-s3</a>. This allowed us to move fast, quickly implement and test ideas, and quantify the publish/install size reductions. We went through each file being published to npm in the client-s3 package and asked ourselves what role it plays. We brainstormed ideas and documented them in GitHub issues. Then, we implemented those ideas in descending order of return on investment.</p>\\n<p>We released each version of <a href=\\"https://www.npmjs.com/package/@trivikr-test/client-s3\\" target=\\"_blank\\">@trivikr-test/client-s3 </a>with a specific change and documented the metrics from npm publish. We tested the debugging experience in <a href=\\"https://github.com/trivikr/debug-temp-client-s3\\" target=\\"_blank\\">trivikr/debug-temp-client-s3</a>, and verified functionality in Node.js, browser and react-native environments in <a href=\\"https://github.com/aws-samples/aws-sdk-js-tests/pull/103\\" target=\\"_blank\\">aws-samples/aws-sdk-js-tests#103</a>.</p>\\n<p>We shared details about these improvements across multiple JavaScript interest channels including Github, Twitter, and even internal Slack rooms to get early feedback from the community.</p>\n<p>We received overall positive feedback from developers:</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/2a361eed688041d0a212ddd8b5b16192_image.png\\" alt=\\"image.png\\" /></p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/d3298d0fb4564abdbac3cdd246057e91_image.png\\" alt=\\"image.png\\" /></p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/2ef7e95f8a92437bbcccb08a77f40ae5_image.png\\" alt=\\"image.png\\" /></p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/607b7d2f9ddc4afebd800878a7a5b47b_image.png\\" alt=\\"image.png\\" /></p>\n<p>The key learnings are:</p>\n<ul>\\n<li>Provide developers with experimental artifacts to play with.</li>\n<li>Ask specific questions to get responses.</li>\n<li>Iterate on their feedback!</li>\n</ul>\\n<h3><a id=\\"What_did_we_change_125\\"></a><strong>What did we change?</strong></h3>\\n<p>Once we quantified npm publish numbers for changes, we shortlisted the four best improvements to implement in v3:</p>\n<ul>\\n<li>We removed comments from transpiled <code>*.js</code> files.</li>\\n<li>We removed comments from downleveled <code>*.d.ts files</code>.</li>\\n<li>We removed TypeScript source code.</li>\n<li>We removed source map files.</li>\n</ul>\\n<p>The v3 SDK is written in <a href=\\"https://www.typescriptlang.org/\\" target=\\"_blank\\">TypeScript</a> programming language. TypeScript extends JavaScript by adding types, and saves you time catching errors and providing fixes before you run your code. We’ve covered in detail why we chose TypeScript in the blog post on <a href=\\"https://aws.amazon.com/blogs/developer/first-class-typescript-support-in-modular-aws-sdk-for-javascript/\\" target=\\"_blank\\">first-class TypeScript support</a>.</p>\\n<h3><a id=\\"We_removed_comments_from_transpiled__js_files_137\\"></a><strong>We removed comments from transpiled <code>* .js</code> files</strong></h3>\n<p>We transpile TypeScript code to JavaScript in commonjs target for Node.js and es5 target for browser. We also ship types as a distribution in a different folder.</p>\n<p>To assist customers, the services ship with extensive documentation for service and operations. We add this documentation in JSDoc comments. In our TSConfig setup, we shipped redundant comments in every distribution.</p>\n<p>The JSDoc comments appear when you hover over Symbols in your code. In the below example, you see JSDoc for DynamoDB when hovering over the import.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/80e3c8cd50d24a60be80019718e13720_image.png\\" alt=\\"image.png\\" /></p>\n<p>This JSDoc is from <code>*.d.ts</code> files. We removed the redundant comments from transpiled *.js files and it led to <a href=\\"https://github.com/aws/aws-sdk-js-v3/pull/2893\\" target=\\"_blank\\"><strong>~6%</strong> reduction</a> in unpacked publish size.</p>\n<pre><code class=\\"lang-\\">\$ pwd\\n/home/trivikr/workspace/aws-sdk-js-v3/clients/client-sts\\n\\n# Before the change\\n\$ du -sh --apparent-size dist/cjs | cut -f1\\n301K\\n\\n\$ npx cloc dist/cjs\\n...\\nLanguage files blank comment code\\n-------------------------------------------------------------------------------\\nJavaScript 22 0 974 2328\\n...\\n\\n# After the change\\n\$ du -sh --apparent-size dist/cjs | cut -f1 \\n239K\\n\\n\$ npx cloc dist/cjs\\n...\\nLanguage files blank comment code\\n-------------------------------------------------------------------------------\\nJavaScript 22 0 0 1354\\n...\\n</code></pre>\\n<h3><a id=\\"We_removed_comments_from_downleveled__dts_files_175\\"></a><strong>We removed comments from downleveled <code>* .d.ts</code> files</strong></h3>\n<p>To support customers who use older versions of TypeScript, we downlevel types using <a href=\\"https://www.npmjs.com/package/downlevel-dts\\" target=\\"_blank\\">downlevel-dts</a> which converts code with new TypeScript features into code that uses equivalent old features. This feature adds duplicate comments in downleveled types which increases publish size.</p>\\n<p>We removed comments from downleveled <code>*.d.ts</code> files which led to <a href=\\"https://github.com/aws/aws-sdk-js-v3/pull/2891\\" target=\\"_blank\\"><strong>~10%</strong> reduction</a> in unpacked publish size.</p>\n<pre><code class=\\"lang-\\">\$ pwd\\n/home/trivikr/workspace/aws-sdk-js-v3/clients/client-sts\\n\\n# Before the change\\n\$ du -sh --apparent-size dist-types | cut -f1\\n742K\\n\\n\$ npx cloc dist-types\\n...\\nLanguage files blank comment code\\n-------------------------------------------------------------------------------\\nTypeScript 59 0 4962 1813\\n...\\n\\n# After the change\\n\$ du -sh --apparent-size dist-types | cut -f1\\n396K\\n\\n\$ npx cloc dist-types\\n...\\nLanguage files blank comment code\\n-------------------------------------------------------------------------------\\nTypeScript 59 185 2481 1813\\n...\\n</code></pre>\\n<p>As a consequence, customers using versions of TypeScript older than 4.0 will not see JSDoc comments in their IDE although the downleveled types will work. We went ahead with this change to encourage customers to switch to TypeScript 4.0+ which was released in <a href=\\"https://devblogs.microsoft.com/typescript/announcing-typescript-4-0/\\" target=\\"_blank\\">August 2020</a>.</p>\\n<p>If you are using TypeScript &lt;4.0, the JSDoc comments will appear for SDK versions &lt; 3.36.0.</p>\n<h4><a id=\\"We_removed_TypeScript_source_code_213\\"></a>We removed TypeScript source code</h4>\\n<p>The authors of JavaScript libraries decide the language to write a library in based on variety of reasons. For example, we use TypeScript for v3 for reasons explained in the blog post on <a href=\\"https://aws.amazon.com/blogs/developer/first-class-typescript-support-in-modular-aws-sdk-for-javascript/\\" target=\\"_blank\\">first-class TypeScript support</a>. Other maintainers may choose a different language for writing their library: JavaScript, ReScript, PureScript, ClosureScript, CoffeeScript, Reason, Elm, Flow, etc. The consumer of their library does not have to know the language a library is written in. At the end, the engine processes JavaScript code.</p>\\n<p>To provide first-class TypeScript support, a library needs to ship types. If a library is not written in TypeScript, they either manually write types, or use TypeScript to generate type declarations.</p>\n<p>We asked the question on Twitter if maintainers ship the source code in npm packages. Here is a quote from one of the replies: “Shipping source code is going against the spirit of module definition”.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/9ae81381007949a6981da76f92341048_image.png\\" alt=\\"image.png\\" /></p>\n<p>We removed the source code from our published packages along with other dev/test configurations, which led to <a href=\\"https://github.com/aws/aws-sdk-js-v3/pull/2873\\" target=\\"_blank\\"><strong>~28%</strong> reduction</a> in unpacked publish size.</p>\n<pre><code class=\\"lang-\\">\$ pwd\\n/home/trivikr/workspace/aws-sdk-js-v3/clients/client-sts\\n\\n# Before the change\\n\$ npm pack --dry-run\\n...\\npackage size: 141.7 kB \\nunpacked size: 1.2 MB \\ntotal files: 177\\n...\\n\\n# After the change\\n\$ npm pack --dry-run\\n...\\npackage size: 99.9 kB \\nunpacked size: 783.8 kB \\ntotal files: 88 \\n...\\n</code></pre>\\n<h4><a id=\\"We_removed_source_map_files_248\\"></a><strong>We removed source map files</strong></h4>\\n<p><a href=\\"https://developer.mozilla.org/en-US/docs/Tools/Debugger/How_to/Use_a_source_map\\" target=\\"_blank\\">Source map files</a> allow debuggers and other tools to display the original TypeScript source code when actually working with the emitted JavaScript files. In TypeScript, the source map files are emitted as <code>.js.map</code> (or <code>.jsx.map</code>) files next to the corresponding .js output file. TypeScript also allows embedding the source map content in the <code>.js</code> files. TypeScript also allows for inclusion of the original content of the <code>.ts</code> file as an embedded string in the source map.</p>\\n<p>The source map files are helpful with debugging the application code. They are not useful for libraries and dependencies who have to meet strict publish/install size restrictions. A better solution is to release debug versions of the SDK. If you have feedback about source maps, or would like to explain your debugging or other use cases, please comment on GitHub issue <a href=\\"https://github.com/aws/aws-sdk-js-v3/issues/2895\\" target=\\"_blank\\">aws/aws-sdk-js-v3/#2895</a>.</p>\\n<p>In the <a href=\\"https://www.typescriptlang.org/docs/handbook/declaration-files/publishing.html\\" target=\\"_blank\\">TypeScript publishing guide</a>, there is no recommendation on publishing source maps. In TSConfig, the settings for <a href=\\"https://www.typescriptlang.org/tsconfig#sourceMap\\" target=\\"_blank\\">sourceMap</a>, <a href=\\"https://www.typescriptlang.org/tsconfig#inlineSources\\" target=\\"_blank\\">inlineSourceMap</a> and <a href=\\"https://www.typescriptlang.org/tsconfig#inlineSources\\" target=\\"_blank\\">inlineSources</a> are turned off by default.</p>\\n<p>We removed sourceMaps from v3, which led to <a href=\\"https://github.com/aws/aws-sdk-js-v3/pull/2867\\" target=\\"_blank\\"><strong>~20%</strong> reduction</a> in unpacked publish size.</p>\n<pre><code class=\\"lang-\\">\$ pwd\\n/home/trivikr/workspace/aws-sdk-js-v3/clients/client-sts\\n\\n# Before the change\\n\$ du -sh --apparent-size dist-cjs | cut -f1\\n246K\\n\\n\$ npx cloc dist-cjs\\n...\\n 42 text files.\\n 42 unique files. \\n 21 files ignored.\\n...\\n\\n# After the change\\n\$ du -sh --apparent-size dist-cjs | cut -f1\\n174K\\n\\n\$ npx cloc dist-cjs\\n...\\n 21 text files.\\n 21 unique files. \\n 0 files ignored.\\n...\\n\\n</code></pre>\\n<h3><a id=\\"What_was_communitys_reaction_287\\"></a><strong>What was community’s reaction?</strong></h3>\\n<p>We had an overwhelming response from the Twitter community to these exciting news!</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/73feea42ed974ed0b7b4de6f37d5d21e_image.png\\" alt=\\"image.png\\" /></p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/b82f1562fbcd49d7ab76d08099aeb2e4_image.png\\" alt=\\"image.png\\" /></p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/c654723999db44829643591d09e293f6_image.png\\" alt=\\"image.png\\" /></p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/d21d26df36004ef69293d5bd062f8f9e_image.png\\" alt=\\"image.png\\" /></p>\n<p>Join <a href=\\"https://twitter.com/intent/tweet?text=%23aws%20%23javascript%20%23nodejs\\" target=\\"_blank\\">the conversation on Twitter </a>and let us know how you have reduced publish/install/bundle sizes in your npm packages or any other experiences you’ve had with AWS SDK for JavaScript.</p>\\n<h3><a id=\\"What_do_we_plan_to_do_in_the_future_301\\"></a><strong>What do we plan to do in the future?</strong></h3>\\n<p>To ensure that we keep bloat from sneaking back in, <a href=\\"https://github.com/aws/aws-sdk-js-v3/issues/2816\\" target=\\"_blank\\">we plan to add CI</a> to check publish size.</p>\\n<p>We’ve not reduced the size of the actual source code yet. For example, a <a href=\\"https://github.com/trivikr/temp-client-s3/pull/40\\" target=\\"_blank\\">generic function for API calls</a> will reduce the size of the source code by ~0.5%. If you have any ideas to reduce publish size, do post them in our experimental repo at <a href=\\"https://github.com/trivikr/temp-client-s3/issues\\" target=\\"_blank\\">trivikr/temp-client-s3/issues</a>.</p>\\n<p>We have also not considered using advanced or alternative compilation options like Google Closure Compiler, Babel or SWC. If you have ideas/suggestions or examples of how they can help, please comment on GitHub issue <a href=\\"https://github.com/aws/aws-sdk-js-v3/issues/2897\\" target=\\"_blank\\">aws/aws-sdk-js-v3/#2897</a>.</p>\\n<p>We are also considering shipping Node.js specific distributions and a type definitions distribution in separate prerelease version numbers, which could further reduce npm install size from between <code>60%</code> and <code>75%</code>. Although the expected improvements are huge, it would take lot of discussion and testing prior to implementation. Please post your ideas/suggestions/concerns on GitHub Issue <a href=\\"https://github.com/aws/aws-sdk-js-v3/issues/2889\\" target=\\"_blank\\">aws/aws-sdk-js-v3/#2889</a>.</p>\\n<p>If you have a feedback about TypeScript source code and source maps, or would like to explain your debugging or other use cases, please comment on GitHub issue <a href=\\"https://github.com/aws/aws-sdk-js-v3/issues/2895\\" target=\\"_blank\\">aws/aws-sdk-js-v3/#2895</a>.</p>\\n<h3><a id=\\"How_can_you_contribute_313\\"></a><strong>How can you contribute?</strong></h3>\\n<p>We value your feedback, so please tell us what you like and don’t like by <a href=\\"https://github.com/aws/aws-sdk-js-v3/issues/new/choose\\" target=\\"_blank\\">opening an issue on GitHub</a>. Reducing install/publish size is a <a href=\\"https://www.inc.com/jeff-haden/amazon-founder-jeff-bezos-this-is-how-successful-people-make-such-smart-decisions.html\\" target=\\"_blank\\">two-way doors decision</a>. Do inform us on GitHub if it breaks your developer experience.</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/2a48b5da7aa345c7b0265821b4859a5b_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"Trivikram_Kamathttpstwittercomtrivikram_319\\"></a><a href=\\"https://twitter.com/trivikram\\" target=\\"_blank\\"><strong>Trivikram Kamat</strong></a></h3>\n<p>Trivikram is maintainer of AWS SDK for JavaScript in Node.js and browser. Trivikram is also a Node.js Core collaborator and have contributed to HTTP, HTTP/2 and HTTP/3 over QUIC implementations in the past. He has been writing JavaScript for over a decade. You can find him on Twitter <a href=\\"https://twitter.com/trivikram\\" target=\\"_blank\\">@trivikram</a> and GitHub <a href=\\"https://github.com/trivikr\\" target=\\"_blank\\">@trivikr</a>.</p>\n"}