How automated reasoning improves the Prime Video experience

{"value":"Automated reasoning is the ability of machines to make logical deductions. One common application of automated reasoning is in software verification, or establishing that computer programs will do what they’re supposed to. While this has been an area of active research for five decades, it is only very recently that verification techniques have become applicable to industrial code bases with millions of lines of code.\n\nSince 2019, we in the Prime Video Automated Reasoning group have been creating software development tools that use these verification techniques to give Amazon developers greater confidence in the code they write for the Prime Video App, the app that controls video playback for all Prime Video content.\n\nThe Prime Video App provides a uniform end-user experience regardless of the type of content, from movies on demand to live streaming of major sports events. Because the application has multiple components developed by dozens of independent teams worldwide in a range of programming languages, and it has to run on thousands of different hardware configurations, it provides a particularly tough environment for automated reasoning.\n\nSince March 26, 2021, all Prime Video developers have been using an automated-reasoning bot called BugBear, which conducts automatic code reviews, providing clear and actionable comments when it detects potential issues — or confirming that no issues were found. BugBear has analyzers for C/C++, Java, and TypeScript, the main languages used in Prime Video.\n\n\n\nAn example of BugBear in action (the names of the program functions and developers have been changed).\n\nIn a pilot campaign that started in the second half of 2020, BugBear conducted more than 1,000 automated code reviews and identified potential issues on about 100 of them. Prime Video developers agreed that approximately 80% of those issues required code modification.\n\nBugBear is implemented entirely using Amazon Web Services, and it provides feedback on developers’ code within 15 minutes. It can identify both generic issues and violations of Prime-Video-specific business-logic properties, something that we call code contracts.\n\nA generic issue is a problem in the code that is always considered wrong, irrespective of what the application is supposed to do — for instance, not closing a file after opening it, or trying to use the value of a variable that has not been previously initialized. \n\nA code contract is usually a restriction on possible behaviors of the code or a specific business rule that needs to be implemented; think of rules such as “before enabling a touch-screen keyboard, check that the device supports a touch screen as an input mechanism”.\n\nBugbear’s analyzers are static-analysis tools, meaning that they do not need to execute the code to find issues. In the case of C/C++ and Java source code, we employ an existing tool called [Infer](https://fbinfer.com/), adapted for Prime Video, to detect generic issues such as memory and concurrency problems — problems that arise when multiple processes operate in parallel on shared variables. \n\n#### **Enforcing contracts**\n\nFor C++ and TypeScript contracts, we have developed our own tool that encodes the program under analysis as a database of logical facts. Using a custom-built set of rules, it can deduce properties of the program automatically. We employ the declarative programming language Datalog to represent both the facts and the rules.\n\nSuppose, for instance, that a contract requires that, in function F, the function open_resource should always be called before the function use_resource(). In Datalog, the corresponding rule might look like this:\n\n\n\nThe key issue here is the construction of the relation called_before, which imposes constraints on the shape of the so-called call graph, a graphical representation of possible sequences of calls through the program’s instruction set. \n\nThe construction of the call graph is an undecidable problem, meaning that it’s impossible to construct a fully accurate, finite graph that correctly encodes all function calls in the program. Consequently, we can evaluate our rule only over approximations of the call graph. The precision of these approximations and their performance have been among the main topics of our research to deliver BugBear.\n\nIn ongoing work, we are building a system that would enable users to prove assertions on-demand in code review. For instance, users will be able to write “BugBear assertions” in their code, and these will be analyzed automatically by our tools. These assertions will also be used to scale the analysis, as we will be able to focus only on the code relevant to proving the assertion.\n\nABOUT THE AUTHOR\n\n#### **[Franco Raimondi](https://www.amazon.science/author/franco-raimondi)**\n\nFranco Raimondi is an Amazon Scholar and a professor of computer science at Middlesex University, London.\n\n#### **[Bor-Yuh Evan Chang](https://www.amazon.science/author/bor-yuh-evan-chang)**\n\nBor-Yuh Evan Chang is an Amazon Scholar and an associate professor of computer science at the University of Colorado Boulder.\n\n","render":"<p>Automated reasoning is the ability of machines to make logical deductions. One common application of automated reasoning is in software verification, or establishing that computer programs will do what they’re supposed to. While this has been an area of active research for five decades, it is only very recently that verification techniques have become applicable to industrial code bases with millions of lines of code.</p>\n<p>Since 2019, we in the Prime Video Automated Reasoning group have been creating software development tools that use these verification techniques to give Amazon developers greater confidence in the code they write for the Prime Video App, the app that controls video playback for all Prime Video content.</p>\n<p>The Prime Video App provides a uniform end-user experience regardless of the type of content, from movies on demand to live streaming of major sports events. Because the application has multiple components developed by dozens of independent teams worldwide in a range of programming languages, and it has to run on thousands of different hardware configurations, it provides a particularly tough environment for automated reasoning.</p>\n<p>Since March 26, 2021, all Prime Video developers have been using an automated-reasoning bot called BugBear, which conducts automatic code reviews, providing clear and actionable comments when it detects potential issues — or confirming that no issues were found. BugBear has analyzers for C/C++, Java, and TypeScript, the main languages used in Prime Video.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/c1e61270ef9844a39ad84bc3b49d1a3f_image.png\\" alt=\\"image.png\\" /></p>\n<p>An example of BugBear in action (the names of the program functions and developers have been changed).</p>\n<p>In a pilot campaign that started in the second half of 2020, BugBear conducted more than 1,000 automated code reviews and identified potential issues on about 100 of them. Prime Video developers agreed that approximately 80% of those issues required code modification.</p>\n<p>BugBear is implemented entirely using Amazon Web Services, and it provides feedback on developers’ code within 15 minutes. It can identify both generic issues and violations of Prime-Video-specific business-logic properties, something that we call code contracts.</p>\n<p>A generic issue is a problem in the code that is always considered wrong, irrespective of what the application is supposed to do — for instance, not closing a file after opening it, or trying to use the value of a variable that has not been previously initialized.</p>\n<p>A code contract is usually a restriction on possible behaviors of the code or a specific business rule that needs to be implemented; think of rules such as “before enabling a touch-screen keyboard, check that the device supports a touch screen as an input mechanism”.</p>\n<p>Bugbear’s analyzers are static-analysis tools, meaning that they do not need to execute the code to find issues. In the case of C/C++ and Java source code, we employ an existing tool called <a href=\\"https://fbinfer.com/\\" target=\\"_blank\\">Infer</a>, adapted for Prime Video, to detect generic issues such as memory and concurrency problems — problems that arise when multiple processes operate in parallel on shared variables.</p>\\n<h4><a id=\\"Enforcing_contracts_22\\"></a><strong>Enforcing contracts</strong></h4>\\n<p>For C++ and TypeScript contracts, we have developed our own tool that encodes the program under analysis as a database of logical facts. Using a custom-built set of rules, it can deduce properties of the program automatically. We employ the declarative programming language Datalog to represent both the facts and the rules.</p>\n<p>Suppose, for instance, that a contract requires that, in function F, the function open_resource should always be called before the function use_resource(). In Datalog, the corresponding rule might look like this:</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/94537b21c02d4078a0d5c80a58f58d9f_image.png\\" alt=\\"image.png\\" /></p>\n<p>The key issue here is the construction of the relation called_before, which imposes constraints on the shape of the so-called call graph, a graphical representation of possible sequences of calls through the program’s instruction set.</p>\n<p>The construction of the call graph is an undecidable problem, meaning that it’s impossible to construct a fully accurate, finite graph that correctly encodes all function calls in the program. Consequently, we can evaluate our rule only over approximations of the call graph. The precision of these approximations and their performance have been among the main topics of our research to deliver BugBear.</p>\n<p>In ongoing work, we are building a system that would enable users to prove assertions on-demand in code review. For instance, users will be able to write “BugBear assertions” in their code, and these will be analyzed automatically by our tools. These assertions will also be used to scale the analysis, as we will be able to focus only on the code relevant to proving the assertion.</p>\n<p>ABOUT THE AUTHOR</p>\n<h4><a id=\\"Franco_Raimondihttpswwwamazonscienceauthorfrancoraimondi_38\\"></a><strong><a href=\\"https://www.amazon.science/author/franco-raimondi\\" target=\\"_blank\\">Franco Raimondi</a></strong></h4>\n<p>Franco Raimondi is an Amazon Scholar and a professor of computer science at Middlesex University, London.</p>\n<h4><a id=\\"BorYuh_Evan_Changhttpswwwamazonscienceauthorboryuhevanchang_42\\"></a><strong><a href=\\"https://www.amazon.science/author/bor-yuh-evan-chang\\" target=\\"_blank\\">Bor-Yuh Evan Chang</a></strong></h4>\n<p>Bor-Yuh Evan Chang is an Amazon Scholar and an associate professor of computer science at the University of Colorado Boulder.</p>\n"}