From smart televisions to desktops and from USB streaming devices to mobile phones, all Prime Video users enjoy the same interface and the same user experience (UX). This is the result of development by many Prime Video teams working on multiple code bases that interact seamlessly but are subject to continuous change in a continuous integration and continuous delivery (CI/CD) pipeline. In some cases, the range of programming languages and code can make it difficult for a single developer to see the potential consequences of proposed changes.
Typically, every code change is subject to code reviews by other developers and is tested to guarantee adequate levels of coverage before being shipped to production. Despite these safety mechanisms, it’s still possible for developers to make changes that might result in issues and for these changes to go unnoticed by reviews and testing. This is especially the case when changes impact potentially unrelated areas of the Prime Video application.
Since March 2021, Prime Video developers can incorporate a bot called “BugBear” into their code-review workflow. BugBear uses automated reasoning techniques to analyze code changes and provides feedback to developers in parallel with human reviewers. The following image shows how it adds in-line comments within 15 minutes of a code submission.
BugBear can analyze code in C, C++, Java and, more recently, TypeScript/JavaScript. It can detect generic issues such as unhandled null-pointer exceptions, memory and resource leaks, and concurrency issues such as data races and deadlocks. It can also detect mistakes in the implementation of the application’s business logic.
For instance, developers should always ensure that “before enabling a touchscreen keyboard, there should be a check that the device supports a touchscreen as an input mechanism.” A violation of this rule wouldn’t result in a crash but it would make the application unusable.
In a preliminary study that we conducted before making BugBear available to all Prime Video developers, BugBear detected around 100 issues and 80% of them resulted in an action by developers. And in ongoing work, we are improving the BugBear experience by building an analyzer that would enable users to not only get comments on potential issues but also prove assertions on-demand in code review.
For more information about BugBear and our approach, see the How automated reasoning improves the Prime Video experience article on the Amazon Science website.