Google Online Security Blog: android security

Pixel 6: Setting a new standard for mobile security

October 27, 2021

Posted by Dave Kleidermacher, Jesse Seed, Brandon Barbello, and Stephan Somogyi, Android, Pixel & Tensor security teams With Pixel 6 and Pixel 6 Pro, we’re launching our most secure Pixel phone yet, with 5 years of security updates and the most layers of hardware security. These new Pixel smartphones take a layered security approach, with innovations spanning across the Google Tensor system on a chip (SoC) hardware to new Pixel-first features in the Android operating system, making it the first Pixel phone with Google security from the silicon all the way to the data center. Multiple dedicated security teams have also worked to ensure that Pixel’s security is provable through transparency and external validation. Secure to the Core Google has put user data protection and transparency at the forefront of hardware security with Google Tensor. Google Tensor’s main processors are Arm-based and utilize TrustZone™ technology. TrustZone is a key part of our security architecture for general secure processing, but the security improvements included in Google Tensor go beyond TrustZone.

Figure 1. Pixel Secure EnvironmentsThe Google Tensor security core is a custom designed security subsystem dedicated to the preservation of user privacy. It's distinct from the application processor, not only logically, but physically, and consists of a dedicated CPU, ROM, one-time-programmable (OTP) memory, crypto engine, internal SRAM, and protected DRAM. For Pixel 6 and 6 Pro, the security core’s primary use cases include protecting user data keys at runtime, hardening secure boot, and interfacing with Titan M2^TM. Your secure hardware is only as good as your secure OS, and we are using Trusty, our open source trusted execution environment. Trusty OS is the secure OS used both in TrustZone and the Google Tensor security core. With Pixel 6 and Pixel 6 Pro your security is enhanced by the new Titan M2^TM, our discrete security chip, fully designed and developed by Google. In this next generation chip, we moved to an in-house designed RISC-V processor, with extra speed and memory, and made it even more resilient to advanced attacks. Titan M2^TM has been tested against the most rigorous standard for vulnerability assessment, AVA_VAN.5, by an independent, accredited evaluation lab. Titan M2™ supports Android Strongbox, which securely generates and stores keys used to protect your PINs and password, and works hand-in-hand with Google Tensor security core to protect user data keys while in use in the SoC. Moving a step higher in the system, Pixel 6 and Pixel 6 Pro ship with Android 12 and a slew of Pixel-first and Pixel-exclusive features. Enhanced Controls We aim to give users better ways to control their data and manage their devices with every release of Android. Starting with Android 12 on Pixel, you can use the new Security hub to manage all your security settings in one place. It helps protect your phone, apps, Google Account, and passwords by giving you a central view of your device’s current configuration. Security hub also provides recommendations to improve your security, helping you decide what settings best meet your needs.

For privacy, we are launching Privacy Dashboard, which will give you a simple and clear timeline view of the apps that have accessed your location, microphone and camera in the last 24 hours. If you notice apps that are accessing more data than you expected, the dashboard provides a path to controls to change those permissions on the fly.

To provide additional transparency, new indicators in Pixel’s status bar will show you when your camera and mic are being accessed by apps. If you want to disable that access, new privacy toggles give you the ability to turn off camera or microphone access across apps on your phone with a single tap, at any time.

The Pixel 6 and Pixel 6 Pro also include a toggle that lets you remove your device’s ability to connect to less-secure 2G networks. While necessary in certain situations, accessing 2G networks can open up additional attack vectors; this toggle helps users mitigate those risks when 2G connectivity isn’t needed. Built-in security By making all of our products secure by default, Google keeps more people safe online than anyone else in the world. With the Pixel 6 and Pixel 6 Pro, we’re also ratcheting up the dial on default, built-in protections. Our new optical under-display fingerprint sensor ensures that your biometric information is secure and never leaves your device. As part of our ongoing security development lifecycle, Pixel 6 and 6 Pro’s fingerprint unlock has been externally validated by security experts as a strong and secure biometric unlock mechanism meeting the Class 3 strength requirements defined in the Android 12 Compatibility Definition Document (CDD). Phishing continues to be a huge attack vector, affecting everyone across different devices. The Pixel 6 and Pixel 6 Pro introduce new anti-phishing protections. Built-in protections automatically scan for potential threats from phone calls, text messages, emails, and links sent through apps, notifying you if there’s a potential problem. Users are also now better protected against bad apps by enhancements to our on-device detection capabilities within Google Play Protect. Since its launch in 2017, Google Play Protect has provided the ability to detect malicious applications even when the device is offline. The Pixel 6 and Pixel 6 Pro uses new machine learning models that improve the detection of malware in Google Play Protect. The detection runs on your Pixel, and uses a privacy preserving technology called federated analytics to discover commonly-run bad apps. This will help to further protect over 3 billion users by improving Google Play Protect, which already analyzes over 100 billion apps every day to detect threats. Many of Pixel’s privacy-preserving features run inside Private Compute Core, an open source sandbox isolated from the rest of the operating system and apps. Our open source Private Compute Services manages network communication for these features, and uses federated learning, federated analytics, and private information retrieval to improve features while preserving privacy. Some features already running on Private Compute Core include Live Caption, Now Playing, and Smart Reply suggestions. Google Binary Transparency (GBT) is the newest addition to our open and verifiable security infrastructure, providing a new layer of software integrity for your device. Building on the principles pioneered by Certificate Transparency, GBT helps ensure your Pixel is only running verified OS software. It works by using append-only logs to store signed hashes of the system images. The logs are public and can be used to verify that what’s published is the same as what’s on the device – giving users and researchers the ability to independently verify OS integrity for the first time. Beyond the Phone Defense-in-depth isn’t just a matter of hardware and software layers. Security is a rigorous process. Pixel 6 and Pixel 6 Pro benefit from in-depth design and architecture reviews, memory-safe rewrites to security critical code, static analysis, formal verification of source code, fuzzing of critical components, and red-teaming, including with external security labs to pen-test our devices. Pixel is also part of the Android Vulnerability Rewards Program, which paid out $1.75 million last year, creating a valuable feedback loop between us and the security research community and, most importantly, helping us keep our users safe. Capping off this combined hardware and software security system, is the Titan Backup Architecture, which gives your Pixel a secure foot in the cloud. Launched in 2018, the combination of Android’s Backup Service and Google Cloud’s Titan Technology means that backed-up application data can only be decrypted by a randomly generated key that isn't known to anyone besides the client, including Google. This end-to-end service was independently audited by a third party security lab to ensure no one can access a user's backed-up application data without specifically knowing their passcode. To top it all off, this end-to-end security from the hardware across the software to the data center comes with no fewer than 5 years of guaranteed Android security updates on Pixel 6 and Pixel 6 Pro devices from the date they launch in the US. This is an important commitment for the industry, and we hope that other smartphone manufacturers broaden this trend. Together, our secure chipset, software and processes make Pixel 6 and Pixel 6 Pro the most secure Pixel phone yet.

Introducing Android’s Private Compute Services

September 9, 2021

Posted by Suzanne Frey, VP, Product, Android & Play Security and Privacy

We introduced Android’s Private Compute Core in Android 12 Beta. Today, we're excited to announce a new suite of services that provide a privacy-preserving bridge between Private Compute Core and the cloud. Recap: What is Private Compute Core? Android’s Private Compute Core is an open source, secure environment that is isolated from the rest of the operating system and apps. With each new Android release we’ll add more privacy-preserving features to the Private Compute Core. Today, these include:

Live Caption, which adds captions to any media using Google’s on-device speech recognition

Now Playing, which recognizes music playing nearby and displays the song title and artist name on your device’s lock screen

Smart Reply, which suggests relevant responses based on the conversation you’re having in messaging apps

For these features to be private, they must:

Keep the information on your device private. Android ensures that the sensitive data processed in the Private Compute Core is not shared to any apps without you taking an action. For instance, until you tap a Smart Reply, the OS keeps your reply hidden from both your keyboard and the app you’re typing into.

Let your device use the cloud (to download new song catalogs or speech-recognition models) without compromising your privacy. This is where Private Compute Services comes in.

Introducing Android’s Private Compute Services Machine learning features often improve by updating models, and Private Compute Services helps features get these updates over a private path. Android prevents any feature inside the Private Compute Core from having direct access to the network. Instead, features communicate over a small set of purposeful open-source APIs to Private Compute Services, which strips out identifying information and uses a set of privacy technologies, including Federated Learning, Federated Analytics, and Private information retrieval. We will publicly publish the source code for Private Compute Services, so it can be audited by security researchers and other teams outside of Google. This means it can go through the same rigorous security programs that ensure the safety of the Android platform. We’re enthusiastic about the potential for machine learning to power more helpful features inside Android, and Android’s Private Compute Core will help users benefit from these features while strengthening privacy protections via the new Private Compute Services. Android is the first open source mobile OS to include this kind of externally verifiable privacy; Private Compute Services helps the Android OS continue to innovate in machine learning, while also maintaining the highest standards of privacy and security.

Rust/C++ interop in the Android Platform

June 8, 2021

Posted by Joel Galenson and Matthew Maurer, Android TeamOne of the main challenges of evaluating Rust for use within the Android platform was ensuring we could provide sufficient interoperability with our existing codebase. If Rust is to meet its goals of improving security, stability, and quality Android-wide, we need to be able to use Rust anywhere in the codebase that native code is required. To accomplish this, we need to provide the majority of functionality platform developers use. As we discussed previously, we have too much C++ to consider ignoring it, rewriting all of it is infeasible, and rewriting older code would likely be counterproductive as the bugs in that code have largely been fixed. This means interoperability is the most practical way forward. Before introducing Rust into the Android Open Source Project (AOSP), we needed to demonstrate that Rust interoperability with C and C++ is sufficient for practical, convenient, and safe use within Android. Adding a new language has costs; we needed to demonstrate that Rust would be able to scale across the codebase and meet its potential in order to justify those costs. This post will cover the analysis we did more than a year ago while we evaluated Rust for use in Android. We also present a follow-up analysis with some insights into how the original analysis has held up as Android projects have adopted Rust. Language interoperability in Android Existing language interoperability in Android focuses on well defined foreign-function interface (FFI) boundaries, which is where code written in one programming language calls into code written in a different language. Rust support will likewise focus on the FFI boundary as this is consistent with how AOSP projects are developed, how code is shared, and how dependencies are managed. For Rust interoperability with C, the C application binary interface (ABI) is already sufficient. Interoperability with C++ is more challenging and is the focus of this post. While both Rust and C++ support using the C ABI, it is not sufficient for idiomatic usage of either language. Simply enumerating the features of each language results in an unsurprising conclusion: many concepts are not easily translatable, nor do we necessarily want them to be. After all, we’re introducing Rust because many features and characteristics of C++ make it difficult to write safe and correct code. Therefore, our goal is not to consider all language features, but rather to analyze how Android uses C++ and ensure that interop is convenient for the vast majority of our use cases. We analyzed code and interfaces in the Android platform specifically, not codebases in general. While this means our specific conclusions may not be accurate for other codebases, we hope the methodology can help others to make a more informed decision about introducing Rust into their large codebase. Our colleagues on the Chrome browser team have done a similar analysis, which you can find here. This analysis was not originally intended to be published outside of Google: our goal was to make a data-driven decision on whether or not Rust was a good choice for systems development in Android. While the analysis is intended to be accurate and actionable, it was never intended to be comprehensive, and we’ve pointed out a couple of areas where it could be more complete. However, we also note that initial investigations into these areas showed that they would not significantly impact the results, which is why we decided to not invest the additional effort. Methodology Exported functions from Rust and C++ libraries are where we consider interop to be essential. Our goals are simple:

Rust must be able to call functions from C++ libraries and vice versa.

FFI should require a minimum of boilerplate.

FFI should not require deep expertise.

While making Rust functions callable from C++ is a goal, this analysis focuses on making C++ functions available to Rust so that new Rust code can be added while taking advantage of existing implementations in C++. To that end, we look at exported C++ functions and consider existing and planned compatibility with Rust via the C ABI and compatibility libraries. Types are extracted by running objdump on shared libraries to find external C++ functions they use¹ and running c++filt to parse the C++ types. This gives functions and their arguments. It does not consider return values, but a preliminary analysis² of those revealed that they would not significantly affect the results. We then classify each of these types into one of the following buckets: Supported by bindgen These are generally simple types involving primitives (including pointers and references to them). For these types, Rust’s existing FFI will handle them correctly, and Android’s build system will auto-generate the bindings. Supported by cxx compat crate These are handled by the cxx crate. This currently includes std::string, std::vector, and C++ methods (including pointers/references to these types). Users simply have to define the types and functions they want to share across languages and cxx will generate the code to do that safely. Native support These types are not directly supported, but the interfaces that use them have been manually reworked to add Rust support. Specifically, this includes types used by AIDL and protobufs. We have also implemented a native interface for StatsD as the existing C++ interface relies on method overloading, which is not well supported by bindgen and cxx³. Usage of this system does not show up in the analysis because the C++ API does not use any unique types. Potential addition to cxx This is currently common data structures such as std::optional and std::chrono::duration and custom string and vector implementations. These can either be supported natively by a future contribution to cxx, or by using its ExternType facilities. We have only included types in this category that we believe are relatively straightforward to implement and have a reasonable chance of being accepted into the cxx project. We don't need/intend to support Some types are exposed in today’s C++ APIs that are either an implicit part of the API, not an API we expect to want to use from Rust, or are language specific. Examples of types we do not intend to support include:

Mutexes - we expect that locking will take place in one language or the other, rather than needing to pass mutexes between languages, as per our coarse-grained philosophy.

native_handle - this is a JNI interface type, so it is inappropriate for use in Rust/C++ communication.

std::locale& - Android uses a separate locale system from C++ locales. This type primarily appears in output due to e.g., cout usage, which would be inappropriate to use in Rust.

Overall, this category represents types that we do not believe a Rust developer should be using. HIDL Android is in the process of deprecating HIDL and migrating to AIDL for HALs for new services.We’re also migrating some existing implementations to stable AIDL. Our current plan is to not support HIDL, preferring to migrate to stable AIDL instead. These types thus currently fall into the “We don't need/intend to support'' bucket above, but we break them out to be more specific. If there is sufficient demand for HIDL support, we may revisit this decision later. Other This contains all types that do not fit into any of the above buckets. It is currently mostly std::string being passed by value, which is not supported by cxx. Top C++ libraries One of the primary reasons for supporting interop is to allow reuse of existing code. With this in mind, we determined the most commonly used C++ libraries in Android: liblog, libbase, libutils, libcutils, libhidlbase, libbinder, libhardware, libz, libcrypto, and libui. We then analyzed all of the external C++ functions used by these libraries and their arguments to determine how well they would interoperate with Rust. Chart

Overall, 81% of types are in the first three categories (which we currently fully support) and 87% are in the first four categories (which includes those we believe we can easily support). Almost all of the remaining types are those we believe we do not need to support. Mainline modules In addition to analyzing popular C++ libraries, we also examined Mainline modules. Supporting this context is critical as Android is migrating some of its core functionality to Mainline, including much of the native code we hope to augment with Rust. Additionally, their modularity presents an opportunity for interop support. We analyzed 64 binaries and libraries in 21 modules. For each analyzed library we examined their used C++ functions and analyzed the types of their arguments to determine how well they would interoperate with Rust in the same way we did above for the top 10 libraries. Chart

Here 88% of types are in the first three categories and 90% in the first four, with almost all of the remaining being types we do not need to handle. Analysis of Rust/C++ Interop in AOSP With almost a year of Rust development in AOSP behind us, and more than a hundred thousand lines of code written in Rust, we can now examine how our original analysis has held up based on how C/C++ code is currently called from Rust in AOSP.⁴ Chart

The results largely match what we expected from our analysis with bindgen handling the majority of interop needs. Extensive use of AIDL by the new Keystore2 service results in the primary difference between our original analysis and actual Rust usage in the “Native Support” category.A few current examples of interop are:

Cxx in Bluetooth - While Rust is intended to be the primary language for Bluetooth, migrating from the existing C/C++ implementation will happen in stages. Using cxx allows the Bluetooth team to more easily serve legacy protocols like HIDL until they are phased out by using the existing C++ support to incrementally migrate their service.

AIDL in keystore - Keystore implements AIDL services and interacts with apps and other services over AIDL. Providing this functionality would be difficult to support with tools like cxx or bindgen, but the native AIDL support is simple and ergonomic to use.

Manually-written wrappers in profcollectd - While our goal is to provide seamless interop for most use cases, we also want to demonstrate that, even when auto-generated interop solutions are not an option, manually creating them can be simple and straightforward. Profcollectd is a small daemon that only exists on non-production engineering builds. Instead of using cxx it uses some small manually-written C wrappers around C++ libraries that it then passes to bindgen.

Conclusion Bindgen and cxx provide the vast majority of Rust/C++ interoperability needed by Android. For some of the exceptions, such as AIDL, the native version provides convenient interop between Rust and other languages. Manually written wrappers can be used to handle the few remaining types and functions not supported by other options as well as to create ergonomic Rust APIs. Overall, we believe interoperability between Rust and C++ is already largely sufficient for convenient use of Rust within Android. If you are considering how Rust could integrate into your C++ project, we recommend doing a similar analysis of your codebase. When addressing interop gaps, we recommend that you consider upstreaming support to existing compat libraries like cxx. Acknowledgements Our first attempt at quantifying Rust/C++ interop involved analyzing the potential mismatches between the languages. This led to a lot of interesting information, but was difficult to draw actionable conclusions from. Rather than enumerating all the potential places where interop could occur, Stephen Hines suggested that we instead consider how code is currently shared between C/C++ projects as a reasonable proxy for where we’ll also likely want interop for Rust. This provided us with actionable information that was straightforward to prioritize and implement. Looking back, the data from our real-world Rust usage has reinforced that the initial methodology was sound. Thanks Stephen!

Also, thanks to:

Andrei Homescu and Stephen Crane for contributing AIDL support to AOSP.

Ivan Lozano for contributing protobuf support to AOSP.

David Tolnay for publishing cxx and accepting our contributions.

The many authors and contributors to bindgen.

Jeff Vander Stoep and Adrian Taylor for contributions to this post.

We used undefined symbols of function type as reported by objdump to perform this analysis. This means that any header-only functions will be absent from our analysis, and internal (non-API) functions which are called by header-only functions may appear in it. ↩
We extracted return values by parsing DWARF symbols, which give the return types of functions. ↩
Even without automated binding generation, manually implementing the bindings is straightforward. ↩
In the case of handwritten C/C++ wrappers, we analyzed the functions they call, not the wrappers themselves. For all uses of our native AIDL library, we analyzed the types used in the C++ version of the library. ↩

Introducing Security By Design

May 26, 2021

Posted by Jon Markoff and Sean Smith, Android Security and Privacy Team

Integrating security into your app development lifecycle can save a lot of time, money, and risk. That’s why we’ve launched Security by Design on Google Play Academy to help developers identify, mitigate, and proactively protect against security threats.

The Android ecosystem, including Google Play, has many built-in security features that help protect developers and users. The course Introduction to app security best practices takes these protections one step further by helping you take advantage of additional security features to build into your app. For example, Jetpack Security helps developers properly encrypt their data at rest and provides only safe and well known algorithms for encrypting Files and SharedPreferences. The SafetyNet Attestation API is a solution to help identify potentially dangerous patterns in usage. There are several common design vulnerabilities that are important to look out for, including using shared or improper file storage, using insecure protocols, unprotected components such as Activities, and more. The course also provides methods to test your app in order to help you keep it safe after launch. Finally, you can set up a Vulnerability Disclosure Program (VDP) to engage security researchers to help.

In the next course, you can learn how to integrate security at every stage of the development process by adopting the Security Development Lifecycle (SDL). The SDL is an industry standard process and in this course you’ll learn the fundamentals of setting up a program, getting executive sponsorship and integration into your development lifecycle.

Threat modeling is part of the Security Development Lifecycle, and in this course you will learn to think like an attacker to identify, categorize, and address threats. By doing so early in the design phase of development, you can identify potential threats and start planning for how to mitigate them at a much lower cost and create a more secure product for your users.

Improving your app’s security is a never ending process. Sign up for the Security by Design module where in a few short courses, you will learn how to integrate security into your app development lifecycle, model potential threats, and app security best practices into your app, as well as avoid potential design pitfalls.

Integrating Rust Into the Android Open Source Project

May 11, 2021

Posted by Ivan Lozano, Android Team

The Android team has been working on introducing the Rust programming language into the Android Open Source Project (AOSP) since 2019 as a memory-safe alternative for platform native code development. As with any large project, introducing a new language requires careful consideration. For Android, one important area was assessing how to best fit Rust into Android’s build system. Currently this means the Soong build system (where the Rust support resides), but these design decisions and considerations are equally applicable for Bazel when AOSP migrates to that build system. This post discusses some of the key design considerations and resulting decisions we made in integrating Rust support into Android’s build system. Rust integration into large projects A RustConf 2019 meeting on Rust usage within large organizations highlighted several challenges, such as the risk that eschewing Cargo in favor of using the Rust Compiler, rustc, directly (see next section) may remove organizations from the wider Rust community. We share this same concern. When changes to imported third-party crates might be beneficial to the wider community, our goal is to upstream those changes. Likewise when crates developed for Android could benefit the wider Rust community, we hope to release them as independent crates. We believe that the success of Rust within Android is dependent on minimizing any divergence between Android and the Rust community at large, and hope that the Rust community will benefit from Android’s involvement. No nested build systems Rust provides Cargo as the default build system and package manager, collecting dependencies and invoking rustc (the Rust compiler) to build the target crate (Rust package). Soong takes this role instead in Android and calls rustc directly for several reasons:

In Cargo, C dependencies are handled independently in an ad-hoc manner via build.rs scripts. Soong already provides a mechanism for building C libraries and defining them as dependencies, and Android carefully controls the compiler version and global compilation flags to ensure libraries are built a particular way. Relying on Cargo would introduce a second non-Soong mechanism for defining/building C libraries that would not be constrained by the carefully selected compilation controls implemented in Soong. This could also lead to multiple different versions of the same library, negatively impacting memory/disk usage.

Calling compilers directly through Soong provides the stability and control Android requires for the variety of build configurations it supports (for example, specifying where target-specific dependencies are and which compilation flags to use). While it would technically be possible to achieve the necessary level of control over rustc indirectly through Cargo, Soong would have no understanding of how the Cargo.toml (the Cargo build file) would influence the commands Cargo emits to rustc. Paired with the fact that Cargo evolves independently, this would severely restrict Soong’s ability to precisely control how build artifacts are created.

Builds which are self-contained and insensitive to the host configuration, known as hermetic builds, are necessary for Android to produce reproducible builds. Cargo, which relies on build.rs scripts, doesn’t yet provide hermeticity guarantees.

Incremental builds are important to maintain engineering productivity; building Android takes a considerable amount of resources. Cargo was not designed for integration into existing build systems and does not expose its compilation units. Each Cargo invocation builds the entire crate dependency graph for a given Cargo.toml, rebuilding crates multiple times across projects¹. This is too coarse for integration into Soong’s incremental build support, which expects smaller compilation units. This support is necessary to scale up Rust usage within Android.

Using the Rust compiler directly allows us to avoid these issues and is consistent with how we compile all other code in AOSP. It provides the most control over the build process and eases integration into Android’s existing build system. Unfortunately, avoiding it introduces several challenges and influences many other build system decisions because Cargo usage is so deeply ingrained in the Rust crate ecosystem.

No build.rs scripts

A build.rs script compiles to a Rust binary which Cargo builds and executes during a build to handle pre-build tasks, commonly setting up the build environment, or building libraries in other languages (for example C/C++). This is analogous to configure scripts used for other languages.

Avoiding build.rs scripts somewhat flows naturally from not relying on Cargo since supporting these would require replicating Cargo behavior and assumptions. Beyond this however, there are good reasons for AOSP to avoid build scripts as well:

build.rs scripts can execute arbitrary code on the build host. From a security perspective, this introduces an additional burden when adding or updating third-party code as the build.rs script needs careful scrutiny.
Third-party build.rs scripts may not be hermetic or reproducible in potentially subtle ways. It is also common for build.rs files to access files outside the build directory (such as /usr/lib). When they are not hermetic, we would need to either carry a local patch or work with upstream to resolve the issue.
The most common task for build.rs is to build C libraries which Rust code depends on. We already support this through Soong.
Android likewise avoids running build scripts while building for other languages, instead, simply using them to inform the structure of the Android.bp file.

For instances in third-party code where a build script is used only to compile C dependencies, we either use existing cc_library Soong definitions (such as boringssl for quiche) or create new definitions for crate-specific code. When the build.rs is used to generate source, we try to replicate the core functionality in a Soong rust_binary module for use as a custom source generator. In other cases where Soong can provide the information without source generation, we may carry a small patch that leverages this information. Why proc_macro but not build.rs? Why do we support proc_macros, which are compiler plug-ins that execute code on the host within the compiler context, but not build.rs scripts? While build.rs code is written as one-off code to handle building a single crate, proc_macros define reusable functionality within the compiler which can become widely relied upon across the Rust community. As a result popular proc_macros are generally better maintained and more scrutinized upstream, which makes the code review process more manageable. They are also more readily sandboxed as part of the build process since they are less likely to have dependencies external to the compiler. proc_macros are also a language feature rather than a method for building code. These are relied upon by source code, are unavoidable for third-party dependencies, and are useful enough to define and use within our platform code. While we can avoid build.rs by leveraging our build system, the same can’t be said of proc_macros. There is also precedence for compiler plugin support within the Android build system. For example see Soong’s java_plugin modules. Generated source as crates Unlike C/C++ compilers, rustc only accepts a single source file representing an entry point to a binary or library. It expects that the source tree is structured such that all required source files can be automatically discovered. This means that generated source either needs to be placed in the source tree or provided through an include directive in source: include!("/path/to/hello.rs"); The Rust community depends on build.rs scripts alongside assumptions about the Cargo build environment to get around this limitation. When building, the cargo command sets an OUT_DIR environment variable which build.rs scripts are expected to place generated source code in. This source can then be included via: include!(concat!(env!("OUT_DIR"), "/hello.rs")); This presents a challenge for Soong as outputs for each module are placed in their own out/ directory²; there is no single OUT_DIR where dependencies output their generated source. For platform code, we prefer to package generated source into a crate that can be imported. There are a few reasons to favor this approach:

Prevent generated source file names from colliding.

Reduce boilerplate code checked-in throughout the tree and which needs to be maintained. Any boilerplate necessary to make the generated source compile into a crate can be centrally maintained.

Avoid implicit³ interactions between generated code and the surrounding crate.

Reduce pressure on memory and disk by dynamically liking commonly used generated sources.

As a result, all of Android’s Rust source generation module types produce code that can be compiled and used as a crate.

We still support third-party crates without modification by copying all the generated source dependencies for a module into a single per-module directory similar to Cargo. Soong then sets the OUT_DIR environment variable to that directory when compiling the module so the generated source can be found. However we discourage use of this mechanism in platform code unless absolutely necessary for the reasons described above.

Dynamic linkage by default

By default, the Rust ecosystem assumes that crates will be statically linked into binaries. The usual benefits of dynamic libraries are upgrades (whether for security or functionality) and decreased memory usage. Rust’s lack of a stable binary interface and usage of cross-crate information flow prevents upgrading libraries without upgrading all dependent code. Even when the same crate is used by two different programs on the system, it is unlikely to be provided by the same shared object⁴ due to the precision with which Rust identifies its crates. This makes Rust binaries more portable but also results in larger disk and memory footprints.

This is problematic for Android devices where resources like memory and disk usage must be carefully managed because statically linking all crates into Rust binaries would result in excessive code duplication (especially in the standard library). However, our situation is also different from the standard host environment: we build Android using global decisions about dependencies. This means that nearly every crate is shareable between all users of that crate. Thus, we opt to link crates dynamically by default for device targets. This reduces the overall memory footprint of Rust in Android by allowing crates to be reused across multiple binaries which depend on them.

Since this is unusual in the Rust community, not all third-party crates support dynamic compilation. Sometimes we must carry small patches while we work with upstream maintainers to add support.

Current Status of Build Support

We support building all output types supported by rustc (rlibs, dylibs, proc_macros, cdylibs, staticlibs, and executables). Rust modules can automatically request the appropriate crate linkage for a given dependency (rlib vs dylib). C and C++ modules can depend on Rust cdylib or staticlib producing modules the same way as they would for a C or C++ library.

In addition to being able to build Rust code, Android’s build system also provides support for protobuf and gRPC and AIDL generated crates. First-class bindgen support makes interfacing with existing C code simple and we have support modules using cxx for tighter integration with C++ code.

The Rust community produces great tooling for developers, such as the language server rust-analyzer. We have integrated support for rust-analyzer into the build system so that any IDE which supports it can provide code completion and goto definitions for Android modules.

Source-based code coverage builds are supported to provide platform developers high level signals on how well their code is covered by tests. Benchmarks are supported as their own module type, leveraging the criterion crate to provide performance metrics. In order to maintain a consistent style and level of code quality, a default set of clippy lints and rustc lints are enabled by default. Additionally, HWASAN/ASAN fuzzers are supported, with the HWASAN rustc support added to upstream.

In the near future, we plan to add documentation to source.android.com on how to define and use Rust modules in Soong. We expect Android’s support for Rust to continue evolving alongside the Rust ecosystem and hope to continue to participate in discussions around how Rust can be integrated into existing build systems.

Thank you to Matthew Maurer, Jeff Vander Stoep, Joel Galenson, Manish Goregaokar, and Tyler Mandry for their contributions to this post.

Notes

This can be mitigated to some extent with workspaces, but requires a very specific directory arrangement that AOSP does not conform to. ↩
This presents no problem for C/C++ and similar languages as the path to the generated source is provided directly to the compiler. ↩
Since include! works by textual inclusion, it may reference values from the enclosing namespace, modify the namespace, or use constructs like #![foo]. These implicit interactions can be difficult to maintain. Macros should be preferred if interaction with the rest of the crate is truly required. ↩
While libstd would usually be shareable for the same compiler revision, most other libraries would end up with several copies for Cargo-built Rust binaries, since each build would attempt to use a minimum feature set and may select different dependency versions for the library in question. Since information propagates across crate boundaries, you cannot simply produce a “most general” instance of that library. ↩

How we fought bad apps and developers in 2020

April 21, 2021

Posted by Krish Vitaldevara, Director of Product Management Trust & Safety, Google Play

Providing safe experiences to billions of users and millions of Android developers has been one of the highest priorities for Google Play for many years. Last year we introduced new policies, improved our systems, and further optimized our processes to better protect our users, assist good developers and strengthen our guard against bad apps and developers. Additionally, in 2020, Google Play Protect scanned over 100B installed apps each day for malware across billions of devices. Users come to Google Play to find helpful, reliable apps on everything from COVID-19 vaccine information to new forms of entertainment, grocery delivery, communication and more. As such, we introduced a series of policies and new developer support to continue to elevate information quality on the platform and reduce the risk of user harm from misinformation.

COVID-19 apps requirements: To ensure public safety, information integrity and privacy, we introduced specific requirements for COVID-19 apps. Under these requirements, apps related to sensitive use cases, such as those providing testing information, must be endorsed by either official governmental entities or healthcare organizations and must meet a high standard for user data privacy.

News policy: To promote transparency in news publishing, we introduced minimum requirements that apps must meet in order for developers to declare their app as a “News” app on Google Play. These guidelines help promote user transparency and developer accountability by providing users with relevant information about the app.

Election support: We created teams and processes across Google Play focused on elections to provide additional support and adapt to the changing landscape. This includes support for government agencies, specially trained app reviewers, and a safety team to address election threats and abuse.

Our core efforts around identifying and mitigating bad apps and developers continued to evolve to address new adversarial behaviors and forms of abuse. Our machine-learning detection capabilities and enhanced app review processes prevented over 962k policy-violating app submissions from getting published to Google Play. We also banned 119k malicious and spammy developer accounts. Additionally, we significantly increased our focus on SDK enforcement, as we've found these violations have an outsized impact on security and user data privacy. Last year, we continued to reduce developer access to sensitive permissions. In February, we announced a new background location policy to ensure that apps requesting this permission need the data in order to provide clear user benefit. As a result of the new policy, developers now have to demonstrate that benefit and prominently tell users about it or face possible removal from Google Play. We've begun enforcement on apps not meeting new policy guidelines and will provide an update on the usage of this permission in a future blog post. We've also continued to invest in protecting kids and helping parents find great content. In 2020 we launched a new kids tab filled with “Teacher approved” apps. To evaluate apps, we teamed with academic experts and teachers across the country, including our lead advisors, Joe Blatt (Harvard Graduate School of Education) and Dr. Sandra Calvert (Georgetown University). As we continue to invest in protecting people from apps with harmful content, malicious behaviors, or threats to user privacy, we are also equally motivated to provide trusted experiences to Play developers. For example, we’ve improved our process for providing relevant information about enforcement actions we’ve taken, resulting in significant reduction in appeals and increased developer satisfaction. We will continue to enhance the speed and quality of our communications to developers, and continue listening to feedback about how we can further engage and elevate trusted developers. Android developers can expect to see more on this front in the coming year. Our global teams of product managers, engineers, policy experts, and operations leaders are more excited than ever to advance the safety of the platform and forge a sustaining trust with our users. We look forward to building an even better Google Play experience.

A New Standard for Mobile App Security

April 15, 2021

Posted by Brooke Davis and Eugene Liderman, Android Security and Privacy Team

With all of the challenges from this past year, users have become increasingly dependent on their mobile devices to create fitness routines, stay connected with loved ones, work remotely, and order things like groceries with ease. According to eMarketer, in 2020 users spent over three and a half hours per day using mobile apps. With so much time spent on mobile devices, ensuring the safety of mobile apps is more important than ever. Despite the importance of digital security, there isn’t a consistent industry standard for assessing mobile apps. Existing guidelines tend to be either too lightweight or too onerous for the average developer, and lack a compliance arm. That’s why we're excited to share ioXt’s announcement of a new Mobile Application Profile which provides a set of security and privacy requirements with defined acceptance criteria which developers can certify their apps against. Over 20 industry stakeholders, including Google, Amazon, and a number of certified labs such as NCC Group and Dekra, as well as automated mobile app security testing vendors like NowSecure collaborated to develop this new security standard for mobile apps. We’ve seen early interest from Internet of Things (IoT) and virtual private network (VPN) developers, however the standard is appropriate for any cloud connected service such as social, messaging, fitness, or productivity apps. The Internet of Secure Things Alliance (ioXt) manages a security compliance assessment program for connected devices. ioXt has over 300 members across various industries, including Google, Amazon, Facebook, T-Mobile, Comcast, Zigbee Alliance, Z-Wave Alliance, Legrand, Resideo, Schneider Electric, and many others. With so many companies involved, ioXt covers a wide range of device types, including smart lighting, smart speakers, and webcams, and since most smart devices are managed through apps, they have expanded coverage to include mobile apps with the launch of this profile. The ioXt Mobile Application Profile provides a minimum set of commercial best practices for all cloud connected apps running on mobile devices. This security baseline helps mitigate against common threats and reduces the probability of significant vulnerabilities. The profile leverages existing standards and principles set forth by OWASP MASVS and the VPN Trust Initiative, and allows developers to differentiate security capabilities around cryptography, authentication, network security, and vulnerability disclosure program quality. The profile also provides a framework to evaluate app category specific requirements which may be applied based on the features contained in the app. For example, an IoT app only needs to certify under the Mobile Application profile, whereas a VPN app must comply with the Mobile Application profile, plus the VPN extension. Certification allows developers to demonstrate product safety and we’re excited about the opportunity for this standard to push the industry forward. We observed that app developers were very quick to resolve any issues that were identified during their blackbox evaluations against this new standard, oftentimes with turnarounds in a matter of days. At launch, the following apps have been certified: Comcast, ExpressVPN, GreenMAX, Hubspace, McAfee Innovations, NordVPN, OpenVPN for Android, Private Internet Access, VPN Private, as well as the Google One app, including VPN by Google One. We look forward to seeing adoption of the standard grow over time and for those app developers that are already investing in security best practices to be able to highlight their efforts. The standard also serves as a guiding light to inspire more developers to invest in mobile app security. If you are interested in learning more about the ioXt Alliance and how to get your app certified, visit https://compliance.ioxtalliance.org/sign-up and check out Android’s guidelines for building secure apps here.

Rust in the Linux kernel

April 14, 2021

Posted by Wedson Almeida Filho, Android Team In our previous post, we announced that Android now supports the Rust programming language for developing the OS itself. Related to this, we are also participating in the effort to evaluate the use of Rust as a supported language for developing the Linux kernel. In this post, we discuss some technical aspects of this work using a few simple examples. C has been the language of choice for writing kernels for almost half a century because it offers the level of control and predictable performance required by such a critical component. Density of memory safety bugs in the Linux kernel is generally quite low due to high code quality, high standards of code review, and carefully implemented safeguards. However, memory safety bugs do still regularly occur. On Android, vulnerabilities in the kernel are generally considered high-severity because they can result in a security model bypass due to the privileged mode that the kernel runs in. We feel that Rust is now ready to join C as a practical language for implementing the kernel. It can help us reduce the number of potential bugs and security vulnerabilities in privileged code while playing nicely with the core kernel and preserving its performance characteristics. Supporting Rust We developed an initial prototype of the Binder driver to allow us to make meaningful comparisons between the safety and performance characteristics of the existing C version and its Rust counterpart. The Linux kernel has over 30 million lines of code, so naturally our goal is not to convert it all to Rust but rather to allow new code to be written in Rust. We believe this incremental approach allows us to benefit from the kernel’s existing high-performance implementation while providing kernel developers with new tools to improve memory safety and maintain performance going forward. We joined the Rust for Linux organization, where the community had already done and continues to do great work toward adding Rust support to the Linux kernel build system. We also need designs that allow code in the two languages to interact with each other: we're particularly interested in safe, zero-cost abstractions that allow Rust code to use kernel functionality written in C, and how to implement functionality in idiomatic Rust that can be called seamlessly from the C portions of the kernel. Since Rust is a new language for the kernel, we also have the opportunity to enforce best practices in terms of documentation and uniformity. For example, we have specific machine-checked requirements around the usage of unsafe code: for every unsafe function, the developer must document the requirements that need to be satisfied by callers to ensure that its usage is safe; additionally, for every call to unsafe functions (or usage of unsafe constructs like dereferencing a raw pointer), the developer must document the justification for why it is safe to do so. Just as important as safety, Rust support needs to be convenient and helpful for developers to use. Let’s get into a few examples of how Rust can assist kernel developers in writing drivers that are safe and correct. Example driver We'll use an implementation of a semaphore character device. Each device has a current value; writes of n bytes result in the device value being incremented by n; reads decrement the value by 1 unless the value is 0, in which case they will block until they can decrement the count without going below 0. Suppose semaphore is a file representing our device. We can interact with it from the shell as follows: > cat semaphore When semaphore is a newly initialized device, the command above will block because the device's current value is 0. It will be unblocked if we run the following command from another shell because it increments the value by 1, which allows the original read to complete: > echo -n a > semaphore We could also increment the count by more than 1 if we write more data, for example: > echo -n abc > semaphore increments the count by 3, so the next 3 reads won't block. To allow us to show a few more aspects of Rust, we'll add the following features to our driver: remember what the maximum value was throughout the lifetime of a device, and remember how many reads each file issued on the device. We'll now show how such a driver would be implemented in Rust, contrasting it with a C implementation. We note, however, we are still early on so this is all subject to change in the future. How Rust can assist the developer is the aspect that we'd like to emphasize. For example, at compile time it allows us to eliminate or greatly reduce the chances of introducing classes of bugs, while at the same time remaining flexible and having minimal overhead. Character devices A developer needs to do the following to implement a driver for a new character device in Rust:

Implement the FileOperations trait: all associated functions are optional, so the developer only needs to implement the relevant ones for their scenario. They relate to the fields in C's struct file_operations.

Implement the FileOpener trait: it is a type-safe equivalent to C's open field of struct file_operations.

Register the new device type with the kernel: this lets the kernel know what functions need to be called in response to files of this new type being operated on.

The following outlines how the first two steps of our example compare in Rust and C:

impl FileOpener<Arc<Semaphore>> for FileState {
    fn open(
        shared: &Arc<Semaphore>
    ) -> KernelResult<Box<Self>> {
        [...]
    }
}
 
impl FileOperations for FileState {
    type Wrapper = Box<Self>;
 
    fn read(
        &self,
        _: &File,
        data: &mut UserSlicePtrWriter,
        offset: u64
    ) -> KernelResult<usize> {
        [...]
    }
 
    fn write(
        &self,
        data: &mut UserSlicePtrReader,
        _offset: u64
    ) -> KernelResult<usize> {
        [...]
    }
 
    fn ioctl(
        &self,
        file: &File,
        cmd: &mut IoctlCommand
    ) -> KernelResult<i32> {
        [...]
    }
 
    fn release(_obj: Box<Self>, _file: &File) {
        [...]
    }
 
    declare_file_operations!(read, write, ioctl);
}

static 
int semaphore_open(struct inode *nodp,
                   struct file *filp)
{
    struct semaphore_state *shared =
        container_of(filp->private_data,
                     struct semaphore_state,
                     miscdev);
    [...]
}
 
static
ssize_t semaphore_write(struct file *filp,
                        const char __user *buffer,
                        size_t count, loff_t *ppos)
{
    struct file_state *state = filp->private_data;
    [...]
}
 
static
ssize_t semaphore_read(struct file *filp,
                       char __user *buffer,
                       size_t count, loff_t *ppos)
{
    struct file_state *state = filp->private_data;
    [...]
}
 
static
long semaphore_ioctl(struct file *filp,
                     unsigned int cmd,
                     unsigned long arg)
{
    struct file_state *state = filp->private_data;
    [...]
}
 
static
int semaphore_release(struct inode *nodp,
                      struct file *filp)
{
    struct file_state *state = filp->private_data;
    [...]
}
 
static const struct file_operations semaphore_fops = {
        .owner = THIS_MODULE,
        .open = semaphore_open,
        .read = semaphore_read,
        .write = semaphore_write,
        .compat_ioctl = semaphore_ioctl,
        .release = semaphore_release,
};

Character devices in Rust benefit from a number of safety features:

Per-file state lifetime management: FileOpener::open returns an object whose lifetime is owned by the caller from then on. Any object that implements the PointerWrapper trait can be returned, and we provide implementations for Box<T> and Arc<T>, so developers that use Rust's idiomatic heap-allocated or reference-counted pointers have no additional requirements.

All associated functions in FileOperations receive non-mutable references to self (more about this below), except the release function, which is the last function to be called and receives the plain object back (and its ownership with it). The release implementation can then defer the object destruction by transferring its ownership elsewhere, or destroy it then; in the case of a reference-counted object, 'destruction' means decrementing the reference count (and actual object destruction if the count goes to zero).

That is, we use Rust's ownership discipline when interacting with C code by handing the C portion ownership of a Rust object, allowing it to call functions implemented in Rust, then eventually giving ownership back. So as long as the C code is correct, the lifetime of Rust file objects work seamlessly as well, with the compiler enforcing correct lifetime management on the Rust side, for example: open cannot return stack-allocated pointers or heap-allocated objects containing pointers to the stack, ioctl/read/write cannot free (or modify without synchronization) the contents of the object stored in filp->private_data, etc.

Non-mutable references: the associated functions called between open and release all receive non-mutable references to self because they can be called concurrently by multiple threads and Rust aliasing rules prohibit more than one mutable reference to an object at any given time.

If a developer needs to modify some state (and they generally do), they can do so via interior mutability: mutable state can be wrapped in a Mutex<T> or SpinLock<T> (or atomics) and safely modified through them.

This prevents, at compile-time, bugs where a developer fails to acquire the appropriate lock when accessing a field (the field is inaccessible), or when a developer fails to wrap a field with a lock (the field is read-only).

Per-device state: when file instances need to share per-device state, which is a very common occurrence in drivers, they can do so safely in Rust. When a device is registered, a typed object can be provided and a non-mutable reference to it is provided when FileOperation::open is called. In our example, the shared object is wrapped in Arc<T>, so files can safely clone and hold on to a reference to them.

The reason FileOperation is its own trait (as opposed to, for example, open being part of the FileOperations trait) is to allow a single file implementation to be registered in different ways.

This eliminates opportunities for developers to get the wrong data when trying to retrieve shared state. For example, in C when a miscdevice is registered, a pointer to it is available in filp->private_data; when a cdev is registered, a pointer to it is available in inode->i_cdev. These structs are usually embedded in an outer struct that contains the shared state, so developers usually use the container_of macro to recover the shared state. Rust encapsulates all of this and the potentially troublesome pointer casts in a safe abstraction.

Static typing: we take advantage of Rust's support for generics to implement all of the above functions and types with static types. So there are no opportunities for a developer to convert an untyped variable or field to the wrong type. The C code in the table above has casts from an essentially untyped (void *) pointer to the desired type at the start of each function: this is likely to work fine when first written, but may lead to bugs as the code evolves and assumptions change. Rust would catch any such mistakes at compile time.

File operations: as we mentioned before, a developer needs to implement the FileOperations trait to customize the behavior of their device. They do this with a block starting with impl FileOperations for Device, where Device is the type implementing the file behavior (FileState in our example). Once inside this block, tools know that only a limited number of functions can be defined, so they can automatically insert the prototypes. (Personally, I use neovim and the rust-analyzer LSP server.)

While we use this trait in Rust, the C portion of the kernel still requires an instance of struct file_operations. The kernel crate automatically generates one from the trait implementation (and optionally the declare_file_operations macro): although it has code to generate the correct struct, it is all const, so evaluated at compile-time with zero runtime cost.

Ioctl handling For a driver to provide a custom ioctl handler, it needs to implement the ioctl function that is part of the FileOperations trait, as exemplified in the table below.

fn ioctl(
    &self,
    file: &File,
    cmd: &mut IoctlCommand
) -> KernelResult<i32> {
    cmd.dispatch(self, file)
}
 
impl IoctlHandler for FileState {
    fn read(
        &self,
        _file: &File,
        cmd: u32,
        writer: &mut UserSlicePtrWriter
    ) -> KernelResult<i32> {
        match cmd {
            IOCTL_GET_READ_COUNT => {
                writer.write(
                    &self
                    .read_count
                    .load(Ordering::Relaxed))?;
                Ok(0)
            }
            _ => Err(Error::EINVAL),
        }
    }
 
    fn write(
        &self,
        _file: &File,
        cmd: u32,
        reader: &mut UserSlicePtrReader
    ) -> KernelResult<i32> {
        match cmd {
            IOCTL_SET_READ_COUNT => {
                self
                .read_count
                .store(reader.read()?,
                       Ordering::Relaxed);
                Ok(0)
            }
            _ => Err(Error::EINVAL),
        }
    }
}

#define IOCTL_GET_READ_COUNT _IOR('c', 1, u64)
#define IOCTL_SET_READ_COUNT _IOW('c', 1, u64)
 
static
long semaphore_ioctl(struct file *filp,
                     unsigned int cmd,
                     unsigned long arg)
{
    struct file_state *state = filp->private_data;
    void __user *buffer = (void __user *)arg;
    u64 value;
 
    switch (cmd) {
    case IOCTL_GET_READ_COUNT:
        value = atomic64_read(&state->read_count);
        if (copy_to_user(buffer, &value, sizeof(value)))
            return -EFAULT;
        return 0;
    case IOCTL_SET_READ_COUNT:
        if (copy_from_user(&value, buffer, sizeof(value)))
            return -EFAULT;
        atomic64_set(&state->read_count, value);
        return 0;
    default:
        return -EINVAL;
    }
}

Ioctl commands are standardized such that, given a command, we know whether a user buffer is provided, its intended use (read, write, both, none), and its size. In Rust, we provide a dispatcher (accessible by calling cmd.dispatch) that uses this information to automatically create user memory access helpers and pass them to the caller. A driver is not required to use this though. If, for example, it doesn't use the standard ioctl encoding, Rust offers the flexibility of simply calling cmd.raw to extract the raw arguments and using them to handle the ioctl (potentially with unsafe code, which will need to be justified). However, if a driver implementation does use the standard dispatcher, it will benefit from not having to implement any unsafe code, and:

The pointer to user memory is never a native pointer, so the developer cannot accidentally dereference it.

The types that allow the driver to read from user space only allow data to be read once, so we eliminate the risk of time-of-check to time-of-use (TOCTOU) bugs because when a driver needs to access data twice, it needs to copy it to kernel memory, where an attacker is not allowed to modify it. Excluding unsafe blocks, there is no way to introduce this class of bugs in Rust.

No accidental overflow of the user buffer: we'll never read or write past the end of the user buffer because this is enforced automatically based on the size encoded in the ioctl command. In our example above, the implementation of IOCTL_GET_READ_COUNT only has access to an instance of UserSlicePtrWriter, which limits the number of writable bytes to sizeof(u64) as encoded in the ioctl command.

No mixing of reads and writes: we'll never write buffers for ioctls that are only meant to read and never read buffers for ioctls that are only meant to write. This is enforced by read and write handlers only getting instances of UserSlicePtrWriter and UserSlicePtrReader respectively.

All of the above could potentially also be done in C, but it's very easy for developers to (likely unintentionally) break contracts that lead to unsafety; Rust requires unsafe blocks for this, which should only be used in rare cases and brings additional scrutiny. Additionally, Rust offers the following:

The types used to read and write user memory do not implement the Send and Sync traits, which means that they (and pointers to them) are not safe to be used in another thread context. In Rust, if a driver developer attempted to write code that passed one of these objects to another thread (where it wouldn't be safe to use them because it isn't necessarily in the right memory manager context), they would get a compilation error.

When calling IoctlCommand::dispatch, one might understandably think that we need dynamic dispatching to reach the actual handler implementation (which would incur additional cost in comparison to C), but we don't. Our usage of generics will lead the compiler to monomorphize the function, which will result in static function calls that can even be inlined if the optimizer so chooses.

Locking and condition variablesWe allow developers to use mutexes and spinlocks to provide interior mutability. In our example, we use a mutex to protect mutable data; in the tables below we show the data structures we use in C and Rust, and how we implement a wait until the count is nonzero so that we can satisfy a read:

struct SemaphoreInner {
    count: usize,
    max_seen: usize,
}
 
struct Semaphore {
    changed: CondVar,
    inner: Mutex<SemaphoreInner>,
}
 
struct FileState {
    read_count: AtomicU64,
    shared: Arc<Semaphore>,
}

struct semaphore_state {
    struct kref ref;
    struct miscdevice miscdev;
    wait_queue_head_t changed;
    struct mutex mutex;
    size_t count;
    size_t max_seen;
};
 
struct file_state {
    atomic64_t read_count;
    struct semaphore_state *shared;
};

fn consume(&self) -> KernelResult {
    let mut inner = self.shared.inner.lock();
    while inner.count == 0 {
        if self.shared.changed.wait(&mut inner) {
            return Err(Error::EINTR);
        }
    }
    inner.count -= 1;
    Ok(())
}

static int semaphore_consume(
    struct semaphore_state *state)
{
    DEFINE_WAIT(wait);
 
    mutex_lock(&state->mutex);
    while (state->count == 0) {
        prepare_to_wait(&state->changed, &wait,
                        TASK_INTERRUPTIBLE);
        mutex_unlock(&state->mutex);
        schedule();
        finish_wait(&state->changed, &wait);
        if (signal_pending(current))
            return -EINTR;
        mutex_lock(&state->mutex);
    }
 
    state->count--;
    mutex_unlock(&state->mutex);
 
    return 0;
}

We note that such waits are not uncommon in the existing C code, for example, a pipe waiting for a "partner" to write, a unix-domain socket waiting for data, an inode search waiting for completion of a delete, or a user-mode helper waiting for state change. The following are benefits from the Rust implementation:

The Semaphore::inner field is only accessible when the lock is held, through the guard returned by the lock function. So developers cannot accidentally read or write protected data without locking it first. In the C example above, count and max_seen in semaphore_state are protected by mutex, but there is no enforcement that the lock is held while they're accessed.

Resource Acquisition Is Initialization (RAII): the lock is unlocked automatically when the guard (inner in this case) goes out of scope. This ensures that locks are always unlocked: if the developer needs to keep a lock locked, they can keep the guard alive, for example, by returning the guard itself; conversely, if they need to unlock before the end of the scope, they can explicitly do it by calling the drop function.

Developers can use any lock that implements the Lock trait, which includes Mutex and SpinLock, at no additional runtime cost when compared to a C implementation. Other synchronization constructs, including condition variables, also work transparently and with zero additional run-time cost.

Rust implements condition variables using kernel wait queues. This allows developers to benefit from atomic release of the lock and putting the thread to sleep without having to reason about low-level kernel scheduler functions. In the C example above, semaphore_consume is a mix of semaphore logic and subtle Linux scheduling: for example, the code is incorrect if mutex_unlock is called before prepare_to_wait because it may result in a wake up being missed.

No unsynchronized access: as we mentioned before, variables shared by multiple threads/CPUs must be read-only, with interior mutability being the solution for cases when mutability is needed. In addition to the example with locks above, the ioctl example in the previous section also has an example of using an atomic variable; Rust also requires developers to specify how memory is to be synchronized by atomic accesses. In the C part of the example, we happen to use atomic64_t, but the compiler won't alert a developer to this need.

Error handling and control flow In the tables below, we show how open, read, and write are implemented in our example driver:

fn read(
    &self,
    _: &File,
    data: &mut UserSlicePtrWriter,
    offset: u64
) -> KernelResult<usize> {
   if data.is_empty() || offset > 0 {
        return Ok(0);
    }
 
    self.consume()?;
    data.write_slice(&[0u8; 1])?;
    self.read_count.fetch_add(1, Ordering::Relaxed);
    Ok(1)
}

static
ssize_t semaphore_read(struct file *filp,
                       char __user *buffer,
                       size_t count, loff_t *ppos)
{
    struct file_state *state = filp->private_data;
    char c = 0;
    int ret;
 
    if (count == 0 || *ppos > 0)
        return 0;
 
    ret = semaphore_consume(state->shared);
    if (ret)
        return ret;
 
    if (copy_to_user(buffer, &c, sizeof(c)))
        return -EFAULT;
 
    atomic64_add(1, &state->read_count);
    *ppos += 1;
    return 1;
}

fn write(
    &self,
    data: &mut UserSlicePtrReader,
    _offset: u64
) -> KernelResult<usize> {
   {
        let mut inner = self.shared.inner.lock();
        inner.count = inner.count.saturating_add(data.len());
        if inner.count > inner.max_seen {
            inner.max_seen = inner.count;
        }
    }
 
    self.shared.changed.notify_all();
    Ok(data.len())
}

static
ssize_t semaphore_write(struct file *filp,
                        const char __user *buffer,
                        size_t count, loff_t *ppos)
{
    struct file_state *state = filp->private_data;
    struct semaphore_state *shared = state->shared;
 
    mutex_lock(&shared->mutex);
    shared->count += count;
    if (shared->count < count)
        shared->count = SIZE_MAX;
 
    if (shared->count > shared->max_seen)
        shared->max_seen = shared->count;
 
    mutex_unlock(&shared->mutex);
 
    wake_up_all(&shared->changed);
    return count;
}

fn open(
    shared: &Arc<Semaphore>
) -> KernelResult<Box<Self>> {
    Ok(Box::try_new(Self {
        read_count: AtomicU64::new(0),
        shared: shared.clone(),
    })?)
}

static 
int semaphore_open(struct inode *nodp,
                   struct file *filp)
{
    struct semaphore_state *shared =
        container_of(filp->private_data,
                     struct semaphore_state,
                     miscdev);
    struct file_state *state;
 
    state = kzalloc(sizeof(*state), GFP_KERNEL);
    if (!state)
        return -ENOMEM;
 
    kref_get(&shared->ref);
    state->shared = shared;
    atomic64_set(&state->read_count, 0);
 
    filp->private_data = state;
 
    return 0;
}

They illustrate other benefits brought by Rust:

The ? operator: it is used by the Rust open and read implementations to do error handling implicitly; the developer can focus on the semaphore logic, the resulting code being quite small and readable. The C versions have error-handling noise that can make them less readable.

Required initialization: Rust requires all fields of a struct to be initialized on construction, so the developer can never accidentally fail to initialize a field; C offers no such facility. In our open example above, the developer of the C version could easily fail to call kref_get (even though all fields would have been initialized); in Rust, the user is required to call clone (which increments the ref count), otherwise they get a compilation error.

RAII scoping: the Rust write implementation uses a statement block to control when inner goes out of scope and therefore the lock is released.

Integer overflow behavior: Rust encourages developers to always consider how overflows should be handled. In our write example, we want a saturating one so that we don't end up with a zero value when adding to our semaphore. In C, we need to manually check for overflows, there is no additional support from the compiler.

What's next The examples above are only a small part of the whole project. We hope it gives readers a glimpse of the kinds of benefits that Rust brings. At the moment we have nearly all generic kernel functionality needed by Binder neatly wrapped in safe Rust abstractions, so we are in the process of gathering feedback from the broader Linux kernel community with the intent of upstreaming the existing Rust support. We also continue to make progress on our Binder prototype, implement additional abstractions, and smooth out some rough edges. This is an exciting time and a rare opportunity to potentially influence how the Linux kernel is developed, as well as inform the evolution of the Rust language. We invite those interested to join us in Rust for Linux and attend our planned talk at Linux Plumbers Conference 2021!
Thanks Nick Desaulniers, Kees Cook, and Adrian Taylor for contributions to this post. Special thanks to Jeff Vander Stoep for contributions and editing, and to Greg Kroah-Hartman for reviewing and contributing to the code examples.

Security Blog