Google Online Security Blog: February 2022

The Wayback Machine - https://web.archive.org/web/20220318002537/https://security.googleblog.com/2022/02/

Mitigating kernel risks on 32-bit ARM

February 23, 2022

Posted by Ard Biesheuvel, Google Open Source Security Team

'odd fixesBOMRunning 32-bit Linux on 64-bit ARM systems

Ironically, at these low price points, the DRAM is actually the dominant component in terms of BOM cost, and many of these 32-bit ARM systems incorporate a cheap ARMv8 SoC that happens to be capable of running in 64-bit mode as well. The reason for running 32-bit applications nonetheless is that these generally use less of the expensive DRAM, and can be deployed directly without the need to recompile the binaries. As 32-bit applications don't need a 64-bit kernel (which itself uses more memory due to its internal use of 64-bit pointers), the product ships with a 32-bit kernel instead.

If you're choosing to use a 32-bit kernel for its smaller memory footprint, it's not without risks. You'll likely experience performance issues, unpatched vulnerabilities, and unexpected misbehaviors such as:

32-bit kernels generally cannot manage more than 1 GiB of physical memory without resorting to HIGHMEM bouncing, and cannot provide a full virtual address space of 4 GiB to user space, as 64-bit kernels can.
Side channels or other flaws caused by silicon errata may exist that haven't been mitigated in 32-bit kernels. For example, the hardening against Spectre and Meltdown vulnerabilities were only done for ARMv7 32-bit only CPUs, and many ARMv8 cores running in 32-bit mode may still be vulnerable (only Cortex-A73 and A75 are handled specifically). And in general, silicon flaws in 64-bit parts that affect the 32-bit kernel are less likely to be found or documented, simply because the silicon validation teams don’t prioritize them.
The 32-bit ARM kernel does not implement the elaborate alternatives patching framework that is used by other architectures to implement handling of silicon errata, which are particular to certain revisions of certain CPUs. Instead, on 32-bit multiplatform kernels, we simply enable all errata workarounds that may be needed by any of the cores that may ever run the image in question, potentially affecting performance unnecessarily on cores that have no need for them.
Silicon vendors are phasing out 32-bit support in the longer term. Given an ecosystem containing a handful of operating systems and thousands of applications, support for 32-bit operating systems (which is more complex technically) is highly likely to be dropped first. For products with longer life cycles, long-term procurement contracts for components available today are usually much more costly than adjusting the BOM over time and using newer, cheaper parts.
The 32-bit kernel does not implement kernel address space randomization, and even if it did, its comparatively tiny address space simply leaves very little space for randomization. Other hardening features, such as rodata=full or hierarchical eXecute Never attributes, are missing as well on 32-bit, and are not likely to be implemented, either due to lack of support in the architecture, or because of the complexity of the 32-bit memory management code, which still supports all of the different architecture revisions dating back to the initial Linux port running on the Risc PC.

Keeping the 32-bit ARM kernel secure

There are cases, though, where using the 32-bit kernel is the only option, e.g., if the CPUs are in fact 32-bit only (which is the case even for some ARMv8 cores such as Cortex-A32), or when relying on an existing 32-bit only codebase running in the kernel (drivers for legacy peripherals). Note that in such cases, it still makes sense to use the most recent kernel version compatible with the hardware, since we are in fact making an effort to enable some of the existing hardening features on 32-bit ARM as well.

THREAD_INFO_IN_TASK for v7 SMP cores

The v5.16 release of the Linux kernel implements support for THREAD_INFO_IN_TASK when running on ARMv7 SMP systems. This protects the kernel's per-task bookkeeping (called thread_info), which lives on the far (and normally unused) end of the stack, against stack overflows which may occur in rare -yet sometimes exploitable- cases where the control flow of the program simply ends up accumulating more state than the stack can hold. (Note that a stack overflow is not the same as a stack buffer overflow, where the overflow happens in the opposite direction.)

By moving thread_info off the stack and into the kernel heap, and by using a special SMP CPU register to keep track of its location, we can mitigate the risk of stack overflows resulting in thread_info corruption. However, it does not prevent stack overflows themselves: these may still occur, and result in corruption of other data structures that happen to be adjacent to the task stack in memory.

THREAD_INFO_IN_TASK for other cores

For CPUs that lack this special SMP CPU register, we also proposed an implementation of THREAD_INFO_IN_TASK that is expected to land in v5.18. Instead of a special register, it uses a global variable to keep track of the location of thread_info.

VMAP_STACK support

Preventing stack overflows from corrupting unrelated memory contents is the goal of VMAP_STACK, which we are enabling for 32-bit ARM as well. When VMAP_STACK is enabled, kernel mode stacks are allocated from the kernel heap as before, but mapped into a different part of the kernel's address space, and surrounded by guard regions, which are guaranteed to be kept unpopulated. Given that accesses to such unpopulated regions will trigger an exception, the kernel's memory management layer can step in and terminate the program as soon as a stack overflow occurs, and prevent it from causing memory corruption.

Support for IRQ stacks

Coming up with a bounded worst case on which to base the size of the kernel stack is rather hard, especially given the fact that it is shared between the program itself and any exception handling routines that may be called on its behalf, including interrupt handlers. To mitigate the risk of a pathological worst case occurring, where an interrupt fires that needs a lot of stack space right at a time when most of the stack is already being used by the program, we are also enabling IRQ_STACKS for 32-bit ARM, which will run handlers of both hard and soft interrupts from a dedicated stack, one for each CPU. By decoupling the task and interrupt contexts like this, the likelihood that a well-behaved program needs to be terminated due to stack overflow should be all but eliminated.

Conclusion

With these changes in place, kernel stack overflow protection will be available for all ARM systems supported by Linux, including ancient ones like the Risc PC or Netwinder, provided that it runs a Linux distribution that is keeping up with the times.

However, relying on legacy hardware and software comes with a risk, and even though we try to help keep users of the 32-bit kernel as safe as we reasonably can, it is not the right choice for new designs that incorporate 64-bit capable hardware.

🌹 Roses are red, Violets are blue 💙 Giving leets 🧑‍💻 more sweets 🍭 All of 2022!

February 14, 2022

Posted by Eduardo Vela, Vulnerability Matchmaker
Until December 31 2022 we will pay 20,000 to 91,337 USD for exploits of vulnerabilities in the Linux Kernel, Kubernetes, GKE or kCTF that are exploitable on our test lab.expansionCVE-2021-4154CVE-2021-22600patchCVE-2022-0185writeupSyzkaller

Reporting a 0day will not require including a flag at first. We heard some concerns from participants that exploiting a 0day in the shared cluster could leak it to other participants. As such, we will only ask for the exploit checksum (but you still have to exploit the bug and submit the flag within a week after the patch is merged on mainline). Please make sure that your exploit works on COS with minimal modifications (test it on your own kCTF cluster), as some common exploit primitives (like eBPF and userfaultfd) might not be available.

Reporting a 1day will require including a link to the patch. We will automatically publish the patches of all submissions if the flag is valid. We also encourage you all to include a link to a Syzkaller dashboard report if applicable in order to help reduce duplicate submissions and so you can see which bugs were exploited already.

You will be able to submit the exploit in the same form you submit the flag. If you had submitted an exploit checksum for a 0day, please make sure that you include the original exploit as well as the final exploit and make sure to submit it within a week after the patch is merged on mainline. The original exploit shouldn't require major modifications to work. Note that we need to be able to understand your exploit, so please add comments to explain what it is doing.

We are now running two clusters, one on the REGULAR release channel and another one on the RAPID release channel. This should provide more flexibility whenever a vulnerability is only exploitable on modern versions of the Linux Kernel or Kubernetes.

31,337 USD to the first valid exploit submission for a given vulnerability. This will only be paid once per vulnerability and only once per cluster version/build (available at /etc/node-os-release).

0 USD for exploits for duplicate exploits for the same vulnerability. The bonuses below might still apply.

Bonuses

20,000 USD for exploits for 0day vulnerabilities. This will only be paid once per vulnerability to the first valid exploit submission.

To submit 0days, please test your exploit (we recommend to test it on your own kCTF cluster to avoid leaking it to other participants), make a checksum and send the checksum to us. Within a week after the vulnerability is fixed on the mainline, submit the form as a 1day and include the exploit of which you sent a checksum to us.

20,000 USD for exploits for vulnerabilities that do not require unprivileged user namespaces (CLONE_NEWUSER). This will only be paid once per vulnerability to the first valid exploit submission.

Our test lab allows unprivileged user namespaces, so we will manually check the exploits to check if they work without unprivileged user namespaces when deciding whether to issue the bonus. We decided to issue additional rewards for exploits that do not require unprivileged user namespaces because containers default seccomp policy does not allow the use of unprivileged user namespaces on containers that are run without CAP_SYS_ADMIN. This feature is now available on Kubernetes and all nodes running on GKE Autopilot have it enabled by default.

20,000 USD for exploits using novel exploit techniques. This is a bonus in addition to the base rewards (applies for duplicate exploits). To qualify for this additional reward please send us a write-up explaining it.

An example of something considered as a novel technique could be the exploitation of previously unknown objects to transform a limited primitive into a more powerful one, such as an arbitrary/out-of-bounds read/write or arbitrary free. For example, in all our submissions, researchers leveraged message queues to achieve kernel information leaks. We are looking for similarly powerful techniques that allow heap exploits to be “plugged in” and immediately allow kernel access. Another example is bypassing a common security mitigation or a technique for exploiting a class of vulnerabilities more reliably.

GKE releasesherehttps://dojo.pwn.college/challenges/kernelhttps://github.com/xairy/linux-kernel-exploitationthe rulesbughunters.google.comhereon Discord

Vulnerability Reward Program: 2021 Year in Review

February 10, 2022

Posted by Sarah Jacobus, Vulnerability Rewards Team

Last year was another record setter for our Vulnerability Reward Programs (VRPs). Throughout 2021, we partnered with the security researcher community to identify and fix thousands of vulnerabilities – helping keep our users and the internet safe.

Thanks to these incredible researchers, Vulnerability Reward Programs across Google continued to grow, and we are excited to report that in 2021 we awarded a record breaking $8,700,000 in vulnerability rewards – with researchers donating over $300,000 of their rewards to a charity of their choice.

We also launched bughunters.google.com in 2021, a public researcher portal dedicated to keeping Google products and the internet safe and secure. This new platform brings all of our VRPs (Google, Android, Abuse, Chrome, and Google Play) closer together and provides a single intake form, making security bug submission easier than ever. We’re excited about everything the new Bug Hunters portal has to offer, including:

More opportunities for interaction and a bit of healthy competition through gamification, per-country leaderboards, awards/badges for certain bugs, and more!
A more functional and aesthetically pleasing leaderboard. We know a lot of you are using your achievements in our VRPs to find jobs (we’re hiring!) and we hope this acts as a useful resource.
A stronger emphasis on learning: bug hunters can improve their skills through the content available in our new Bug Hunter University
Streamlined publication process: we know the value that knowledge sharing brings to our community. That’s why we want to make it easier for you to publish your bug reports.
We now offer swag! The first 20 folks who share this blog post on Twitter and tag @GoogleVRP will receive a gift voucher for swag in their DMs.

As in past years, we are sharing our 2021 Year in Review statistics across all of our programs. We would like to give a special thank you to all of our dedicated researchers - we look forward to more collaboration in the future!

Android

The Android VRP doubled its 2020 total payouts in 2021 with nearly $3 million dollars in rewards, and awarded the highest payout in Android VRP history: an exploit chain discovered in Android receiving a reward of $157,000!

Our industry leading prize of $1,500,000 for a compromise of our Titan-M Security chip used in our Pixel device remains unclaimed - for more information on this reward and Android exploit chain rewards, please visit our public rules page.

The program also launched the Android Chipset Security Reward Program (ACSRP), a vulnerability reward program offered by Google in collaboration with manufacturers of certain popular Android chipsets. This private, invite-only program, provides reward and recognition for contributions of security researchers who invest their time and effort into helping make Android devices more secure. In 2021 the ACSRP paid out $296,000 for over 220 valid and unique security reports.

We would like to give a special shoutout to some of our top researchers whose continued hard work keeps Android safe and secure:

Aman Pandey of Bugsmirror Team has skyrocketed to our top researcher last year, submitting 232 vulnerabilities in 2021! Since submitting their first report in 2019, Aman has reported over 280 valid vulnerabilities to the Android VRP and has been a crucial part of making our program so successful.
Yu-Cheng Lin (林禹成) (@AndroBugs) has been another phenomenal researcher for the Android VRP, submitting a whopping 128 valid reports to the program in 2021.
Researcher gzobqq@gmail.com discovered a critical exploit chain in Android (CVE-2021-39698) , receiving the highest payout in Android VRP history of $157,000.

Chrome

This year the Chrome VRP also set some new records – 115 Chrome VRP researchers were rewarded for 333 unique Chrome security bug reports submitted in 2021, totaling $3.3 million in VRP rewards. The contributions not only help us to improve Chrome, but also the web at large by bolstering the security of all browsers based on Chromium.

Of the $3.3 million, $3.1 million was awarded for Chrome Browser security bugs and $250,500 for Chrome OS bugs, including a $45,000 top reward amount for an individual Chrome OS security bug report and $27,000 for an individual Chrome Browser security bug report.

Of these totals, $58,000 was awarded for security issues discovered by fuzzers contributed by VRP researchers to the Chrome Fuzzing program. Each valid report from an externally provided fuzzer received a $1,000 patch bonus, with one fuzzer report receiving a $16,000 reward.

The Chrome VRP would not be able to smash these records over the last year without the efforts of so many exceptional VRP researchers. We’d like to highlight a few researcher achievements made in 2021:

Rory McNamara, a Chrome OS VRP researcher who has been participating in the Chrome VRP for five years, became the highest awarded Chrome VRP researcher of all time. This year he was rewarded for six reports achieving root privilege escalation in Chrome OS, one of which received the highest reward amount achieved for a single Chrome bug report in 2021 at $45,000.

Chrome Browser VRP researcher Leecraso (@leecraso) of 360 Vulnerability Research Institute was the most awarded researcher of 2021, with 18 valid bug reports; a majority of which were for memory corruption vulnerabilities affecting the browser process.

We love when researchers write about their findings (only after we have publicly disclosed the bug, of course)! Chrome Browser VRP researcher Brendon Tiszka wrote an excellent two-part blog series on his discovery and exploitation of a V8 vulnerability, CVE-2021-21225, the analysis and reporting of which earned him a $22,000 VRP reward.

Huge thanks and congratulations to all Chrome VRP researchers that helped us make Chrome and Chrome OS more safe for all users in 2021!.

Google Play

Google Play paid out $550,000 in rewards to over 60 unique security researchers.

The Google Play Security Reward Program also released their Android App Hacking Workshop content and published a blog on their work to empower the next generation of Android Application Security Researchers.

kCTF VRP

In November we expanded our reward amounts for exploits against our kCTF cluster from 5,000-10,000 up to 31,337-50,337 USD. In the last 3 months we were happy to have several participants receive $175,685 USD in rewards. We also extended the timeline of the increased rewards until February 14 (from January 31) which should give everyone a couple more weeks to finalize any almost-working exploits.

GCP VRP Prize

To encourage security researchers to focus on Google Cloud Platform, we initiated the annual GCP VRP Prize in 2019. In March this year, we announced the winners of the 2020 edition of the prize and paid out $313,337 in prizes. Ezequiel Pereira won the top prize of $133,337 for finding an RCE in Google Cloud Deployment Manager. We saw some amazing research on Google Cloud Platform this year too. Stay tuned for the 2021 winners!

Research Grants

Six years ago, the Google VRP launched an experimental Vulnerability Research Grant program to encourage seasoned security researchers to take a detailed and extensive look into the security of Google products and services. And reward them even if there are no vulnerabilities found. Six years later, we are happy to announce that in 2021 we awarded over $200,000 in grants to more than 120 security researchers around the world.

If you are a Google VRP researcher and want to be considered for a Vulnerability Research Grant make sure you opted in on your bughunters profile.

Looking forward

With the launch of the new Bug Hunters portal, we plan to continue improving our platform and listening to you - our researchers - on ways we can improve our platform and Bug Hunter University.

Thank you again for making Google, the Internet, and our users safe and secure! Follow us on @GoogleVRP

Thank you to Adam Bacchus, Dirk Göhmann, Sarah Jacobus, Amy Ressler, Martin Straka, Jan Keller, Jon Bottarini