From f272a54d59e25b03a7cb508049e6830fdd7e0862 Mon Sep 17 00:00:00 2001 From: Michael Comella Date: Fri, 20 Dec 2019 11:38:12 -0800 Subject: [PATCH] For #6931: Add Performance Build Variants to README. --- README.md | 50 +++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 49 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index c2c9a66a2..42511f626 100644 --- a/README.md +++ b/README.md @@ -83,7 +83,55 @@ which app id and settings to use. Here is a description of what each means: - **fenixBeta** is a release build with beta signing which uses the org.mozilla.fenix.beta app id for beta releases to Google Play - **fenixProduction** is a release build with release signing which uses the org.mozilla.fenix app id for production releases to Google Play - **fennecProduction** is an experimental build with release signing which uses the org.mozilla.firefox app id and supports upgrading the older Firefox. **WARNING** Pre-production versions of this may result in data loss. -- **forPerformanceTest** is a release build with the debuggable flag set and test activities enabled for running Raptor performance tests +- **forPerformanceTest**: see "Performance Build Variants" below. + +#### Performance Build Variants +For accurate performance measurements, read this section! + +If you want to analyze performance during **local development**: +- Recommendation: use a `forPerformanceTest` variant with local Leanplum, Adjust, & Sentry API keys: contact the front-end perf group for access to them +- Rationale: `forPerformanceTest` is a release variant with `debuggable` set to true. There are numerous performance-impacting differences between debug and release variants so we need a release variant. To profile, we also need debuggable, which is disabled on other release variants. If API keys are not provided, the SDKs may change their behavior in non-trivial ways. +- Caveats: + - debuggable has a non-trivial & variable impact on performance but is needed to take profiles. + - Random experiment opt-in & feature flags may impact performance (see [perf-frontend-issues#45](https://github.com/mozilla-mobile/perf-frontend-issues/issues/45) for mitigation). + +If you want to run **performance tests/benchmarks** in automation or locally: +- Recommendation: production builds. If debuggable is required, use recommendation above but note the caveat above. If your needs are not met, please contact the front-end perf group to identify a new solution. +- Rationale: like the rationale above, we need release variants so the choice is based on the debuggable flag. + +For additional context on these recommendations, see [the perf build variant analysis](https://docs.google.com/document/d/1aW-m0HYncTDDiRz_2x6EjcYkjBpL9SHhhYix13Vil30/edit#). + +You will **need to sign `forPerformanceTest` variants.** For local development, our recommendation is to add the following configuration to `app/build.gradle`: + +```groovy +android { // this line already exists + + signingConfigs { + forPerformanceTest { + storeFile file("/Users//.android/debug.keystore") + storePassword "android" + keyAlias "androiddebugkey" + keyPassword "android" + } + } + + buildTypes { // this line already exists + // ... + + forPerformanceTest releaseTemplate >> { // this line already exists. + // ... + + signingConfig signingConfigs.forPerformanceTest + } + } +} +``` + +Where `` is your OS user name. This sample assumes you are using macOS: if you're on another OS, please update the path appropriately. + +This recommendation will let you use AS just like you do with debug builds but **please do not check in these changes.** + +See [perf-frontend-issues#44](https://github.com/mozilla-mobile/perf-frontend-issues/issues/44) for efforts to make performance signing easier. ## Pre-push hooks To reduce review turn-around time, we'd like all pushes to run tests locally. We'd