From 3ae80cda4082966038890b4e19d8e2d79a10b17f Mon Sep 17 00:00:00 2001 From: Michael Comella Date: Thu, 8 Apr 2021 11:03:00 -0700 Subject: [PATCH] For #18836: note refactor in metrics docs. --- app/metrics.yaml | 9 +++++++++ docs/metrics.md | 6 +++--- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/app/metrics.yaml b/app/metrics.yaml index 926fff6e4..e701ea5e5 100644 --- a/app/metrics.yaml +++ b/app/metrics.yaml @@ -4207,6 +4207,9 @@ perf.startup: The hope is that these cases will not have a significant impact on the end results but, if they appear to, we can replace it with a more complex implementation. +

+ Around April 8, 2021 the implementation was refactored. Functionally, it + should be the same but it's noted just in case there are bugs. bugs: - https://github.com/mozilla-mobile/fenix/issues/18426 data_reviews: @@ -4231,6 +4234,9 @@ perf.startup:

See the `cold_main_app_to_first_frame` probe docs for other possible known issues and more details. +

+ Around April 8, 2021 the implementation was refactored. Functionally, it + should be the same but it's noted just in case there are bugs. bugs: - https://github.com/mozilla-mobile/fenix/issues/18426 data_reviews: @@ -4253,6 +4259,9 @@ perf.startup:

See the `cold_main_app_to_first_frame` probe docs for known issues and more details. +

+ Around April 8, 2021 the implementation was refactored. Functionally, it + should be the same but it's noted just in case there are bugs. bugs: - https://github.com/mozilla-mobile/fenix/issues/18426 data_reviews: diff --git a/docs/metrics.md b/docs/metrics.md index 4dff0d9e6..c9d2d9a99 100644 --- a/docs/metrics.md +++ b/docs/metrics.md @@ -351,9 +351,9 @@ In addition to those built-in metrics, the following metrics are added to the pi | perf.startup.application_on_create |[timing_distribution](https://mozilla.github.io/glean/book/user/metrics/timing_distribution.html) |The duration of `FenixApplication.onCreate` in the main process. This does not measure the duration of migration code (via `MigratingFenixApplication` included in the Beta and Release channels. |[mozilla-mobile/fenix#17973](https://github.com/mozilla-mobile/fenix/pull/17973#issue-572183889)||2021-08-11 |1 | | perf.startup.base_bfragment_on_create_view |[timing_distribution](https://mozilla.github.io/glean/book/user/metrics/timing_distribution.html) |The duration of `BaseBrowserFragment.onCreateView`. |[mozilla-mobile/fenix#18558](https://github.com/mozilla-mobile/fenix/pull/18558#issue-596791848)||2021-08-11 |1 | | perf.startup.base_bfragment_on_view_created |[timing_distribution](https://mozilla.github.io/glean/book/user/metrics/timing_distribution.html) |The duration of `BaseBrowserFragment.onViewCreated`. |[mozilla-mobile/fenix#18558](https://github.com/mozilla-mobile/fenix/pull/18558#issue-596791848)||2021-08-11 |1 | -| perf.startup.cold_main_app_to_first_frame |[timing_distribution](https://mozilla.github.io/glean/book/user/metrics/timing_distribution.html) |The duration from `*Application`'s initializer to the first Android frame being drawn in a [COLD MAIN start up](https://wiki.mozilla.org/index.php?title=Performance/Fenix/Glossary). Notably, this duration omits the time from process start to the initializer (which includes a lengthy dex operation) and the time from the first frame to visual completeness. This probe doesn't measure Custom Tabs or other uses of `ExternalAppBrowserActivity` to simplify result analysis. The methodology for determining this measurement is imperfect to simplify implementation. Issues may include:
- Not measuring Beta and Release channels (due to `MigrationDecisionActivity` interrupting the logic).
- Not distinguishing between MAIN to homescreen, onboarding, session restore, others?
- Not choosing to record a MAIN based on what the user would see and thus the core code path (i.e. the thing we want to measure) but rather on the initial `Intent` state.

The hope is that these cases will not have a significant impact on the end results but, if they appear to, we can replace it with a more complex implementation. |[mozilla-mobile/fenix#18632](https://github.com/mozilla-mobile/fenix/pull/18632#issue-600193452)||2021-08-11 |1 | -| perf.startup.cold_unknwn_app_to_first_frame |[timing_distribution](https://mozilla.github.io/glean/book/user/metrics/timing_distribution.html) |The duration from `*Application`'s initializer to the first Android frame being drawn in a [COLD start up](https://wiki.mozilla.org/index.php?title=Performance/Fenix/Glossary) where we can't say it was a MAIN or VIEW start up. The methodology for determining this measurement is imperfect to simplify implementation.

See the `cold_main_app_to_first_frame` probe docs for known issues and more details. |[mozilla-mobile/fenix#18632](https://github.com/mozilla-mobile/fenix/pull/18632#issue-600193452)||2021-08-11 |1 | -| perf.startup.cold_view_app_to_first_frame |[timing_distribution](https://mozilla.github.io/glean/book/user/metrics/timing_distribution.html) |The duration from `*Application`'s initializer to the first Android frame being drawn in a [COLD VIEW start up](https://wiki.mozilla.org/index.php?title=Performance/Fenix/Glossary). The methodology for determining this measurement is imperfect to simplify implementation. Issues may include:
-Including VIEW intents that aren't valid so take code paths similar to MAIN (this is speculative)

See the `cold_main_app_to_first_frame` probe docs for other possible known issues and more details. |[mozilla-mobile/fenix#18632](https://github.com/mozilla-mobile/fenix/pull/18632#issue-600193452)||2021-08-11 |1 | +| perf.startup.cold_main_app_to_first_frame |[timing_distribution](https://mozilla.github.io/glean/book/user/metrics/timing_distribution.html) |The duration from `*Application`'s initializer to the first Android frame being drawn in a [COLD MAIN start up](https://wiki.mozilla.org/index.php?title=Performance/Fenix/Glossary). Notably, this duration omits the time from process start to the initializer (which includes a lengthy dex operation) and the time from the first frame to visual completeness. This probe doesn't measure Custom Tabs or other uses of `ExternalAppBrowserActivity` to simplify result analysis. The methodology for determining this measurement is imperfect to simplify implementation. Issues may include:
- Not measuring Beta and Release channels (due to `MigrationDecisionActivity` interrupting the logic).
- Not distinguishing between MAIN to homescreen, onboarding, session restore, others?
- Not choosing to record a MAIN based on what the user would see and thus the core code path (i.e. the thing we want to measure) but rather on the initial `Intent` state.

The hope is that these cases will not have a significant impact on the end results but, if they appear to, we can replace it with a more complex implementation.

Around April 8, 2021 the implementation was refactored. Functionally, it should be the same but it's noted just in case there are bugs. |[mozilla-mobile/fenix#18632](https://github.com/mozilla-mobile/fenix/pull/18632#issue-600193452)||2021-08-11 |1 | +| perf.startup.cold_unknwn_app_to_first_frame |[timing_distribution](https://mozilla.github.io/glean/book/user/metrics/timing_distribution.html) |The duration from `*Application`'s initializer to the first Android frame being drawn in a [COLD start up](https://wiki.mozilla.org/index.php?title=Performance/Fenix/Glossary) where we can't say it was a MAIN or VIEW start up. The methodology for determining this measurement is imperfect to simplify implementation.

See the `cold_main_app_to_first_frame` probe docs for known issues and more details.

Around April 8, 2021 the implementation was refactored. Functionally, it should be the same but it's noted just in case there are bugs. |[mozilla-mobile/fenix#18632](https://github.com/mozilla-mobile/fenix/pull/18632#issue-600193452)||2021-08-11 |1 | +| perf.startup.cold_view_app_to_first_frame |[timing_distribution](https://mozilla.github.io/glean/book/user/metrics/timing_distribution.html) |The duration from `*Application`'s initializer to the first Android frame being drawn in a [COLD VIEW start up](https://wiki.mozilla.org/index.php?title=Performance/Fenix/Glossary). The methodology for determining this measurement is imperfect to simplify implementation. Issues may include:
-Including VIEW intents that aren't valid so take code paths similar to MAIN (this is speculative)

See the `cold_main_app_to_first_frame` probe docs for other possible known issues and more details.

Around April 8, 2021 the implementation was refactored. Functionally, it should be the same but it's noted just in case there are bugs. |[mozilla-mobile/fenix#18632](https://github.com/mozilla-mobile/fenix/pull/18632#issue-600193452)||2021-08-11 |1 | | perf.startup.home_activity_on_create |[timing_distribution](https://mozilla.github.io/glean/book/user/metrics/timing_distribution.html) |The duration of `HomeActivity.onCreate`. |[mozilla-mobile/fenix#17973](https://github.com/mozilla-mobile/fenix/pull/17973#issue-572183889)||2021-08-11 |1 | | perf.startup.home_activity_on_start |[timing_distribution](https://mozilla.github.io/glean/book/user/metrics/timing_distribution.html) |The duration of `HomeActivity.onStart`. This may encapsulate `HomeFragment` or `BrowserFragment` creation, depending on the code path, so we expect this to take varying amounts of time. As such, this probe may not be easy to interpret directly but we believe collecting it may give us more information about different patterns we might see in performance data. |[mozilla-mobile/fenix#18558](https://github.com/mozilla-mobile/fenix/pull/18558#issue-596791848)||2021-08-11 |1 | | perf.startup.home_fragment_on_create_view |[timing_distribution](https://mozilla.github.io/glean/book/user/metrics/timing_distribution.html) |The duration of `HomeFragment.onCreateView`. |[mozilla-mobile/fenix#18558](https://github.com/mozilla-mobile/fenix/pull/18558#issue-596791848)||2021-08-11 |1 |