diff --git a/app/metrics.yaml b/app/metrics.yaml index 7137f5157..202d3face 100644 --- a/app/metrics.yaml +++ b/app/metrics.yaml @@ -18,23 +18,44 @@ events: startups, not just cold startup. Note: There is a short gap between the time application goes into background and the time android reports the application going into the background. - Note: This metric does not cover the following cases: - Case # 1 -> a). open a link(for example, gmail) with in-app - Browser (metric report custom_tab startup) b). press home button - c). open gmail again (which brings us back to in app browser). - Step c will not report startup metric. Case # 2 -> a). open fenix - b). press home button c). launch fenix through app switcher/recent - apps. step c will not report startup type. + Note: This metric does not record souce when app opened from + task switcher: open application -> press home button -> open + recent tasks -> choose fenix. In this case will report + [source = unknown, type = hot, has_saved_instance_state = false]. extra_keys: + type: + description: | + the startup type for opening fenix. the application and HomeActivity + either needs to be created or started again. possible values are + `cold`, `warm`, `hot` or `error`. Error is for impossible cases. + Please file a bug if you see the error case. + app created AND HomeActivity created = cold + app started AND HomeActivity created = warm + app started AND HomeActivity started = hot + app created AND HomeActivity started = error source: description: | The method used to open Fenix. Possible values are `app_icon`, - `custom_tab`, `link` or `unknown` + `custom_tab`, `link` or `unknown`. unknown is for startup sources + where we can't pinpoint the cause. One UNKNOWN case is the app + switcher where we don't know what variables to check to ensure this + startup wasn't caused by something else. + has_saved_instance_state: + description: | + boolean value whether or not startup type has a savedInstance. + using savedInstance, HomeActivity's previous state can be restored. + This is an optional key since it is not applicable to all the cases. + for example, when we are doing a hot start up, we cant have a + savedInstanceState therefore we report only [APP_ICON, HOT] instead + of [APP_ICON, HOT, false]. bugs: - https://github.com/mozilla-mobile/fenix/issues/11830 + - https://github.com/mozilla-mobile/fenix/issues/12573 + - https://github.com/mozilla-mobile/fenix/pull/13494 data_reviews: - https://github.com/mozilla-mobile/fenix/pull/12114#pullrequestreview-445245341 - https://github.com/mozilla-mobile/fenix/pull/13958#issuecomment-676857877 + - https://github.com/mozilla-mobile/fenix/pull/13494#pullrequestreview-474050499 data_sensitivity: - interaction notification_emails: diff --git a/app/src/main/java/org/mozilla/fenix/FenixApplication.kt b/app/src/main/java/org/mozilla/fenix/FenixApplication.kt index a70054518..67c29b552 100644 --- a/app/src/main/java/org/mozilla/fenix/FenixApplication.kt +++ b/app/src/main/java/org/mozilla/fenix/FenixApplication.kt @@ -157,6 +157,8 @@ open class FenixApplication : LocaleAwareApplication(), Provider { // } initVisualCompletenessQueueAndQueueTasks() + + components.appStartupTelemetry.onFenixApplicationOnCreate() } private fun initVisualCompletenessQueueAndQueueTasks() { diff --git a/app/src/main/java/org/mozilla/fenix/HomeActivity.kt b/app/src/main/java/org/mozilla/fenix/HomeActivity.kt index 3aea224cf..bc49ff745 100644 --- a/app/src/main/java/org/mozilla/fenix/HomeActivity.kt +++ b/app/src/main/java/org/mozilla/fenix/HomeActivity.kt @@ -230,13 +230,19 @@ open class HomeActivity : LocaleAwareAppCompatActivity(), NavHostActivity { captureSnapshotTelemetryMetrics() - setAppAllStartTelemetry(intent.toSafeIntent()) + startupTelemetryOnCreateCalled(intent.toSafeIntent(), savedInstanceState != null) StartupTimeline.onActivityCreateEndHome(this) // DO NOT MOVE ANYTHING BELOW HERE. } - protected open fun setAppAllStartTelemetry(safeIntent: SafeIntent) { - components.appAllSourceStartTelemetry.receivedIntentInHomeActivity(safeIntent) + protected open fun startupTelemetryOnCreateCalled(safeIntent: SafeIntent, hasSavedInstanceState: Boolean) { + components.appStartupTelemetry.onHomeActivityOnCreate(safeIntent, hasSavedInstanceState) + } + + override fun onRestart() { + super.onRestart() + + components.appStartupTelemetry.onHomeActivityOnRestart() } @CallSuper @@ -249,6 +255,8 @@ open class HomeActivity : LocaleAwareAppCompatActivity(), NavHostActivity { message = "onResume()" ) + components.appStartupTelemetry.onHomeActivityOnResume() + components.backgroundServices.accountManagerAvailableQueue.runIfReadyOrQueue { lifecycleScope.launch { // Make sure accountManager is initialized. @@ -398,7 +406,7 @@ open class HomeActivity : LocaleAwareAppCompatActivity(), NavHostActivity { .let(::getIntentAllSource) ?.also { components.analytics.metrics.track(Event.AppReceivedIntent(it)) } - setAppAllStartTelemetry(intent.toSafeIntent()) + components.appStartupTelemetry.onHomeActivityOnNewIntent(intent.toSafeIntent()) } /** diff --git a/app/src/main/java/org/mozilla/fenix/components/Components.kt b/app/src/main/java/org/mozilla/fenix/components/Components.kt index a17d6760f..1be2dc64a 100644 --- a/app/src/main/java/org/mozilla/fenix/components/Components.kt +++ b/app/src/main/java/org/mozilla/fenix/components/Components.kt @@ -18,8 +18,7 @@ import mozilla.components.lib.publicsuffixlist.PublicSuffixList import mozilla.components.support.migration.state.MigrationStore import org.mozilla.fenix.BuildConfig import org.mozilla.fenix.HomeActivity -import org.mozilla.fenix.components.metrics.AppAllSourceStartTelemetry -import org.mozilla.fenix.ext.settings +import org.mozilla.fenix.components.metrics.AppStartupTelemetry import org.mozilla.fenix.utils.ClipboardHandler import org.mozilla.fenix.utils.Mockable import org.mozilla.fenix.utils.Settings @@ -84,7 +83,7 @@ class Components(private val context: Context) { } } - val appAllSourceStartTelemetry by lazy { AppAllSourceStartTelemetry(analytics.metrics) } + val appStartupTelemetry by lazy { AppStartupTelemetry(analytics.metrics) } @Suppress("MagicNumber") val addonUpdater by lazy { diff --git a/app/src/main/java/org/mozilla/fenix/components/metrics/AppAllSourceStartTelemetry.kt b/app/src/main/java/org/mozilla/fenix/components/metrics/AppAllSourceStartTelemetry.kt deleted file mode 100644 index ce730d4f1..000000000 --- a/app/src/main/java/org/mozilla/fenix/components/metrics/AppAllSourceStartTelemetry.kt +++ /dev/null @@ -1,59 +0,0 @@ -/* This Source Code Form is subject to the terms of the Mozilla Public - * License, v. 2.0. If a copy of the MPL was not distributed with this - * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ - -package org.mozilla.fenix.components.metrics - -import android.content.Intent -import androidx.annotation.VisibleForTesting -import androidx.annotation.VisibleForTesting.PRIVATE -import androidx.lifecycle.Lifecycle -import androidx.lifecycle.LifecycleObserver -import androidx.lifecycle.OnLifecycleEvent -import androidx.lifecycle.ProcessLifecycleOwner -import mozilla.components.support.utils.SafeIntent - -/** - * Tracks how the application was opened through [Event.AppOpenedAllSourceStartup]. - * We only considered to be "opened" if it received an intent and the app was in the background. - */ -class AppAllSourceStartTelemetry(private val metrics: MetricController) : LifecycleObserver { - - // default value is true to capture the first launch of the application - private var wasApplicationInBackground = true - - init { - ProcessLifecycleOwner.get().lifecycle.addObserver(this) - } - - fun receivedIntentInExternalAppBrowserActivity(safeIntent: SafeIntent) { - setAppOpenedAllSourceFromIntent(safeIntent, true) - } - - fun receivedIntentInHomeActivity(safeIntent: SafeIntent) { - setAppOpenedAllSourceFromIntent(safeIntent, false) - } - - private fun setAppOpenedAllSourceFromIntent(intent: SafeIntent, isExternalAppBrowserActivity: Boolean) { - if (!wasApplicationInBackground) { - return - } - - val source = when { - isExternalAppBrowserActivity -> Event.AppOpenedAllSourceStartup.Source.CUSTOM_TAB - intent.isLauncherIntent -> Event.AppOpenedAllSourceStartup.Source.APP_ICON - intent.action == Intent.ACTION_VIEW -> Event.AppOpenedAllSourceStartup.Source.LINK - else -> Event.AppOpenedAllSourceStartup.Source.UNKNOWN - } - - metrics.track(Event.AppOpenedAllSourceStartup(source)) - - wasApplicationInBackground = false - } - - @OnLifecycleEvent(Lifecycle.Event.ON_STOP) - @VisibleForTesting(otherwise = PRIVATE) - fun onApplicationOnStop() { - wasApplicationInBackground = true - } -} diff --git a/app/src/main/java/org/mozilla/fenix/components/metrics/AppStartupTelemetry.kt b/app/src/main/java/org/mozilla/fenix/components/metrics/AppStartupTelemetry.kt new file mode 100644 index 000000000..d5ab320cc --- /dev/null +++ b/app/src/main/java/org/mozilla/fenix/components/metrics/AppStartupTelemetry.kt @@ -0,0 +1,146 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +package org.mozilla.fenix.components.metrics + +import android.content.Intent +import androidx.annotation.VisibleForTesting +import androidx.lifecycle.Lifecycle +import androidx.lifecycle.LifecycleObserver +import androidx.lifecycle.OnLifecycleEvent +import androidx.lifecycle.ProcessLifecycleOwner +import mozilla.components.support.utils.SafeIntent +import org.mozilla.fenix.components.metrics.Event.AppAllStartup +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Source +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Source.APP_ICON +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Source.CUSTOM_TAB +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Source.LINK +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Source.UNKNOWN +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Type +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Type.ERROR +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Type.HOT +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Type.COLD +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Type.WARM + +/** + * Tracks application startup source, type, and whether or not activity has savedInstance to restore + * the activity from. Sample metric = [source = COLD, type = APP_ICON, hasSavedInstance = false] + * The basic idea is to collect these metrics from different phases of startup through + * [AppAllStartup] and finally report them on Activity's onResume() function. + */ +@Suppress("TooManyFunctions") +class AppStartupTelemetry(private val metrics: MetricController) : LifecycleObserver { + + init { + ProcessLifecycleOwner.get().lifecycle.addObserver(this) + } + + private var isMetricRecordedSinceAppWasForegrounded = false + private var wasAppCreateCalledBeforeActivityCreate = false + + private var onCreateData: AppAllStartup? = null + private var onRestartData: Pair? = null + private var onNewIntentData: Source? = null + + fun onFenixApplicationOnCreate() { + wasAppCreateCalledBeforeActivityCreate = true + } + + fun onHomeActivityOnCreate(safeIntent: SafeIntent, hasSavedInstanceState: Boolean) { + setOnCreateData(safeIntent, hasSavedInstanceState, false) + } + + fun onExternalAppBrowserOnCreate(safeIntent: SafeIntent, hasSavedInstanceState: Boolean) { + setOnCreateData(safeIntent, hasSavedInstanceState, true) + } + + fun onHomeActivityOnRestart() { + // we are not setting [Source] in this method since source is derived from an intent. + // therefore source gets set in onNewIntent(). + onRestartData = Pair(HOT, null) + } + + fun onHomeActivityOnNewIntent(safeIntent: SafeIntent) { + // we are only setting [Source] in this method since source is derived from an intent]. + // other metric fields are set in onRestart() + onNewIntentData = getStartupSourceFromIntent(safeIntent, false) + } + + private fun setOnCreateData( + safeIntent: SafeIntent, + hasSavedInstanceState: Boolean, + isExternalAppBrowserActivity: Boolean + ) { + onCreateData = AppAllStartup( + getStartupSourceFromIntent(safeIntent, isExternalAppBrowserActivity), + getAppStartupType(), + hasSavedInstanceState + ) + wasAppCreateCalledBeforeActivityCreate = false + } + + private fun getAppStartupType(): Type { + return if (wasAppCreateCalledBeforeActivityCreate) COLD else WARM + } + + private fun getStartupSourceFromIntent( + intent: SafeIntent, + isExternalAppBrowserActivity: Boolean + ): Source { + return when { + // since the intent action is same (ACTION_VIEW) for both CUSTOM_TAB and LINK. + // we have to make sure that we are checking for CUSTOM_TAB condition first as this + // check does not rely on intent action + isExternalAppBrowserActivity -> CUSTOM_TAB + intent.isLauncherIntent -> APP_ICON + intent.action == Intent.ACTION_VIEW -> LINK + // one of the unknown case is app switcher, where we go to the recent tasks to launch + // Fenix. + else -> UNKNOWN + } + } + + /** + * The reason we record metric on resume is because we need to wait for onNewIntent(), and + * we are not guaranteed that onNewIntent() will be called before or after onStart() / onRestart(). + * However we are guaranteed onResume() will be called after onNewIntent() and onStart(). Source: + * https://developer.android.com/reference/android/app/Activity#onNewIntent(android.content.Intent) + */ + fun onHomeActivityOnResume() { + recordMetric() + } + + private fun recordMetric() { + if (!isMetricRecordedSinceAppWasForegrounded) { + val appAllStartup: AppAllStartup = if (onCreateData != null) { + onCreateData!! + } else { + mergeOnRestartAndOnNewIntentIntoStartup() + } + metrics.track(appAllStartup) + isMetricRecordedSinceAppWasForegrounded = true + } + // we don't want any weird previous states to persist on our next metric record. + onCreateData = null + onNewIntentData = null + onRestartData = null + } + + private fun mergeOnRestartAndOnNewIntentIntoStartup(): AppAllStartup { + return AppAllStartup( + onNewIntentData ?: UNKNOWN, + onRestartData?.first ?: ERROR, + onRestartData?.second + ) + } + + @OnLifecycleEvent(Lifecycle.Event.ON_STOP) + @VisibleForTesting(otherwise = VisibleForTesting.PRIVATE) + fun onApplicationOnStop() { + // application was backgrounded, we need to record the new metric type if + // application was to come to foreground again. + // Therefore we set the isMetricRecorded flag to false. + isMetricRecordedSinceAppWasForegrounded = false + } +} diff --git a/app/src/main/java/org/mozilla/fenix/components/metrics/Event.kt b/app/src/main/java/org/mozilla/fenix/components/metrics/Event.kt index e9faca46a..1ca03a57c 100644 --- a/app/src/main/java/org/mozilla/fenix/components/metrics/Event.kt +++ b/app/src/main/java/org/mozilla/fenix/components/metrics/Event.kt @@ -316,11 +316,28 @@ sealed class Event { get() = hashMapOf(Events.appOpenedAllStartupKeys.source to source.name) } - data class AppOpenedAllSourceStartup(val source: Source) : Event() { + data class AppAllStartup( + val source: Source, + val type: Type, + val hasSavedInstanceState: Boolean? = null + ) : Event() { enum class Source { APP_ICON, LINK, CUSTOM_TAB, UNKNOWN } + enum class Type { COLD, WARM, HOT, ERROR } override val extras: Map? - get() = hashMapOf(Events.appOpenedAllStartupKeys.source to source.name) + get() { + val extrasMap = hashMapOf( + Events.appOpenedAllStartupKeys.source to source.toString(), + Events.appOpenedAllStartupKeys.type to type.toString() + ) + // we are only sending hasSavedInstanceState whenever we get data from + // activity's oncreate() method. + if (hasSavedInstanceState != null) { + extrasMap[Events.appOpenedAllStartupKeys.hasSavedInstanceState] = + hasSavedInstanceState.toString() + } + return extrasMap + } } data class CollectionSaveButtonPressed(val fromScreen: String) : Event() { diff --git a/app/src/main/java/org/mozilla/fenix/components/metrics/GleanMetricsService.kt b/app/src/main/java/org/mozilla/fenix/components/metrics/GleanMetricsService.kt index 664e4f98a..a89499fce 100644 --- a/app/src/main/java/org/mozilla/fenix/components/metrics/GleanMetricsService.kt +++ b/app/src/main/java/org/mozilla/fenix/components/metrics/GleanMetricsService.kt @@ -106,7 +106,7 @@ private val Event.wrapper: EventWrapper<*>? { Events.appReceivedIntent.record(it) }, { Events.appReceivedIntentKeys.valueOf(it) } ) - is Event.AppOpenedAllSourceStartup -> EventWrapper( + is Event.AppAllStartup -> EventWrapper( { Events.appOpenedAllStartup.record(it) }, { Events.appOpenedAllStartupKeys.valueOf(it) } ) diff --git a/app/src/main/java/org/mozilla/fenix/customtabs/ExternalAppBrowserActivity.kt b/app/src/main/java/org/mozilla/fenix/customtabs/ExternalAppBrowserActivity.kt index 2d99f8bcd..40c64d5bd 100644 --- a/app/src/main/java/org/mozilla/fenix/customtabs/ExternalAppBrowserActivity.kt +++ b/app/src/main/java/org/mozilla/fenix/customtabs/ExternalAppBrowserActivity.kt @@ -35,8 +35,8 @@ open class ExternalAppBrowserActivity : HomeActivity() { final override fun getIntentSessionId(intent: SafeIntent) = intent.getSessionId() - override fun setAppAllStartTelemetry(safeIntent: SafeIntent) { - components.appAllSourceStartTelemetry.receivedIntentInExternalAppBrowserActivity(safeIntent) + override fun startupTelemetryOnCreateCalled(safeIntent: SafeIntent, hasSavedInstanceState: Boolean) { + components.appStartupTelemetry.onExternalAppBrowserOnCreate(safeIntent, hasSavedInstanceState) } override fun getNavDirections( diff --git a/app/src/test/java/org/mozilla/fenix/components/metrics/AppAllSourceStartTelemetryTest.kt b/app/src/test/java/org/mozilla/fenix/components/metrics/AppAllSourceStartTelemetryTest.kt deleted file mode 100644 index f411ae029..000000000 --- a/app/src/test/java/org/mozilla/fenix/components/metrics/AppAllSourceStartTelemetryTest.kt +++ /dev/null @@ -1,95 +0,0 @@ -/* This Source Code Form is subject to the terms of the Mozilla Public - * License, v. 2.0. If a copy of the MPL was not distributed with this - * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ - -package org.mozilla.fenix.components.metrics - -import android.content.Intent -import io.mockk.MockKAnnotations -import io.mockk.every -import io.mockk.impl.annotations.RelaxedMockK -import io.mockk.verify -import mozilla.components.support.utils.toSafeIntent -import org.junit.Before -import org.junit.Test - -class AppAllSourceStartTelemetryTest { - - @RelaxedMockK - private lateinit var metricController: MetricController - - @RelaxedMockK - private lateinit var intent: Intent - - private lateinit var appAllSourceStartTelemetry: AppAllSourceStartTelemetry - - @Before - fun setup() { - MockKAnnotations.init(this) - appAllSourceStartTelemetry = AppAllSourceStartTelemetry(metricController) - } - - @Test - fun `WHEN a main launcher intent is received in HomeActivity THEN an app start metric is recorded from app_icon`() { - - every { intent.action } returns Intent.ACTION_MAIN - every { intent.categories.contains(Intent.CATEGORY_LAUNCHER) } returns true - - appAllSourceStartTelemetry.receivedIntentInHomeActivity(intent.toSafeIntent()) - - val validSource = Event.AppOpenedAllSourceStartup.Source.APP_ICON - verify(exactly = 1) { metricController.track(Event.AppOpenedAllSourceStartup(validSource)) } - } - - @Test - fun `WHEN a VIEW intent is received in HomeActivity THEN an app start metric is recorded from link`() { - every { intent.action } returns Intent.ACTION_VIEW - - appAllSourceStartTelemetry.receivedIntentInHomeActivity(intent.toSafeIntent()) - - val validSource = Event.AppOpenedAllSourceStartup.Source.LINK - verify(exactly = 1) { metricController.track(Event.AppOpenedAllSourceStartup(validSource)) } - } - - @Test - fun `WHEN a intent is received in ExternalAppBrowserActivity THEN an app start metric is recorded from custom_tab`() { - val intent = Intent() - - appAllSourceStartTelemetry.receivedIntentInExternalAppBrowserActivity(intent.toSafeIntent()) - - val validSource = Event.AppOpenedAllSourceStartup.Source.CUSTOM_TAB - verify(exactly = 1) { metricController.track(Event.AppOpenedAllSourceStartup(validSource)) } - } - - @Test - fun `GIVEN an app is in the foreground WHEN an intent is received THEN no startup metric is recorded`() { - appAllSourceStartTelemetry.receivedIntentInHomeActivity(intent.toSafeIntent()) - - appAllSourceStartTelemetry.receivedIntentInHomeActivity(intent.toSafeIntent()) - - verify(exactly = 1) { metricController.track(any()) } - } - - @Test - fun `WHEN application goes in background and comes foreground, THEN an app start metric is recorded`() { - // first startup - appAllSourceStartTelemetry.receivedIntentInHomeActivity(intent.toSafeIntent()) - - // mock application going in the background - appAllSourceStartTelemetry.onApplicationOnStop() - - appAllSourceStartTelemetry.receivedIntentInHomeActivity(intent.toSafeIntent()) - - verify(exactly = 2) { metricController.track(any()) } - } - - @Test - fun `WHEN an intent received in HomeActivity is not launcher or does not have VIEW action, THEN an app start is recorded from unknown`() { - every { intent.action } returns Intent.ACTION_MAIN - - appAllSourceStartTelemetry.receivedIntentInHomeActivity(intent.toSafeIntent()) - - val validSource = Event.AppOpenedAllSourceStartup.Source.UNKNOWN - verify(exactly = 1) { metricController.track(Event.AppOpenedAllSourceStartup(validSource)) } - } -} diff --git a/app/src/test/java/org/mozilla/fenix/components/metrics/AppStartupTelemetryTest.kt b/app/src/test/java/org/mozilla/fenix/components/metrics/AppStartupTelemetryTest.kt new file mode 100644 index 000000000..540fa7402 --- /dev/null +++ b/app/src/test/java/org/mozilla/fenix/components/metrics/AppStartupTelemetryTest.kt @@ -0,0 +1,266 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +package org.mozilla.fenix.components.metrics + +import android.content.Intent +import io.mockk.MockKAnnotations +import io.mockk.clearMocks +import io.mockk.every +import io.mockk.impl.annotations.MockK +import io.mockk.verify +import mozilla.components.support.utils.toSafeIntent +import org.junit.Before +import org.junit.Test +import org.junit.Assert.assertTrue +import org.mozilla.fenix.GleanMetrics.Events +import org.mozilla.fenix.GleanMetrics.Events.appOpenedAllStartupKeys.hasSavedInstanceState +import org.mozilla.fenix.GleanMetrics.Events.appOpenedAllStartupKeys.source +import org.mozilla.fenix.GleanMetrics.Events.appOpenedAllStartupKeys.type +import org.mozilla.fenix.components.metrics.Event.AppAllStartup +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Source +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Source.APP_ICON +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Source.UNKNOWN +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Source.CUSTOM_TAB +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Source.LINK +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Type.COLD +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Type.WARM +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Type.HOT +import org.mozilla.fenix.components.metrics.Event.AppAllStartup.Type.ERROR + +class AppStartupTelemetryTest { + + @MockK + private lateinit var metricController: MetricController + @MockK + private lateinit var intent: Intent + + private lateinit var appStartupTelemetry: AppStartupTelemetry + + @Before + fun setup() { + MockKAnnotations.init(this) + appStartupTelemetry = AppStartupTelemetry(metricController) + every { metricController.track(any()) } returns Unit + } + + @Test + fun `WHEN application is launch for the first time through application icon THEN records the correct values`() { + setupIntentMock(APP_ICON) + + appStartupTelemetry.onFenixApplicationOnCreate() + appStartupTelemetry.onHomeActivityOnCreate(intent.toSafeIntent(), false) + appStartupTelemetry.onHomeActivityOnResume() + + val validMetric = AppAllStartup(APP_ICON, COLD, false) + verify(exactly = 1) { metricController.track(validMetric) } + } + + @Test + fun `WHEN application is launch for the first time through a url link THEN records the correct values`() { + setupIntentMock(LINK) + + appStartupTelemetry.onFenixApplicationOnCreate() + appStartupTelemetry.onHomeActivityOnCreate(intent.toSafeIntent(), false) + appStartupTelemetry.onHomeActivityOnResume() + + val validMetric = AppAllStartup(LINK, COLD, false) + verify(exactly = 1) { metricController.track(validMetric) } + } + + @Test + fun `WHEN application is launch for the first time through an custom tab THEN records the correct values`() { + setupIntentMock(CUSTOM_TAB) + + appStartupTelemetry.onFenixApplicationOnCreate() + appStartupTelemetry.onExternalAppBrowserOnCreate(intent.toSafeIntent(), false) + appStartupTelemetry.onHomeActivityOnResume() + + val validMetric = AppAllStartup(CUSTOM_TAB, COLD, false) + verify(exactly = 1) { metricController.track(validMetric) } + } + + @Test + fun `GIVEN, application exists and is backgrounded, WHEN application is launched again through app icon and HomeActivity is recreated THEN records the correct values`() { + setupIntentMock(APP_ICON) + launchApplicationAndPutApplicationInBackground() + + appStartupTelemetry.onHomeActivityOnCreate(intent.toSafeIntent(), false) + appStartupTelemetry.onHomeActivityOnResume() + + val validMetric = AppAllStartup(APP_ICON, WARM, false) + verify(exactly = 1) { metricController.track(validMetric) } + } + + @Test + fun `GIVEN, application exists and is backgrounded, WHEN application is launched again through url link and HomeActivity is recreated THEN records the correct values`() { + setupIntentMock(LINK) + launchApplicationAndPutApplicationInBackground() + + appStartupTelemetry.onHomeActivityOnCreate(intent.toSafeIntent(), false) + appStartupTelemetry.onHomeActivityOnResume() + + val validMetric = AppAllStartup(LINK, WARM, false) + verify(exactly = 1) { metricController.track(validMetric) } + } + + @Test + fun `GIVEN, application exists and is backgrounded, WHEN application is launched again through custom tab and ExternalAppBrowserActivity is recreated THEN records the correct values`() { + setupIntentMock(CUSTOM_TAB) + launchApplicationAndPutApplicationInBackground() + + appStartupTelemetry.onExternalAppBrowserOnCreate(intent.toSafeIntent(), false) + appStartupTelemetry.onHomeActivityOnResume() + + val validMetric = AppAllStartup(CUSTOM_TAB, WARM, false) + verify(exactly = 1) { metricController.track(validMetric) } + } + + @Test + fun `GIVEN, application exists and is backgrounded, WHEN application is launched again through app icon and HomeActivity is restarted THEN records the correct values`() { + setupIntentMock(APP_ICON) + launchApplicationAndPutApplicationInBackground() + + appStartupTelemetry.onHomeActivityOnRestart() + appStartupTelemetry.onHomeActivityOnNewIntent(intent.toSafeIntent()) + appStartupTelemetry.onHomeActivityOnResume() + + val validMetric = AppAllStartup(APP_ICON, HOT) + verify(exactly = 1) { metricController.track(validMetric) } + } + + @Test + fun `GIVEN, application exists and is backgrounded, WHEN application is launched again through url link and HomeActivity is restarted THEN records the correct values`() { + setupIntentMock(LINK) + launchApplicationAndPutApplicationInBackground() + + appStartupTelemetry.onHomeActivityOnRestart() + appStartupTelemetry.onHomeActivityOnNewIntent(intent.toSafeIntent()) + appStartupTelemetry.onHomeActivityOnResume() + + val validMetric = AppAllStartup(LINK, HOT) + verify(exactly = 1) { metricController.track(validMetric) } + } + + @Test + fun `WHEN application is launched and onResume() is called twice THEN metric is reported only once`() { + setupIntentMock(LINK) + appStartupTelemetry.onExternalAppBrowserOnCreate(intent.toSafeIntent(), false) + appStartupTelemetry.onHomeActivityOnResume() + + appStartupTelemetry.onHomeActivityOnResume() + + verify(exactly = 1) { metricController.track(any()) } + } + + @Test + fun `GIVEN application is in background WHEN application is launched again through unknown source and HomeActivity exists THEN records the correct values`() { + setupIntentMock(UNKNOWN) + launchApplicationAndPutApplicationInBackground() + + appStartupTelemetry.onHomeActivityOnCreate(intent.toSafeIntent(), false) + appStartupTelemetry.onHomeActivityOnResume() + + val validMetric = AppAllStartup(UNKNOWN, WARM, false) + verify(exactly = 1) { metricController.track(validMetric) } + } + + @Test + fun `GIVEN application exists and is backgrounded WHEN application started again through app icon but HomeActivity is recreated from savedInstanceState THEN records the correct values`() { + setupIntentMock(APP_ICON) + launchApplicationAndPutApplicationInBackground() + + appStartupTelemetry.onHomeActivityOnCreate(intent.toSafeIntent(), true) + appStartupTelemetry.onHomeActivityOnResume() + + val validMetric = AppAllStartup(APP_ICON, WARM, true) + verify(exactly = 1) { metricController.track(validMetric) } + } + + private fun launchApplicationAndPutApplicationInBackground() { + appStartupTelemetry.onFenixApplicationOnCreate() + appStartupTelemetry.onHomeActivityOnCreate(intent.toSafeIntent(), false) + appStartupTelemetry.onHomeActivityOnResume() + + // have to clear the mock function calls so it doesnt interfere with tests + clearMocks(metricController, answers = false) + + appStartupTelemetry.onApplicationOnStop() + } + + @Test + fun `GIVEN application is in background WHEN application is launched again HomeActivity only calls onResume THEN records the correct values`() { + setupIntentMock(UNKNOWN) + launchApplicationAndPutApplicationInBackground() + + appStartupTelemetry.onHomeActivityOnResume() + + val validMetric = AppAllStartup(UNKNOWN, ERROR) + verify(exactly = 1) { metricController.track(validMetric) } + } + + @Test + fun `GIVEN application is in background WHEN application is launched again HomeActivity calls onRestart but not onNewIntent THEN records the correct values`() { + setupIntentMock(APP_ICON) + launchApplicationAndPutApplicationInBackground() + + appStartupTelemetry.onHomeActivityOnRestart() + appStartupTelemetry.onHomeActivityOnResume() + + val validMetric = AppAllStartup(UNKNOWN, HOT) + verify(exactly = 1) { metricController.track(validMetric) } + } + + @Test + fun `GIVEN application is in background WHEN application is launched again and HomeActivity calls onNewIntent but not onRestart THEN records the correct values`() { + setupIntentMock(APP_ICON) + launchApplicationAndPutApplicationInBackground() + + appStartupTelemetry.onHomeActivityOnNewIntent(intent.toSafeIntent()) + appStartupTelemetry.onHomeActivityOnResume() + + val validMetric = AppAllStartup(APP_ICON, ERROR) + verify(exactly = 1) { metricController.track(validMetric) } + } + + @Test + fun `WHEN AppAllStartup does not have savedInstanceState THEN do not return savedInstanceState`() { + val expectedExtra: Map? = hashMapOf( + source to APP_ICON.toString(), + type to HOT.toString()) + + val appAllStartup = AppAllStartup(APP_ICON, HOT) + + assertTrue(appAllStartup.extras!! == expectedExtra) + } + + @Test + fun `WHEN AppAllStartup have savedInstanceState THEN return savedInstanceState `() { + val expectedExtra: Map? = hashMapOf( + source to APP_ICON.toString(), + type to COLD.toString(), + hasSavedInstanceState to true.toString()) + + val appAllStartup = AppAllStartup(APP_ICON, COLD, true) + + assertTrue(appAllStartup.extras!! == expectedExtra) + } + + private fun setupIntentMock(source: Source) { + when (source) { + APP_ICON -> { + every { intent.action } returns Intent.ACTION_MAIN + every { intent.categories } returns setOf(Intent.CATEGORY_LAUNCHER) + } + LINK, CUSTOM_TAB -> { + every { intent.action } returns Intent.ACTION_VIEW + every { intent.categories } returns emptySet() + } + UNKNOWN -> { + every { intent.action } returns Intent.ACTION_MAIN + every { intent.categories } returns emptySet() + } + } + } +} diff --git a/docs/metrics.md b/docs/metrics.md index d47b198fe..3f5a30f50 100644 --- a/docs/metrics.md +++ b/docs/metrics.md @@ -99,7 +99,7 @@ The following metrics are added to the ping: | download_notification.try_again |[event](https://mozilla.github.io/glean/book/user/metrics/event.html) |A user tapped on try again when a download fails in the download notification |[1](https://github.com/mozilla-mobile/fenix/pull/6554), [2](https://github.com/mozilla-mobile/fenix/pull/13958#issuecomment-676857877)||2021-04-01 |2 | | error_page.visited_error |[event](https://mozilla.github.io/glean/book/user/metrics/event.html) |A user encountered an error page |[1](https://github.com/mozilla-mobile/fenix/pull/2491#issuecomment-492414486), [2](https://github.com/mozilla-mobile/fenix/pull/13958#issuecomment-676857877)|
  • error_type: The error type of the error page encountered
|2021-04-01 |2 | | events.app_opened |[event](https://mozilla.github.io/glean/book/user/metrics/event.html) |A user opened the app (from cold start, to the homescreen or browser) |[1](https://github.com/mozilla-mobile/fenix/pull/1067#issuecomment-474598673), [2](https://github.com/mozilla-mobile/fenix/pull/13958#issuecomment-676857877)|
  • source: The method used to open Fenix. Possible values are: `app_icon`, `custom_tab` or `link`
|2021-04-01 |2 | -| events.app_opened_all_startup |[event](https://mozilla.github.io/glean/book/user/metrics/event.html) |A user opened the app to the HomeActivity. The HomeActivity encompasses the home screen, browser screen, settings screen, collections and other screens in the nav_graph. This differs from the app_opened probe because it measures all startups, not just cold startup. Note: There is a short gap between the time application goes into background and the time android reports the application going into the background. Note: This metric does not cover the following cases: Case # 1 -> a). open a link(for example, gmail) with in-app Browser (metric report custom_tab startup) b). press home button c). open gmail again (which brings us back to in app browser). Step c will not report startup metric. Case # 2 -> a). open fenix b). press home button c). launch fenix through app switcher/recent apps. step c will not report startup type. |[1](https://github.com/mozilla-mobile/fenix/pull/12114#pullrequestreview-445245341), [2](https://github.com/mozilla-mobile/fenix/pull/13958#issuecomment-676857877)|
  • source: The method used to open Fenix. Possible values are `app_icon`, `custom_tab`, `link` or `unknown`
|2021-06-01 |2 | +| events.app_opened_all_startup |[event](https://mozilla.github.io/glean/book/user/metrics/event.html) |A user opened the app to the HomeActivity. The HomeActivity encompasses the home screen, browser screen, settings screen, collections and other screens in the nav_graph. This differs from the app_opened probe because it measures all startups, not just cold startup. Note: There is a short gap between the time application goes into background and the time android reports the application going into the background. Note: This metric does not record souce when app opened from task switcher: open application -> press home button -> open recent tasks -> choose fenix. In this case will report [source = unknown, type = hot, has_saved_instance_state = false]. |[1](https://github.com/mozilla-mobile/fenix/pull/12114#pullrequestreview-445245341), [2](https://github.com/mozilla-mobile/fenix/pull/13958#issuecomment-676857877), [3](https://github.com/mozilla-mobile/fenix/pull/13494#pullrequestreview-474050499)|
  • has_saved_instance_state: boolean value whether or not startup type has a savedInstance. using savedInstance, HomeActivity's previous state can be restored. This is an optional key since it is not applicable to all the cases. for example, when we are doing a hot start up, we cant have a savedInstanceState therefore we report only [APP_ICON, HOT] instead of [APP_ICON, HOT, false].
  • source: The method used to open Fenix. Possible values are `app_icon`, `custom_tab`, `link` or `unknown`. unknown is for startup sources where we can't pinpoint the cause. One UNKNOWN case is the app switcher where we don't know what variables to check to ensure this startup wasn't caused by something else.
  • type: the startup type for opening fenix. the application and HomeActivity either needs to be created or started again. possible values are `cold`, `warm`, `hot` or `error`. Error is for impossible cases. Please file a bug if you see the error case. app created AND HomeActivity created = cold app started AND HomeActivity created = warm app started AND HomeActivity started = hot app created AND HomeActivity started = error
|2021-06-01 |2 | | events.app_received_intent |[event](https://mozilla.github.io/glean/book/user/metrics/event.html) |The system received an Intent for the HomeActivity. An intent is received an external entity wants to the app to display content. Intents can be received when the app is closed – at which point the app will be opened – or when the app is already opened – at which point the already open app will make changes such as loading a url. This can be used loosely as a heuristic for when the user requested to open the app. The HomeActivity encompasses the home screen and browser screen but may include other screens. This differs from the app_opened probe because it measures all startups, not just cold startup. |[1](https://github.com/mozilla-mobile/fenix/pull/11940/), [2](https://github.com/mozilla-mobile/fenix/pull/13958#issuecomment-676857877)|
  • source: The method used to open Fenix. Possible values are `app_icon`, `custom_tab`, `link` or `unknown`
|2021-06-01 | | | events.browser_menu_action |[event](https://mozilla.github.io/glean/book/user/metrics/event.html) |A browser menu item was tapped |[1](https://github.com/mozilla-mobile/fenix/pull/1214#issue-264756708), [2](https://github.com/mozilla-mobile/fenix/pull/5098#issuecomment-529658996), [3](https://github.com/mozilla-mobile/fenix/pull/6310), [4](https://github.com/mozilla-mobile/fenix/pull/13958#issuecomment-676857877)|
  • item: A string containing the name of the item the user tapped. These items include: Settings, Help, Desktop Site toggle on/off, Find in Page, New Tab, Private Tab, Share, Report Site Issue, Back/Forward button, Reload Button, Quit, Reader Mode On, Reader Mode Off, Open In app, Add To Top Sites, Add-ons Manager, Bookmarks, History
|2021-04-01 |2 | | events.entered_url |[event](https://mozilla.github.io/glean/book/user/metrics/event.html) |A user entered a url |[1](https://github.com/mozilla-mobile/fenix/pull/1067#issuecomment-474598673), [2](https://github.com/mozilla-mobile/fenix/pull/13958#issuecomment-676857877)|
  • autocomplete: A boolean that tells us whether the URL was autofilled by an Autocomplete suggestion
|2021-04-01 |2 |