A summary of my findings while helping a business increase their user acquisition by performing modifications and experiments to their onboarding funnel. A increase of ~15% was seen when synchronous email verification was removed from the flow, this however did come at the cost of a few risks to the business.

TLDR:

Statistics from the onboarding funnel of a Healthcare app I was working on showed that 15% of users that made it to an email verification screen, dropped off. To combat this problem the team brainstormed a wide range of solutions to make the process more streamlined. The solution we decided to go with was to remove the ‘synchronous’ email verification, and add in the ability for users to verify their emails after signing up. This increased the overall onboarding success percentage by ~15%, with no negative impact for users with unverified emails.

The Problem

While developing an app for a healthcare company we noticed a high percentage of users (15%) were being lost on our ‘verify your email’ screen of the onboarding process. This healthcare company was in the start-up phase of its lifecycle and looking to progress into its series A funding round, during this stage it was quite critical for the business to show proof of adoption of its intellectual property. This was mainly measured by two metrics, the first and most of important was the amount of users on the app (user acquisition), and the second was if the users were interacting with the app (user activity/ user retention). Since user acquisition was then the main focus for the business, it was in our interest to make the onboarding funnel to get users onto the app as painless as possible. This ~15% drop off then became a huge focus as it meant 3 out of every 20 users who tried to onboard onto app left and never came back at the email verification stage.

As can be seen from the onboarding funnel breakdown above, there are two clear drop-offs, one right after the first screen if the users are not interested in the app, and one after the verification screen.

The hypothesis was that this screen actively dragged users attention away from the app, causing them to lose focus or interest and simply not complete the onboarding journey. A few case studies were done on ways we could improve this process, ideas such as an onboarding completion stepper, clearer wording, identity federation signup and others were explored. However, we found that the most pragmatic solution was to simply remove email verification from the onboarding process. This was due to the fact that doing this showed the biggest amount of potential gain (~15% more user acquisition) for a medium sized amount of work, whereas all of the other improvements would’ve seen minor gains at best.

Execution (Tech Implementation)

The tech stack for this project consisted of a React-Native frontend with a AWS serverless backend, and our identity service provider was AWS’s Cognito service. The biggest challenge that came with our method was surprisingly not actually the infrastructure required to do an email verification asynchronously. It was how to do it so that none of the production users would experience any downtime or broken accounts.

There are two main functions that Cognito provides that are important to this particular userflow, these functions are SignIn and SignUp.

SignUp allows users to start their account creation within the Cognito service, while a user is in this state they are not regarded as confirmed and cannot be authorised to use the protected lambdas via Appsync.

The SignIn function then completes the Cognito onboarding by requiring the user to confirm with either their phone number or email that they are indeed that user.

Infrastructure:

There are two possible ways that AWS allows you to have a user signup in your cognito userpool and not be ‘verified’:

1.) Configure your Cognito infrastructure to have the autoVerifyAttributes flag set to which attribute you want to verify, either ‘email’ or ‘phone_number’, this will auto-verify this attribute as soon as the user ‘signs up’

2.) Create a OnCognitoPreSignUp trigger which invokes a lambda when a signUp call is triggered, this lambda will run before all of the actual signup logic. In this lambda you can set the user to be verified. This will stop the user needing to be ‘verified’ before proceeding with sign-in.

App Version Considerations:

In this project users could be on a variety of app versions since we restricted the amount of ‘codepush’ changes for app store releases. Due to the fact we had all the apps pointing to a single backend, we needed to keep this in mind as backend and frontend releases were not simultaneous on this project. This is a huge consideration for this kind of project and can very easily lead to broken app versions and downtime for users not on the latest version (something we were trying to avoid).

For this project method 2 was chosen as it allowed us to incrementally do the work to update the app behind the scenes using feature flags (handled by Firebase). Approach 2 also gave us more control on what each version of the app should be seeing, where as approach 1 would have repercussions on all app versions. Due to this decision we were able to do continuous releases of the app, and then trigger the switch from synchronous verification to asynchronous verification without any users having a broken experience.

Changes to the user flow

The frontend changes required for transitioning users from a verified to unverified flow were quite straightforward. Essentially the email verification screen just needed to be hidden and the signup and signin Cognito commands needed to be called from the password screen, as we wanted the user to be signed in for the second half of the onboarding process to access our protected lambdas. These changes were all controlled by feature flags from the frontends perspective.

Standard User Flow:

It is important to note in this flow that the user is actually ‘signed in’ and authenticated with AWS partway through the onboarding flow, this allows the user to access lambdas which return business specific options for the user to select.

Auto-verified User Flow:

All of the magic here happened when Cognitos Signup function was called from the frontend, in the latest version of the app we called this function with clientMetadata, sending a Confirm attribute if the user should be automatically confirmed. This lambda is called before any internal Cognito sign-up logic is run, as can be seen by its invocation being before the sign-up step in the flow. The invocation of this lambda is fairly straightforward, if the feature flag is switched on and the frontend is sending a ‘autoConfirm’ flag, then the lambda verifies the users. This then allows the user to call Cognito’s Sign-in function and become a verified user which allows them to call our protected lambdas.

Risk analysis

The risks of not having a user verify their email during the onboarding can be quite severe if taken advantage of, however most can be mitigated. All of the risk related to this stems from the possibility of the user mistyping their email, if a user gets their email wrong and logs into the app there are a variety of concerns for the business. These ‘unverified’ users could:

• use features that may send emails to them or ask for a password reset, if their email is unverified this can lead to another person (who may not even be aware of the application) getting emails about this user while completely unaware.
  • This can be mitigated by ensuring no marketing emails or sensitive application features can be used by ‘unverified’ users.
• a larger risk is if the ‘unverified’ users mistypes their email such that it lands in the inbox of a person that intends to do harm, this person would be able to verify their email for them if the verification process is done via a ‘magic link’.
• If this attacker is then able to verify their email, they could request a password reset and then completely have access to the users account.
  • To mitigate for this you could either disable password resets or use user information or a code to verify their email.

However, as dire as these risks seem they are quite large edge cases, that in reality, probably wont happen at all, due to this the increased risk was acceptable for the goal of trying to get a smoother user onboarding into the platform. One way the business did choose to decrease the risk even further was by making the user double check the email they typed in was correct through use of a prompt.

Risk Reflection

Even though the risk was mitigated at the time, and it was the most pragmatic solution to take, I am still unsettled by any increasing security risk. To further reduce the risk of a breach or a phoney verification there are some actions that can be taken, at a further time cost for the user. such as:

• Instead of using a magic link that authenticates anyone who has received an email with it, other methods such as a link that asks for a verification code found within the users account can be used.
• Use a link that asks for a crucial bit of information that user has already entered on the app to add a bit of extra security around this email verification process.

Result

Looking at the comparison between conversion rates of the app version before the tech implementation and after (seen below). We can see there is about a 15% improvement on the overall funnel.

Overall the experiment run by this business to decrease the amount of users lost by the onboarding flow was a great success with a 15% increase of signups and no security breaches reported.