aNewSpring

Product | Research | Agile | 2023 | Netherlands

Designing a Scalable Feature Validation Framework

Implementing a data-driven Design Ops strategy that resulted in a 15% usability lift and eliminated early-stage adoption barriers.

aNewSpring learning platform on a desktop computer

Project Overview

Context

aNewSpring is a technology platform for online training that helps people study and learn more quickly and efficiently.

This project emerged as an OKR in the design team with the aim of evaluating the effectiveness and performance of the features developed by the tech team.

The goal was to analyze the design process within the company's context, identify opportunities for improvement and establish strategic moments for research, testing and validation of new features.

Challenge

  • Design Process Mapping

    Identification of flaws and opportunities for improvement in the internal process.

  • New Validation Process

    Implementation of integrated processes and documentation to track features.

  • Increase Data Collecting

    Ensure improvement results on the platform through all this data.

Solution

An overview of the main solutions and implementations carried out in this project:

  • New Process

  • Documentation

  • Feedback Data

  • Strong Learnings

  • Fast Corrections

  • Platform Improvement

Impacts

Usability of the feature increased by 15% following the implementation of the Validation Process.

This substantial improvement stands in stark contrast to the low utilization we experienced after initially implementing the new validation process.

Onboarding checklist in the aNewSpring platform

My role

  • Mapping Out

    Understanding of the current design process followed by the design team, such as identifying gaps and opportunities.

  • Methods Definition

    Research and definition of the most important metrics and methodologies to be used in the validation process.

  • Tracking Overview

    Define and implement all the steps into the current design process in order to track the feature to get all the important data.

  • Documentation

    Created a document to organise the whole validation process, including strategic definition, metrics, results and action plan.

aNewSpring brand illustration: tailored around the learner

Project Details

Process Mapping

The beginning of this project took place with a detailed understanding of the team's entire design process - from receiving a new task, to the release of the feature already in operation. To achieve this, it was necessary to use different approaches:

  • Observation

    Understand how the team has carried out tasks.

  • Research

    Study of current processes implemented.

  • Feedback

    Collecting feedback and insights from forms.

Current Workflow

Building a detailed organization chart provided a comprehensive view of the routines and steps followed by the design team.

The initial objective was to analyze the current scenario, identifying possible flaws and areas for improvement in the process. Below is a simplified view:

Current design team workflow from product team to release

Pain Points

This first step helped identify what was or wasn't working in the design team's current validation process.

  • Methodologies

    Lack of refined processes to validate different features.

  • Tracking

    Tracking the operation and usability of features.

  • Documentation

    Lack of a document that centralizes feature objectives and goals.

Methodologies

Now that the first visualization of the team's processes was completed and the pain points were identified, I did extensive research into the validation methodologies used by other teams and companies, always prioritizing being aligned with the user experience.

The selection of appropriate methodologies was organized according to the different moments and needs of the process.

Development

During development, rigorous validation methodologies like usability testing and prototyping ensure product quality and user experience alignment before release.

Before
User Interview
Current & New
Personas
Main Audiences
Forms
Quali & Quanti
Stakeholders
What they want/need?
Goals & Outcomes
Clear Definition
Success Criteria
What we are aiming?
After
Results Analyses
Dive into all results
AB & Internal Test
Test different approaches
Technical Goals
How the feature must work
User Test
Clear insights from users
Wireframing
How it looks
Release

Validation methodologies at the release stage focus on ensuring that the product is ready for deployment and meets all specified criteria.

Pre
Adoption
Percentage of users
QA Assurance
Test, adjust & test
Onboarding
Guide the user
Tracking
Track the feature
Post
Retention
Users keep coming back
User Feedback
Get their opinions

New Processes

Once the gaps were identified, a new flow was created highlighting steps to optimize and track each new feature from inception to launch, with a special focus on post-launch adjustments to provide a more complete experience for users.

The proposal was analyzed by the team and management with the intention of only keeping what was most important and valuable for the final process.

New Work Flow

Three new stages were included in the validation process, each designed and structured to address the key validation needs according to the company's reality. We will look at each of them below.

New workflow with validation, tracking and managing stages

1. Validation Plan

At this stage, we seek to identify the target audience, understand their needs and expectations, and delve deeper into the business objectives, aiming to understand the most significant results.

During the development of the functionality, we adopted methodologies that provide detailed guidance on its operability and usability.

The data collected at this stage is then integrated to map the most crucial definitions of functionality, considering both the end user's needs and the company's objectives.

This robust process ensures that functionality is designed and implemented in line with established expectations and goals.

Outcomes

Define all the expected results

Business
Expected impacts
Product
Expected usability
Stakeholders
Expected results
Success Criteria

Choose the best tools to evaluate

Engagement
- User Interaction - Time of usage - Interactions - Sessions per user
Satisfaction
- User feedback - Surveys - NPS & CSAT scores - Reviews
Retention
- Follow up - Performance - Applications
Methodology

Define how to validate

GainSight
Track the feature and get data to measure progress
Surveys
Ask users to measure effectiveness
Satisfaction
Rate user happiness
Documentation

Organise findings and strategies

Audience
Who will use the feature?
Problem
What the audience needs?
Solution
How to solve the problem?
Outcomes
What is expected?
Objectives
The top priorities
Methodology
How to validate?
Conclusions
Findings & Next steps

2. Tracking Plan

Before launching the functionality, it is essential to carry out a round of tracking to refine the delivery and ensure that all elements are operating optimally, aligned with the initial strategy.

This process not only improves the quality of functionality, but also ensures a more robust and effective experience for users, reinforcing our commitment to excellence and customer satisfaction.

GainSight

Setup the tracking tool to get data

Dashboard
Reports and widgets
Audience
User profiles
Accounts
All user details
Analytics
Feature performance
Engagement
User interaction
Onboarding
Guide your audience
Product Mapper
Feature Tree Structure
Segments
Every user click

3. Managing Plan

The availability of functionality to users marks the beginning of a continuous process.

It is essential to regularly analyze tracking results, collect feedback through interactive sessions, user testing and reviews.

If the results do not meet established expectations, we develop an action plan for necessary adjustments, thus restarting the cycle of continuous improvement.

This iterative approach ensures we are always aligned with user needs and delivers an improved experience over time.

Process Data

Analyse all data

Gainsight
Interview
Feedback
QA & Tests
Action Plan

Next Steps

Document
Report
User Test
Survey

Testing the New Validation Plan

With the plan set, we chose one of the platform's features to implement and evaluate the performance of each of the new stages in the feature validation process.

Below is an overview of the most relevant points found during the test:

Bundle Courses feature on a tablet
  • The Feature

    The feature tested was the platform's Bundle Courses function, which consolidates multiple courses into a single location, allowing for more comprehensive management of the entire teaching cycle.

  • Background

    An update was implemented on the Bundle Courses management page to enable mass actions, such as moving large groups of students from one course to another and organizing them separately.

  • Pain Points

    - Feedback continued to report the same difficulties as before the update.

    - Despite being active, many customers did not notice the update and continued to use the feature inefficiently.

  • Solution

    Based on the data collected, we implemented an onboarding process that explained the feature update in greater detail.

  • Results

    Within just 30 days, according to usability tracking data, we observed a 15% increase in the usage of the updated feature.

Impacts

The validation of the new process was applied to the development of the latest platform feature for the Design Team, uncovering valuable insights to boost user engagement.

This process has been integrated into the team workflow, and now every new feature must undergo these steps.

This way, we ensure that everything delivered to the consumer is always aligned with their expectations for usability and efficiency.

Early Understanding

Getting a clear vision of what is being validated and the expected outcomes of the feature is crucial from the start of the project to minimize usability issues on the platform.

Guiding Structure

The structure of the documentation was essential, guiding the process from the beginning. The flowchart provided a comprehensive understanding, identifying gaps.

Validation Methods

With different validation approaches, the team could choose the methodologies that best suited the reality and needs of each project.

Documentation

The documentation increased the visibility of the work, supporting development and providing crucial insights for the entire company.

Portrait of Kaio Parmeggiani
Open to Work

Hello, I am

Kaio Parmeggiani

With over 10 years of experience across Product Design, UX Research and Strategic Planning, I specialize in translating complex problems into intuitive and scalable digital experiences.

I work end-to-end across discovery, interaction design and design systems, collaborating closely with product managers and engineering teams to deliver meaningful product outcomes.

Lets Talk
SAP Fioneer expense management on laptop and phone

Product Design | UX-UI | Figma | Germany

Digitalizing Corporate Financial Operations for Enterprise Scale

Redesigning ad-hoc expense management through an integrated multi-platform ecosystem focused on automation, compliance, and operational efficiency.

Read the Case Study