Project: Email Builder 2.0 MVP
Email Builder 2.0 is Salesfusion's (now SugarMarket) hybrid code and drag + drop email campaign builder that streamlines the campaign creation process for technical and non-technical users.
My role: UX Strategist
Activities: Discovery research, Requirements documentation, Design + development facilitation, Usability testing, Go-to-market planning
What?
Developing, testing, and releasing the MVP of an email campaign build tool.
Challenges:
- No formal or documented testing processes in the company. Planned from scratch.
- Limited budget for testing and marketing.
- Managing simultaneous projects as the sole product manager at the company.
Why?
Sunset outdated features and create enhanced solution that supports a mix of user types.
- There were two older versions of an email builder that existed in the platform: 1) a code editor and a 2) drag + drop builder.
- Both features had a hefty backlog of bugs + improvements
- Based on feedback across the org users wanted the ability to build a drag + drop campaign and use HTML to make detailed edits within the same experience.
Process
Discover
- Utilized existing customer data, surveys, interviews, and competitive research to learn what is important when building an email campaign and what to focus on in the new solution.
- Found two main user types: non-tech savvy email marketing managers who prefer the visual drag + drop and technical email marketing developers who prefer to code + make details edits.
- Learned both user types want the ability to use both drag + drop features and coding features in the same email building experience.
- Developed personas to inform activities throughout the development cycle.
Define
- Developed a matrix documenting user needs, goals, pain points, and 'How Might We' statements.
- Captured key steps in the email building experience in a workflow.
- Worked with stakeholders to align business and user goals.
- Documented requirements and crafted user stories for the MVP.
Ideate
- Worked with our UX Designer to iteratively design wireframes, mockups, and prototypes that brought the requirements to life.
- Gathered feedback from internal + external users on concepts.
- Translated MVP user stories into JIRA tickets for development team.
- Planned + managed sprint cycles, groomed the backlog, led sprint planning, kick-off, status, and retrospective meetings
Validate
Nearing completion of developing the MVP, we wanted to:
- Test the tool’s usability.
- Capture user feedback and ‘critical’ bugs.
- Validate that the user experience is SLC (simple + loveable + correct).
Methods
In-person moderated testing:
- Planned and executed internal testing to catch as many critical bugs as possible before external testing.
- Developed scripts and test scenarios to be utilized.
- Moderated live testing sessions with 7 internal team members.
Remote un-moderated testing:
- Worked the Customer Support and Professional Services teams to develop a Customer Advisory Board and utilized in-app surveys to recruit current users.
- Developed test scenarios for participants to follow.
- Managed email campaign to guide and communicate with the 36 participants on the timeline and any changes/improvements made during testing.
- Utilized FullStory to view users' actions on screen and captured users' feedback via Doorbell.io, a tool that allows users to submit real time feedback within the platform as they tested.
Follow up surveys + interviews:
- Sent follow-up survey to gauge users' level of satisfaction with the solution.
- Interviewed 6 participants (3 who provided positive feedback and 3 who provided negative feedback in the survey) to learn more about their thoughts and feelings while testing the tool.
Metrics
I established the baseline metrics for testing during the Discovery phase. Metrics included:
- Task completion
- Time to build (an email campaign from first to last step)
- # of errors
- OSAT (overall satisfaction)
Findings
- Internal testing allowed us to catch and fix critical bugs before starting external testing.
- External testing confirmed our assumption that users who began building their email in drag + drop switched to the code editor to make final edits.
- After talking with users we found there were features in the previous tools that were missing in Email Builder 2.0 that would keep them from using the tool.
- Testing uncovered some minor and moderate UX/UI issues that were prioritized into future sprints for the next iteration of the tool.
- Users did not want to have to recreate their existing templates in the new builder.
- Time to build metric improved (from baseline) and we learned the streamlined steps made participants feel more confident before deploying their email.
Impact
Deliver
Using competitive analysis, insights from testing, and knowledge from cross-functional teams, I created a go-to-market strategy for the release.
Created external marketing content including interest generating emails, release emails, release notes, training documentation and videos, in-app notifications, webinars, and information for the marketing website that aligned with the company's product vision and positioning as the “all you need marketing automation platform”.
After monitoring metrics over 4 weeks, the time to build metric decreased by 66%.
Recap
Through usability testing, we validated that our hybrid tool solved our customers' pain points in the campaign creation process and also caught critical bugs prior to release. Our design provided users more flexibility when designing an email and decreased the time it took them to build an email and deploy an email campaign.