top of page

03.

Group 8 (1).png
Communication Leadership.png

Chatbot

PHASE 1

2022/03 - 2022/06

TEAM

2 Project Managers
2 Researchers
12 Designers (My role)

PHASE 2

2022/06 - 2023/06

TEAM

1 UX Designer & Researcher (My role)

1 Content strategist 

Group 26.png

THE PROBLEM

How do we design a chatbot for answering prospective students applying for the Comm Lead program?

CONTEXT

On phase1, a team consisted of 16 students executed the project in different roles -project management, research and design.

On phase2, I was hired to continue working on the project with the Academic Advisor (the project's client).

CONTRIBUTION

  • Collaborated with student project managers, researchers and other designers, researched various chatbot products and pitched the product's weakness and strength to the team.

  • Contributed to the research efforts  and designed one main conversation flow.

Team reps from Phase 1 presented the chatbot project at the annual department summit

76350c3d-0127-4000-bdf9-483b4bfd6d22.jpg

Won the design award

SKILLS PRACTICED

Product Research | Conversation Design | User Flows | Chatbot | Human-centered Design Thinking | Competitive Analysis | Card Sorting | Tree Testing | User Testing | Information Architecture | NLP Bot | Project Management 

PREVIEW

未命名設計 (13).png
Ask the bot when you have questions

The frequent asked questions (FAQ) navigation

The FAQs are organized into 5 key categories, allowing students to easily access the answers by selecting the relevant tab.

Screen Shot 2023-01-14 at 22.31.08.png

The chatting function

Users can type their questions into the chat to receive answers.

Screen Shot 2023-01-14 at 22.41.19.png
Preview
Untitled 1.png

​The chatbot backend - How I connected the conversation flows

THE PROBLEM

The transparency of information is a crucial aspect in the school admission process. Students struggle to find information as it is scattered across websites and emails.

 

Additionally, the high volume of emails from prospective students asking similar questions results in a tedious workload for the Academic Advisor.

 

To address these challenges, a chatbot solution can be designed to answer basic questions and guide prospective students to the information they require.

USER ANALYSIS

Understanding prospective students
persona.png
Persona2.png

We created two typical personas representing for international and domestic student in Comm Lead Program.

 

The geographic information was gathered from the Comm Lead program website traffic -  the highest volume of website visitors and the students' quotes and the questions were determined through the user survey.

Comm Lead website analysis (09/2021 - 02/2022)

Prospective students' FAQs

The User survey

User Analysis

STYLE GUIDE

Fostering the team alignment

The visual design and content guide serve as a reference for all, establishing the preferred terminology and tone to use in the bot's responses.

 

The bot should sound professional without being cold, friendly without being too casual, and provide assistance while still empowering the user.

PLANNING

In addition to website analysis, user research, and the creation of a style guide, the team completed the design of the FAQ conversation on the main menu in phase 1.

 

The conversation starts by asking the user if they need help and then determining whether they are an international or domestic student, as this affects the direction of the conversation.

The bot demo on the phase 1

USABILITY AUDITING

Aim at launching in 3 months

I looked back to the user testing done at the end of phase 1, I observed the following comments:

Mask group (6).png

“At the end, I select 'I can't find what I want' but I don't have any menu showed up, I guess that's the end of the conversation. Wish there's option to go back to main menu.”  

Mask group (7).png

“My only recommendation would be to think about adding a ‘back to main menu’ option on some of the prompts. When I wanted to check out a second option, I clicked ‘go back’ but it only took me one step back within the same category. I think it might be helpful to have both?”

Mask group (8).png

“It was really great! One thing I thought might be improved was the timing of "Was this answer helpful?" shows up. While I was reading the answers from the chat bot, this "Was this answer helpful?" showed up and the answers I was reading moved up before I could finish reading them.

Mask group (9).png

“Make the not answer what the full cost of the course is , not per credit - international students don’t know what the US system is when they are joining.” 

Based on these feedback, several problems with the conversation flow were identified, including the inability to return to the main menu, disruptions caused by automatic messages, and a lack of understanding of the needs of international students.

 

Additionally, users suggested improvements such as adding notifications for program activities in follow-up messages and enabling the chat function.

COMPARATIVE STUDY

Comparative Study
Comparing 5 universities' chatbots 

To create an effective main menu information navigation for users, I created a list of useful and primary question types and compared it to the offerings of chatbots designed by other universities.

 

This allowed me to determine the most important information to include.

Screen Shot 2023-01-21 at 17.25.09.png

The chatbot at the New Jersey City University's website

Screen Shot 2023-01-21 at 17.24.23.png
Main takeaways:
Done.png
Strengths: inclusive, career oriented and straightforward

The Comm Lead chatbot provides comprehensive answers for both international and domestic students. It sets itself apart by offering a wide range of career resources information. 

After the usability auditing and comparative study, I decided to overhaul these features:

1. Show the main menu tabs first

2. Enhance the interaction experience

3. Remove the initial question asking students' identity (international or domestic)

Untitled design (7).png

IMPROVEMENNTS

What are the next steps after the phase 1?

1. The default  chatbot status 

A circular widget with a chatbot introduction tooltip is positioned by default, minimizing disruption when not in use.

2. The welcoming message is sent to users when they click the widget

Upon arrival, users are greeted with a welcome message and main menu, allowing for easy navigation to answers.

INFORMATION ARCHITECTURE

Did we perform well on the phase 1?
Card Sorting

In phase 1, user validation and performance evaluation were not conducted. To fill this gap, I conducted the card sorting (10 participants) and tree testing (10 participants).

Screen Shot 2023-01-23 at 19.42.59.png

The FAQ tabs designed on the phase 1

Screen Shot 2023-01-23 at 20.11.00.png

The participants overview of the card sorting

Similarity matrix

I facilitated an open card sorting exercise using 20 name-labelled cards, as this method is more effective in gaining insights into the users' mental model with groups named by the participants themselves.

 

I moderated 3 testing sessions and received 7 unmoderated testing data. During the sorting process, I asked participants to verbalize their thoughts, reasoning, and frustrations.

 

The insights gained from this exercise:
Untitled design (5).png
Listen to the majority choices, leave the edge cases
Untitled design (4).png
Ask follow-up questions
Unfamiliar terms should be avoided
Tree testing

Following the card sorting, I reorganized the main question groups for the chatbot and conducted a tree testing to validate my adjustments.  I designed separate testing tasks for both international and domestic students, considering their differing needs.

Testing tasks for international students

Testing overview of international students group

Outcome

From the tree testing, the new IA received 63% average success score in the international students testing group and 69% average success score in the domestic students group.

 

The testing also allowed me to identify groups that received a 100% success rate and areas that received a 0% success rate, providing insight into areas that need improvement

The specific success scores date examples

A new IA in the chatbot that match to users' needs and mental modal

The new structure in the chatbot

The combination of international and domestic student experiences has streamlined the process and saved time.

 

The main menus now include highly requested questions and answers related to student life, program updates, and visa information. 

USER TESTING

Untitled design (8).png
Iteration
Predict the failure before launch
Group 76334.png
Participants
Overall satisfaction rate
Willingness to use again

By introducing the chatbot to potential users and observing their direct responses, I was able to validate my design decisions and gather further insight into users' expectations, habits and suggestions.

Group 76323.png

Participant 1  09/01/2022

Group 76325 (1).png

Participant 3  09/02/2022

Participant 2  09/02/2022

Participant 4  09/02/2022

PAIN POINTS AND SOLUTIONS

Untitled design (10).png
Pain points
Solutions

1

The access to talk with real people

The preferred method of communication for users remains talking real people.

1

Adding contact information to the auto-reply

Activated by the keywords: "bye" and "thank you", etc. The advisor's email and program's social media links will be sent to users.

2

The instant replies are distracting

The lack of a delay between sending questions and receiving replies interferes with the reading experience.

2

Setting the delay time

Set a delay time to mimic a human-like response from the bot. 

3

Seek for a more engaging navigation experience

The navigation was unengaging and tedious.

3

Multiple "back" routes 

I designed alternative navigation methods in the bot, including presenting related questions to click on, the option to type "hi" or "back" to return to previous pages, etc.

ITERATION

Screen Shot 2023-01-24 at 22.37.52.png
The access to talk with real people

The ending auto-reply is triggered when users need to end the conversation.

 

This auto-reply sends the information to connect the program staff.

Slower replying speed to mimic a human-like conversation

A 0.5 second delay time has been implemented when the chatbot responds, this creates a more personal interaction between the bot and the user.

 

delay.gif
Screen Shot 2023-01-24 at 22.55.54.png
Screen Shot 2023-01-24 at 22.56.27.png
4 different ways to navigate back and forth in the chatbot
Screen Shot 2023-01-24 at 22.56.39.png
  • ​Asking the user if they have further questions

  • Displaying other related questions 

  • Allowing the user to type "hi" or "back" to navigate the chatbot on their own.

  • "Go back" button

REFLECTION

Teamwork or independent work

In the first three months of the project, I worked with a team of 16 people, which was efficient in planning and distribution of work. However, working independently in phase 2 was advantageous for quicker decision-making and impactful results.

Gained user research skills

I honed user research skills in this project using card sorting and tree testing techniques to validate the information architecture. Developed testing trees, assigned user tasks, conducted participant interviews, analyzed results, and extracted findings. 

Ability to translate UX knowledge to non-UX audience

Clear communication of the research processes and the reasoning behind them is essential when presenting results to clients. It helps them understand the value of the invested time. In phase two, I shared the research process and key findings in a clear and concise manner during weekly one-on-one meetings.

Group 7.png
Interested in viewing more projects?
Group 76362.png
Group 76357.png

🔒

bottom of page