Usability Testing

Testing with users

Cycle 1

YouTube Link: Usability Testing Cycle 1

Cycle 2

YouTube Link: Usability Testing Cycle 2

INTRODUCTION

In the realm of Human-Computer Interaction (HCI), usability testing plays a vital role in evaluating the effectiveness, efficiency, and satisfaction with which users accomplish specific tasks using a system. For CEO Finance, a prototype application developed to streamline financial management, usability testing is an essential step. The app aims to simplify tasks such as splitting bills, managing personal finances, and settling bills, making financial management more accessible and efficient for users. Conducting usability testing ensures the app meets user needs and provides a seamless experience.

Usability testing is crucial for understanding how real users interact with CEO Finance and identifying areas for improvement. This process helps pinpoint usability issues, comprehend user behavior, and gather feedback to enhance the design. By observing users completing specific tasks, we can collect valuable insights and ensure the app is intuitive and user-friendly. This iterative process ensures that the final design is optimized for user satisfaction and meets the intended goals of CEO Finance.

METHODS AND PROCEDURES

The aims of the design process and the characteristics of the evaluated mobile application elements are ideally matched to the moderated, in-person, and explorative usability testing methods.

With the use of in-person testing fosters a more personal and engaging atmosphere by facilitating conversation between participants and the moderator. In-person testing allows for additional questioning and rapid settlement of doubts. This testing allows for the discovery of unanticipated problems or possibilities.

There are a total of 5 participants representing the target user demographic were selected to ensure a comprehensive evaluation. The participants are typically individuals with lower incomes, such as students, who frequently hang out with friends, tend to keep track of daily expenses and incomes, and have plans on saving money. For the tasks, participants were assigned a set of tasks representing key functionalities within the mobile application:

    1. Adding a new group to start recording the bills.

    2. Choosing a suitable icon to represent the group type and category of bills.

    3. Splitting bills through equally, amount, and sharing methods.

    4. Recording settlements of balances.

    5. Sending reminders to payers and recipients.

    6. Recording daily expenses and incomes.

Before testing, participants were provided with realistic scenarios to contextualize the tasks. For task 1, the scenario is having dinner with a group of friends, consuming and sharing food together. Then, splitting bills among friends is needed after dinner. For task 2, the scenario is to settle debts and record payments that have been made so that the users will remember to pay recipients and be able to review who owes who money. For task 3, the scenario is that after you have made a purchase, you want to record your expenses. Then, the users have to record and track their expenses after purchasing based on what they paid. The purpose of these scenarios is to provide the tasks in a realistic setting and make sure that the participants understand the beneficial functions of the application they are using. The testing attempts to assess the app's usability and functioning in real-life situations by immersing users in travel with friends, clearing debts, and tracking everyday expenses. This approach assists in discovering any possible problems and improvements that are required for a greater user experience, ensuring that the app effectively fulfills the demands of its target users.

A quantitative metric was employed to assess the usability of the mobile application, which is time-on-task. The objective of time-on-task measurement is to determine the efficiency at which users can complete given app tasks. In general, shorter times reflect a more intuitive and user-friendly design, whereas longer times may indicate that usability and navigation still need improvement. This metric helps further improve the user experience and offers helpful details about the app's overall effectiveness. Moreover, qualitative insights have also been gathered through post-task interviews and open-ended questions. Through the collection of comprehensive user input, including their likes, challenges, and suggestions for development, the problems that a quantitative metric alone might miss can be identified. These insights help identify pain points, understand user behaviour, and capture the emotional aspect of user interactions.

As for the tools used to conduct the testing, one of the designers’ laptops has been used to preview the prototype on Figma. Besides, the participants used one of the designers’ handphones to access and complete the tasks within the mobile application. There are three additional smartphones have been used in order to record the audio and process of usability testing.

Overall procedures:

1. Introduction: Information about the usability test's goal was explained to participants.

2. Pre-task Instructions: To establish expectations and provide context, participants were given explicit instructions and scenarios before each task.

3. Task Execution: Participants worked individually to complete the given tasks while having their interactions noted and documented.

4. Post-Task Interview: Participants discussed their experiences verbally, emphasizing any difficulties, preferences, or recommendations.

5. Debriefing: After hearing about the goal of the usability test, participants were given the chance to ask questions and offer more feedback.

RESULTS

Results (Cycle 1)

Result (Cycle 2)

1. How do you feel about using our app?

Cycle 1:-

• User 1: It is a good app. It includes the combinations of multiple functions. The interface and layout are quite good. The colour used is good also.
• User 2: So far so good. The auto-counting and the reminder are very helpful. Most of the interface can be easily understood, but the main task bar is hard to understand at first.
• User 3: The app consists of many features. It records the debt, lent, expenses, and income, which is very informative.

Cycle 2:-

• User 1: A lot of improvements have been made, particularly with smoother button functionality and a new sliding onboarding. Feeling impressed and find the app much better than their previous money management app,
• User 2: The interface is very detailed, able to create groups, review expenses, and send reminders to remind users to pay debts.
• User 3: Did not know how to start and wasn’t sure what the group icon meant

2. ⁠Do you find any difficulties or navigation that make it difficult to use this app?

Cycle 1:-

• User 1: The backward buttons are difficult to click, and some buttons are too small and difficult to click.
• User 2: No.
• User 3: Not much. Overall, it is ok.

Cycle 2:-

• User 1: No.
• User 2: Not really. Easy to explore and discover.
• User 3: Confused about the section on adding a new group; some buttons are small and make it hard to click.

3. What improvements would you like to see in our app? Would you like to give some suggestions?

Cycle 1:

• User 1: Make the backward button larger and change the phrases in task 2 to be more understood by users.
• User 2: Add some guidelines or instructions to guide the first users.
• User 3: Add some labels on the Task Bar or at the top of the main page for each task.

Cycle 2:-

• User 1: Don't have.
• User 2: Expenses can link with online payment apps such as Touch n’ Go for easy tracking
• User 3: Provide instructions on how to click the group type and display "Please choose the category" to guide users; reduce the text and tick box size on the contact page and enlarge the “CONFIRM” button in Task 1; increase the cross button in Task 2, and increase the “Record, Overview, Budget” button in Task 3.

4. Please rate our app (1-5).

Cycle 1:

• User 1: 4.4 over 5. Overall, it is ok. The design interface is good. It's just that some of the buttons are hard to click, and some are unclickable.
• User 2: 4 over 5. It is not complex to explore. The layout and design are modern and good.
• User 3: 4 over 5. The app is simple and consists of features. Icons are attractive and understandable, except for the main task icons.

Cycle 2:

• Overall, 4 over 5, designs are colorful and beautiful; detailing; great app

IMPROVEMENTS

Based on the two usability testing cycles, we summarized several user recommendations and issues we observed during the testing to improve the app functionality and user experience.

The pictures below show the comparison of the changes.

Cycle 1

Before                                     After

Figure 1: Label of the main bar

Based on the usability testing, users reported difficulty in recognizing icons with their respective tasks. Specifically, the icons alone did not clearly indicate which page or function they represented. To improve clarity and user experience, we are implementing descriptive labels beneath each icon. These short text descriptions will help users easily recognize and understand the purpose of each page, ensuring they know exactly which task they are performing.

Before                                     After

Figure 2: Size of the button

Based on usability testing, most users found it challenging to click the back button and other buttons. To address this, we decided to enlarge the back button and add a circular area around the arrow. This modification makes the button easier to click by expanding the clickable area from just the arrow to the entire circle, improving the user experience compared to the previous prototype.

Figure 3: Onboarding UX

Based on usability testing, most users were unfamiliar with the bill-splitting application. To address this, we introduced an onboarding process before users sign in. This onboarding will help users understand the purpose and concept of CEO Finance and the functions it offers, providing a clearer introduction to the application.

Figure 4: Interactive Calendar

Based on the usability testing, we found that most users want to click and interact with the calendar. Thus, we decided to make the calendar interactive and clickable to enhance the user experience.

Cycle 2

Figure 5: In-App Guidance

Based on usability testing, many users were unsure of their initial steps and unfamiliar with the app. To address this, we introduced an in-app guidance feature that provides step-by-step instructions to help users navigate their journey more easily.

Figure 6: Interactive Notification

 Based on usability testing, we added an interactive notification that appears before users proceed to the next step. This feature enhances the user experience by preventing sudden redirections to other pages after clicking a button and also provides users with a clear alert and confirmation of their action.

Figure 7: Interactive Features

Based on usability testing, we added an interactive keypad and keyboard with clickable numbers to improve the user experience and ensure users feel more engaged with the application. Additionally, we incorporated a toggle and slider to further enhance interactivity and user engagement.

CONCLUSION

Throughout CEO Finance's usability testing phases, iterative improvements refined the application based on user feedback. Insights from Cycle 1 led to significant enhancements: enlarging buttons, improving menu bar text visibility, and adding an onboarding process with in-app tutorials. These changes addressed critical user concerns and substantially enhanced CEO Finance's overall usability.

Feedback from Cycle 2 indicated a positive response to these improvements, with users reporting increased ease of use and efficiency in tasks like bill splitting and financial management. Task success rates and user satisfaction notably improved, validating the effectiveness of these iterative changes. By continually refining based on user feedback, we aim to ensure CEO Finance not only meets but exceeds user expectations.