Prototyping: Getting the design right

Share this post:

(Last of three blog posts in this series)

Sketching, wireframing and prototyping

Showing is always better than telling. Prototypes enable you to test your idea with the customer and fix issues fast before any code is written. People are more likely to invest time in your idea and provide valuable feedback when you show something to them compared to when you only tell them about your idea. Internally, prototypes can help your team align, where words can be interpreted differently. By simulating some of the functionality of a product in a prototype, you may also discover constraints or additional requirements that were not considered before. This assists in making better complexity and time estimates. Start early with small investments and add a higher level of fidelity when you know you’re going in the right direction. Sketching and wireframing are two activities that can bring some great nuances to prototyping.

“The cost to fix a bug found during implementation is 6 times costlier than one identified during design”. Source: IBM Systems Sciences Institute, 2010

Once we worked with prototyping with a client in the pension industry. The task was to figure out how to use and present insights from an AI service. Based on user insights we created 16 drawings addressing different pain points and needs. These drawings were not at all pretty, and that was actually an advantage because it invited the users to edit, mark and add to the ideas. At this stage, you would call the drawings sketches. Sketches are exploring ways to design an experience. At the end of the session after a lot of editing, we had only a few ideas left, that we considered viable for the final design, in this case, a dashboard.

Then we structured the ideas on the sketches into one dashboard with a higher level of detail. This activity is called wireframing. Wireframes are more detailed drawings of the functionalities, structure, and content of the solution. In this example the wireframe allowed us to combine the best parts of the remaining sketches into a structured concept of the final dashboard. It is important to highlight that wireframes can be very simple in terms of visual design and in terms of content. Its main purpose is to show the position, size, and functionality of different parts.

Often people confuse prototypes with high fidelity design that look good and ready to be build in code.
Prototypes can actually be similar to sketches in terms of fidelity but built in a way that allows users to experience and test at least parts of the design. Other times prototypes can be hard to separate from a fully functional and intelligent product.

It is very easy to get confused about the differences between sketches, wireframes, and prototypes. But in general you can say that:

• Sketches are exploring the concept and design options.
• Wireframes are proposed structure, content, and functionalities.
• Prototypes are built to test if a certain design is viable.

In the example with the dashboard for the pension company, we added fidelity in a wireframe from which we could explain and get additional input. Then we built a prototype with code that had working functionalities and designed it similar to the wireframe.

There are two very important things to remember when prototyping. First, it is a bad idea to move to a high fidelity visualization from the beginning even if the client or project manager asks for it. This quickly narrows the space of the solution and takes a lot of time to make and change. Also, people often focus on details such as color and buttons when presented with a high fidelity visualization, whereas you might actually want their feedback on the concept itself, which you might not get when presenting users to what seems to be a “finished” product that looks nice. Secondly, prototypes should be tested with users as early as with a simple paper prototype and as late as a fully functional product to guide the design.

Test early, succeed sooner

“Research shows that just 5 test users are enough to discover 85% of usability problems”.
Source: Nielsen Norman Group

User testing provides us with insights of how a product is actually used in a real-life scenario oppose to a hypothesis of how it should be used. This illuminates pain points and makes it obvious where to improve so we can build better products. If a user is having a problem with the solution it is our problem. When testing ideas, you might feel a sense of failure at times. However, what we generally label as “failure” provides massive learning opportunities, which will eventually lead to new insights, better ideas, and eventual success.

To exemplify how user testing can ensure quality before going to production, we can refer to a client case within the domain of finances, who wanted to enhance customer service. The customers struggled to obtain the right information on the client’s website, and there was a long waiting time at the call center. The solution, in this case, was a virtual assistant that could prevent the pressure on the phones, while guiding the customer to the right information. In this case, it was necessary to be extremely cautious about the abilities of the virtual assistant, so it did not provide the user with false information, that in worst case scenarios might end up in lawsuits.

In order to ensure that the virtual assistant was in line with the purpose and worked as intended, a user test process was initiated. Before launching to the end-users, it was tested in four phases. First, the virtual assistant was tested internally at IBM to secure a decent amount of quality, before the second phase where it was tested internally with the client. The third phase was performed with the actual end-users to really understand potential problems, and not build the solution on our own assumptions. The last test was a technical test, that served the purpose of getting insights in functionalities as well as areas where it struggled to provide the correct answers.

Internal test

The internal test was built on best-practices in chatbot design and was based on seven parameters that are essential to chatbot design. It was executed as a remote test, where selected questions important for this case and founded in the design parameters made the foundation. The test was composed as a survey, that was sent out together with a link to the virtual assistant. In the test, IBM’ers was encouraged to perform some action in the virtual assistant, and then consider the questions. Already in this phase, we obtained great insights into problems, and thereby how to optimize the solution, which was accommodated before testing with the client.

Client test

The client test was performed onsite, as a cooperative evaluation which is similar to the think-aloud method, where you encourage the test participant to speak out loud while performing some predefined tasks in the solution. In this way, you enable the chance to understand the participant’s thoughts behind the actions made.

The purpose of the onsite cooperative evaluation was to test how well the virtual assistant performed within the scope of the pilot. The tasks were founded in end-users intentions e.g. repetitive questions in the call center, in emails, etc. The test setup made it possible to discuss the different actions with the participant, and if complications occurred and a user failed to perform a task, it was possible to ask into specifics of why. Since it was performed with clients (subject matter experts), they had a great amount of knowledge concerning the end-user, and the corrections made based on the findings, ensured that the assistant ready for a test with five actual end-users.

End-user test

The test performed with the end-users had the same setup as the previous test. The purpose was the same, to elucidate whether the virtual assistant performed well within the scope. We performed the test with five participants in their normal environment (see picture below).

The results showed a minimal amount of problems, and of the few problems that occurred, the majority were similar or the same. Because the virtual assistant had been tested thoroughly in the initial phases, both within the scope and broader based on best-practices in chatbot design, most of the problems had been illuminated, and the solution was well-suited to the end-user.

If you did not read the first two blog posts in this series yet, you can find them through the links below:
1/3: UX as an integrated practice in innovation projects
2/3: Design the “right it”, before you design “it right”

If you have any further questions, please do not hesitate to contact one of us at jjuhl-cic@ibm.com or assoerensen-cic@ibm.com