Conclusion & Discussion

Reflection

I had been troubled that few participants got to the point of structuring entire effective conversations. A core theory-grounded assumption for this project was by following the conversation framework, participants may be able to create more trustworthy experiences. Because few conversations were able to reach this level of development, I was unable to test if these conversations could be viewed as trustworthy. Therefore, I was unable to validate a connection between conversation design and trust.

In trying to understand why participants were unable to structure effective conversations, I realized I fixated on having teams develop conversations that clearly mapped to the conversation framework. My earlier understanding of effective conversations was the literal “lasting change in the participants” (Dubberly and Pangaro, 2009). I was not admitting several assumptions to myself.

1. My expectations were set on helping people develop effective conversations. In my own unexamined assumption, I was looking for transactional, goal oriented experiences like you would have in commercial services. I forgot to consider that, in a majority of human’s conversations, the goal is for people to learn, coordinate, and collaborate (Dubberly and Pangaro, 2009). While the bots people designed in the workshops did not aspire highly, they absolutely accomplished the goals of conversation.

2. I was seeking the right methods to make cooperative conversation. There are no right methods. These methods are only few of many that could help teams develop chatbots (Stickdorn and Schneider, 2012; Tassi, 2008). These were chosen based on insights on the needs for chatbot design and more broadly conversation design. There are probably more appropriate methods for more specific contexts. “A successful project simply involves finding a workable combination, that can conceptualise, develop, and prototype ideas through an iterative process of gradual improvement” (Stickdorn and Schneider, 2012). In this perspective, the methods were useful for learning about chatbot design.

Changes for the Future

With the above in mind, there are several elements of each method that I would further modify for future use.

On workshop design, the plan was very ambitious for a single evening. Perhaps future workshops can split between personality design and conversation mapping. Some teams may be better helped by focusing on conceptual elements, such as personality design and conversation structuring. Other teams may better helped with practical guidance, such as dialogue design and testing the experience. Alternately, workshops may benefit by having participants recreate existing services. I personally prefer participants developing new services in the moment with a team. However, teams may be helped to focus on the conversation design process than also simultaneously developing a new service.

Bot Personas were very successful in helping teams discern the qualities and goals of their chatbots. One minor issue was the difficulty teams had empathizing with multiple identities simultaneously – between user identity, chatbot identity, and perhaps team members as well. The key to developing empathy is by “interacting with the people you’re designing for” (Hall and Zeldman, 2013). It would be interesting to understand how to better support designers to maintain both perspectives, particularly with greater focus on user research and richer personality design.

I discovered the difficulty in creating the right climate for effective exchange with the improv method. Changing the instruction was less important perhaps than working with specific scenarios (Arvola & Artman, 2006). Additionally, using more constrained role-playing techniques may produce more actionable learnings. For example, rather than having teams explore elements of the chatbot’s personality, instead acting out various ways users may achieve their goal. Or, providing more concrete materials, such as “prompt cards detailing a specific persona, problem, mood, or personal characteristic focus the exercises around specific insights” (Stickdorn and Schneider, 2012).

For conversation mapping, the workshop was not long enough for teams to reflect on conversational structures and to experience solving more complex conversations. In hindsight, I realize conversation mapping needs to be split into two parts – starting with mapping the general structure of the conversation then filling the structure with dialogue. I would like to see these methods tested with more complex experiences, such as commercial chatbots or longer-term projects.

Opportunities for Further Research

This project also revealed several areas that deserve more research and experimentation: the role of on-boarding to develop trust in conversations, maintaining designerly perspective of the chatbot and users, collaborative versus individual design of conversations, visualizing conversations, empathizing with chatbots, chatbot personality design, ethics of chatbots, and so much more.

In Jeff Gothelf’s book Lean UX, he says “remember, these artifacts are a transient part of the project—like a conversation. Get it done. Get it out there. Discuss. Move on” (Gothelf, 2013). This project aimed to develop methods to help teams design conversations for chatbots in order to create more trustworthy experiences. I do not believe this project fully achieved this goal. These methods did, to various degrees, help teams understand how to design conversations for chatbots.

Conclusion

There is still a gap in developing methods for designing conversations. The persona and conversation mapping methods do indicate promise. Within industry, many teams are fixated on the technology rather than human needs. If we want human-centric experiences, then teams need to be educated on how to employ human-centric methods in ways that are responsive to their practice and endeavors. This project used a similar process to develop several methods that could be used to aid in designing of chatbot conversations. There is much more opportunity to explore the application of design thinking methods in the creation of chatbots.

Bibliography

Arvola, M. and Artman, H. (2006) ‘Interaction walkthroughs and improvised role play’, Design and semantics of form and movement, 42.

Dubberly, H. and Pangaro, P. (2009) What is conversation? How can we design for effective conversation?. Available at: http://www.dubberly.com/articles/what-is-conversation.html (Accessed: 4 February 2016).

Gothelf, J. (2013) Lean UX: Applying lean principles to improve user experience. Edited by Josh Seiden. United States: O’Reilly Media, Inc, USA.

Hall, E. and Zeldman, J. (2013) Just enough research. New York: A Book Apart.

Stickdorn, M. and Schneider, J. (2012) This is service design thinking: Basics, tools, cases. United States: John Wiley & Sons.

Tassi, R. (2008) Stakeholder Design Activities. Available at: http://www.servicedesigntools.org/taxonomy/term/17 (Accessed: 2 February 2016).