CUQ (Calculation Tool Chatbot Usability Questionnaire)

Context:
- It can aim to address the unique usability challenges and requirements of chatbot interfaces.
- It can provide a more targeted approach to usability assessment compared to general tools like System Usability Scale (SUS).
- It can be used alongside other metrics for a comprehensive assessment of chatbot usability.
- It can involve both positive and negative aspects of user experience, calculated on a 100-point scale.
- It can have been validated through a study involving 26 participants and three different chatbots.
- It can offer an accessible method for usability assessment, with tools like an Excel spreadsheet for easy score calculation.
- ...
Example(s):
- CUQ v1, by Ulster University.
- ...
Counter-Example(s):
- a General Web Usability Questionnaire.
- A Technical Chatbot Performance Measure.
- ...
See: Chatbot Design, User Experience (UX) Evaluation, Cognitive Ergonomics, Human-Computer Interaction, Usability Testing.

References

https://www.ulster.ac.uk/research/topic/computer-science/artificial-intelligence/projects/cuq
- NOTE: ** Overview The CUQ (Calculation Tool Chatbot Usability Questionnaire) is a new tool designed to measure the usability of chatbots. It was created by an interdisciplinary team at Ulster University. Other chatbot usability metrics exist like ALMA, but CUQ is unique in its focus. Advantages of CUQ Popular metrics like SUS focus on general system usability. CUQ is designed specifically for chatbots. Chatbots have unique design/testing needs, so CUQ may be better than just using SUS. CUQ scores are comparable to SUS and can be used alongside it or other metrics for comprehensive assessment. Using CUQ Administered after usability testing, via paper or online survey. Scores calculated out of 100 like SUS. Positive aspects in odd questions, negative in even. Excel spreadsheet available to easily calculate scores. Validation Validated via a PhD study at Ulster University in 2019. 26 participants tested 3 chatbots of varying quality. Showed construct validity and test-retest reliability. Limitations: small participant sample, only 3 chatbots. More research needed. Published Research Initial CUQ research presented at European Conference on Cognitive Ergonomics 2019. Published conference proceedings paper available - should be cited when using CUQ.

(Ulster University Team, 2019) ⇒ Interdisciplinary Team at Ulster University. (2019). “CUQ: A New Tool for Chatbot Usability Assessment.” In: Proceedings of the European Conference on Cognitive Ergonomics. URL