Smile through the Mask: Emotion Measurement for Stationary Retail

The global pandemic caused by the coronavirus disease (COVID-19) changes the lives of many people all over the world. In the context of stationary retail, a strong change of customer behavior occurs as mandatory safety measures like wearing facemasks and distance regulations have come into place. The sales personnel’s ability to understand and react to customers’ emotions is critical for service interactions and the customers’ overall satisfaction. Unfortunately, facemasks make it difficult to recognize other’s emotions and may lead to misinterpretation and confusion. To address this problem, this paper proposes the design of self-assessment interfaces that offer the customer an easy way to enter their emotions. As part of a Design Science Research (DSR) project, we designed three interfaces and evaluated them over the course of a design cycle. The results indicate that it is possible to use self-assessment technology in stationary retail to measure customer emotions. 196 34TH BLED ECONFERENCE DIGITAL SUPPORT FROM CRISIS TO PROGRESSIVE CHANGE


Introduction
The global pandemic caused by the coronavirus disease  changes the lives of many people all over the world. Besides the physical health issues, COVID-19 evokes negative emotions like fear, sadness and anger (Aslam et al., 2020). In the context of stationary retail, a strong change of customer behavior occurs. Mandatory safety measures like wearing face masks and distance regulations have an influence on the customer (Yang et al., 2021). Stationary retail is a domain that particularly suffers from the grip of the pandemic, because it is already scarred by the digital transformation. The advancing digitalization ensures high sales in e-commerce, but simultaneously poses challenges to stationary retail (Doherty & Ellis-Chadwick, 2010). Since personal interaction with sales personnel is the main advantage of stationary retail, it is crucial to properly support the customer during this challenging time (Otto & Chung, 2000). Customer behavior cannot be explained solely by considering rational aspects (Kahneman, 2003) and is often affected by emotions (van Dolen et al., 2004). The sales personnel's ability to understand and react to the customers' emotions is critical for service interactions and customers' overall satisfaction (Bahadur et al., 2018). To protect oneself and others, the wearing of facemasks is recommended or mandatory, especially in indoor environments. Unfortunately, facemask make it difficult to recognize the emotions of others and may lead to misinterpretation and confusion (Carbon, 2020). To address this problem and support the interaction between customers and sales personnel, this paper proposes the design of self-assessment interfaces that provide a simple way for customers to enter their emotions. Our aim is to determine whether ITsupported self-assessment offers a suitable approach to measure customer emotions in stationary retail. Furthermore, the goal of this contribution is to generate design knowledge in order to provide digital support for stationary retail. Our research follows the Design Science Research (DSR) paradigm (Hevner et al., 2004) and in particular the General Methodology of Design Science Research (Vaishnavi et al., 2015). The research question of this paper is: How can emotion-self-assessment interfaces (ESAI) for stationary retail be designed? The paper is structured as follows: In section 2 and 3, we explain the importance of interaction between customers and sales personnel as well as the relevance of emotions. In section 4, the underlying methodology and the resulting design cycle are explained. Section 5 describes the design of three ESAI based on existing emotion theories, that were later evaluated in a user test. In section 6, the results of the evaluation are presented. Finally, the papers ends with a conclusion.

Customer-Salesperson Interaction
The current difficulties in stationary retail are mainly caused by the digital transformation (Hagberg et al., 2016). Thereby, stationary retail is being threatened by the shift from physical to digital. In particular, e-commerce and the possibility of mobile shopping through smart devices are creating new challenges (Fulgoni, 2015;Reinartz et al., 2019). Although some retailers are able to retain customers through multi-channel strategies, so-called 'internet pure players' account for a large share of the trade (Keyes, 2018). The advantages stationary retail still has are the qualification of the sales personnel and the resulting customer services which can be offered (Otto & Chung, 2000), particularly the option of social interaction with the sales personnel (Gutek et al., 1999). This interaction creates a connection between the sales personnel and the customer within a common interaction space (Fyrberg & Jüriado, 2009), that may contribute to a mutual value creation (Grönroos, 2006). However, the mere existence of an interaction is not enough to ensure value creation (Fyrberg & Jüriado, 2009). An unfitting interaction between the sales personnel and the customer negatively impacts customer satisfaction and salesperson comfort (Groth & Grandey, 2012), whereas a successful interaction can lead to increased trust, stronger loyalty, and improved comfort in future interactions (Geiger et al., 2020b;Grönroos & Voima, 2013). The currently predominant COVID-19 pandemic intensifies the critical situation for stationary retail, not only because stores are oftentimes temporarily closed but because the sales personnel has difficulties to fully recognize the customer's emotions displayed by facial expressions due to facemasks (Adolphs, 2003;Carbon, 2020). In a situation in which the stationary retail is reliant on offering customers good service, this can lead to inappropriate responses. Empathy, care and concern are especially important for ensuring appropriate interactions (Diebner et al., 2020). Therefore, it is vital for the sales personnel to be able to correctly recognize and respond to customer emotions, in order to form the basis for a valuable interaction (Geiger et al., 2020a;Mattila & Enz, 2002;Meyer et al., 2021). ESAI offer the possibility of opening up a common interaction space in which the customer can actively participate in service creation. DIGITAL SUPPORT FROM CRISIS TO PROGRESSIVE CHANGE

Customer Emotions
Emotions influence how people think, communicate and interact, and lead to high mental activities that are perceived as positive or negative (Cabanac, 2002). The digital transformation has already produced various technical options for emotion measurement. In contrast to automatic measurement, self-assessment offers a simple way of measuring emotions that does not require cameras, microphones, or biofeedback readings (Betella & Verschure, 2016;Meyer et al., 2019). Furthermore, emotions are subjectively experienced in different ways (Barrett et al., 2006). Thus, self-assessment provides a suitable way to subjectively assess customer emotions (Barrett et al., 2006;Mau, 2009). There are significant correlations between the customers emotion, behavior and satisfaction (Burns & Neisner, 2006;Martin et al., 2008). Positive emotions are caused by a friendly and pleasant behavior of the sales personnel as well as the negotiation of good prices (Menon & Dubé, 2000). Furthermore, a successful interaction between the sales personnel and the customer evokes positive emotions because social needs are met (Lee & Dubinsky, 2003). Customers with positive emotions show higher satisfaction and improved loyalty (Burns & Neisner, 2006). Negative emotions on the other hand reduce customer satisfaction, which can lead to cancellation of purchases, lasting damage to the customer relationship and negative word-of-mouth (Gelbrich, 2010;Ou & Verhoef, 2017). Negative emotions occur when customers are treated rudely or when they are unsure which product to buy or whether they should by it in the first place (Menon & Dubé, 2000). The sales personnel acts as a critical link between the store and the customer and therefore has a strong influence on the customer's emotions (Lee & Dubinsky, 2003). The empathic ability of the sales personnel can support customer satisfaction, whereas a lack of empathy and the resulting inability to understand the customer's emotion can have a negative impact on the interaction and the perception of the service (Agnihotri & Krush, 2015).

Methodology
Our research project is based on the systematic and iterative DSR paradigm proposed by Hevner et al. (2004) that combines behavioral science and designoriented research, and adds rigor as well as theory to generate prescriptive knowledge about the design of artifacts, such as software, methods, models or concepts (Hevner et al., 2004). Hevner et al. state that in order to create design knowledge, the development, demonstration, justification and evaluation of an artifact is important. This design knowledge covers three fundamental aspects in DSR: (1) the problem space, (2) the solution space and (3) knowledge that describes the effectiveness of the solution through the generated artifact(s), which is called evaluation. The evaluation describes to what extent the constructed novel artifacts (solution space) address the problem space and satisfy the stakeholders of the problem. Our objective is to design ESAI for stationary retail. In this way, we plan to contribute design knowledge (solution space) to address an emotion-based support for the interaction between the customer and the sales personnel (problem space). This covers knowledge on how ESAI can be designed, including expository instantiations as representations of the design knowledge for purposes of testing (Iivari, 2015). We follow a model for comprehensive DSR projects that involve multiple design steps (see Figure 1) (Vaishnavi et al., 2015).

Artifact Design
To approach our research objective, asking whether IT-supported self-assessment offers a suitable approach to measure customer emotions in stationary retail, we designed three different ESAI (see Figure 2). Following the "exploring by building"principle, the design of the ESAI is very diverse and has an explorative character (Vaishnavi et al., 2015). Due to the high diffusion of smartphones among retail customers, all interfaces were implemented in the form of mobile applications (Fulgoni, 2015). The interface design on the left side of Figure 2 is based on the Circumplex Model of Affect (CMoA) (Russell, 1980). The CMoA classifies a variety of emotions using the two dimensions valence and arousal. However, the interface was modified by reducing the complexity of the original. This resulted in four emotional situations. The situations 1 and 2 represent high arousal situations, which are either negative (angry/frustrated) or positive (excited/happy), whereas situations 3 and 4 represent situations of low arousal, which can be negative (sad/tired) or positive (pleased/relaxed). For each emotional situation, an input field was realized in the interface. The emotional situation of the customer can be entered by a tap on one of the input fields. The Affective Slider (AS)-Interface in the middle of Figure 2 was designed close to the Affective Slider by Betella & Verschure (2016). This ESAI measures the two dimensions valence and arousal by adjusting emoticons on sliders. In the AS-Interface, the user is therefore able to choose how "happy" and how "excited" he or she is. The emoticons provide feedback about the status of the slider through their facial expressions. For example, if the emoticon for valence is on the left edge, it appears sad; if it is on the right edge, it appears happy. The slider for excitement behaves accordingly: If the emoticon is on the left, it looks relaxed, if it is on the right, it looks excited. After the appropriate emotional situation of the customer is set, it can be entered via the submit button. Finally the Wheel of Emotions (WoE)-Interface on the right side of Figure 2 is based on the wheel of emotions by Plutchik (2001). Plutchik distinguishes between eight basic emotions which can be expressed in different intensities. Again, the interface was modified by reduction of the complexity. For the design of the interface only the medium intensity emotions (e. g. anger instead of rage, sadness instead of grief) were used, since these fit better with the context of use. To distinguish the eight input fields more clearly, different colors and symbols were applied. To input an emotion, the customer rotates the wheel until the suitable emotion is found, then selects it by tapping on it.

Interface Evaluation
Although designed for mobile applications, the user test was implemented through web applications in a browser window (due to the safety reasons during the COVID-19 pandemic). The user test itself consisted of a short introduction to the topic, the interaction with the three ESAI, and subsequent questioning. In order to avoid a preference of one interface, the three interfaces were presented to the participants in a random order. The evaluation of the ESAI was based on the User Experience Questionnaire (UEQ) respectively its modular extension (UEQ+), which allows to freely select and combine individual user experience scales (Schrepp & Thomaschewski, 2019). For the evaluation of the ESAI, the scales Clarity, Visual Aesthetics, and Efficiency were chosen. All UEQ+ scales used a seven-point Likert scale (1 = totally disagree, 7 = totally agree). Clarity describes the impression of arrangement, structure and visual complexity of a graphical user interface (Otten et al., 2020). Visual Aesthetics measures whether the user perceives the interface as beautiful and appealing (Schrepp & Thomaschewski, 2019). Efficiency measures whether the user has the impression that he or she can achieve the goals related to the usage of the interface with minimal effort (Laugwitz et al., 2008). In addition to the UEQ+ scales, various statements and questions concerning emotions and interactions were formulated and rated on a five-point Likert scale (1 = totally disagree, 5 = totally agree  Table 1 shows the descriptive statistics (mean (M), standard deviation (SD)) of the user test as well as Cronbach´s alphas (α) for the scales Clarity, Visual Aesthetics, and Efficiency.  . For further analysis, the data was tested for normal distribution using the Shapiro-Wilk test. The results indicate non-normal distribution (p < 0.01 for all scales). A non-parametric Friedman test of differences among repeated measures was conducted that retendered a Chi-square value of 55.46, which showed significant differences between the three ESAI (p < 0.01). Post-hoc tests (Dunn-Bonferroni tests) pointed out two significantly differences: First, Visual Aesthetics significantly differs between the AS and the WoM (Z = -1.03, p < 0.01, Cohen's effect size: r = 0.16). Second, Visual Aesthetics significantly differs between the CMoA and the WoE (Z = -0.60, p < 0.05, Cohen's effect size: r = 0.09). The effect sizes correspond to a weak effect.  Tables 2 and 3  participant expressed that he/she would like to give the reason for his/her emotion. In addition, two participant would like to indicate whether contact with sales personnel is desired or not. Moreover, one participant would like to indicate his/her location in the store as well as an automatic recognition of whether he/she is entering or leaving the store. Three participants stated that they would not use an ESAI under any circumstances and would rather keep their emotions to themselves, because far too much customer data is already collected in general. Two participants stated that they would only use an ESAI if they received discounts or coupons. In total, 26 participants would use an ESAI in stationary retail, whereas 18 would not use it or were uncertain (Question 2).

Conclusion & Outlook
The aim of this paper was to answer the question "How can emotion-selfassessment interfaces (ESAI) for stationary retail be designed?". Furthermore, it should be examined whether IT-supported self-assessment offers a suitable approach to measure customer emotions in stationary retail. As part of a DSR project, we designed three ESAI and evaluated them over the course of a design cycle. The user test (n = 44) shows that all ESAI were generally rated as positive (above the midpoint of the used UEQ+ scales) and continued to be perceived as useful by customers (Statement 5). Statistically significant differences could be shown in the evaluation of Visual Aesthetics between the WoM and the CMoA as well as the WoM and the AS. The CMoA and the WoE were rated the highest in all UEQ+ scales and were also perceived to be the best overall (Question 1). The reason for this may be due to the fact that the CMoA and the WoM offer the customer discrete emotions or concrete emotional situations. In contrast, the AS offers a more abstract representation of emotions through its two dimensions. In addition, the CMoA and WoM are characterized by a greater use of colors. The use of colors was highlighted as positive along with the use of emoticons and symbols in the openended questions. Therefore, important design knowledge includes the use of discrete emotions as well as the use of colors and emoticons or other symbols to describe the emotions. In the course of this paper, we were able to show that it is possible to use IT-supported self-assessment in stationary retail to capture customer emotions (Statement 5 & Question 2). Furthermore, the evaluation emphasizes the importance of emotions for a successful interaction between customers and sales personnel (Statement 3) (Adolphs, 2003;Geiger et al., 2020a). Derived from the open questions, it became clear that emotions are a very personal topic. Therefore, special attention should be paid to the secure processing and storage of customer data. The resulting design knowledge should be further refined in the next design cycle. A direct comparison of different design features would be a logical step in this process. In addition, a suitable method must be created to display the customer's emotions to the sales personnel. Furthermore, the open-ended questions offer a variety of additional features besides the actual measurement of the customer's emotions, which will be addressed in further research projects. A strong limitation of this work is its highly exploratory nature, which does not allow for direct comparison of individual design features such as the input method, the number and form of emotions displayed, and the use of colors and symbols. Another limitation derives from the fact that the user test was implemented digitally using browser applications instead of the originally planned -and still intended -mobile applications. The designed ESAIs give the customer the opportunity to actively participate in creating a successful interaction between him and the sales personnel by entering his/her own emotions. In this way, stationary retail is supported in utilizing its key competencies of personal interaction properly. However, the success of the ESAI ultimately depends on whether customers want to participate and whether they are honest.