Do, T. M. T., Blom, J. & Gatica-Perez, D. Smartphone usage in the wild: a large-scale analysis of applications and context. In Proceedings of the 13th International Conference on Multimodal Interfaces, ICMI’ 11, 353–360, https://doi.org/10.1145/2070481.2070550 (Association for Computing Machinery, 2011).
Chhabra, R., Krishna, C. R. & Verma, S. Smartphone based context-aware driver behavior classification using dynamic bayesian network. J. Intell. & Fuzzy Syst. 36, 4399–4412, https://doi.org/10.3233/JIFS-169995 (2019).
Kang, S., Kim, A., Lee, J., Shin, I. & Lee, U. Understanding customers’ interests in the wild. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, UbiComp’ 18, 90–93, https://doi.org/10.1145/3267305.3267625 (Association for Computing Machinery, 2018).
Cha, N. et al. Hello there! is now a good time to talk? opportune moments for proactive interactions with smart speakers. Proc. ACM on Interactive, Mobile, Wearable Ubiquitous Technol. 4, https://doi.org/10.1145/3411810 (2020).
Kim, A., Choi, W., Park, J., Kim, K. & Lee, U. Interrupting drivers for interactions: predicting opportune moments for in-vehicle proactive auditory-verbal tasks. Proc. ACM on Interactive, Mobile, Wearable Ubiquitous Technol. 2, https://doi.org/10.1145/3287053 (2018).
Sağbaş, E. A., Korukoglu, S. & Balli, S. Stress detection via keyboard typing behaviors by using smartphone sensors and machine learning techniques. J. Med. Syst. 44, 1–12, https://doi.org/10.1007/s10916-020-1530-z (2020).
Zhang, X., Li, W., Chen, X. & Lu, S. MoodExplorer: towards compound emotion detection via smartphone sensing. Proc. ACM on Interactive, Mobile, Wearable Ubiquitous Technol. 1, https://doi.org/10.1145/3161414 (2018).
Canzian, L. &Musolesi,M. Trajectories of depression: unobtrusive monitoring of depressive states by means of smartphone mobility traces analysis. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp’ 15, 1293–1304, https://doi.org/10.1145/2750858.2805845 (Association for Computing Machinery, 2015).
Harari, G. M. et al. Using smartphones to collect behavioral data in psychological science: opportunities, practical considerations, and challenges. Perspectives on Psychol. Sci. 11, 838–854, https://doi.org/10.1177/1745691616650285 (2016).
Burkhardt, F. et al. A database of german emotional speech. In Interspeech 5, 1517–1520, https://doi.org/10.21437/INTERSPEECH.2005-446 (2005).
Haq, S., Jackson, P. J. B. & Edge, J. D. Audio-visual feature selection and reduction for emotion classification. Proceedings of International Conference on Auditory-Visual Speech Processing, AVSP’ 08, 185–190 (2008).
Soleymani, M., Lichtenauer, J., Pun, T. & Pantic, M. A multimodal database for affect recognition and implicit tagging. IEEE Transactions on Affect. Comput. 3, 42–55, https://doi.org/10.1109/T-AFFC.2011.25 (2012).
Koelstra, S. et al. DEAP: a database for emotion analysis using physiological signals. IEEE Transactions on Affect. Comput. 3, 18–31, https://doi.org/10.1109/T-AFFC.2011.15 (2012).
Zheng, W.-L. & Lu, B.-L. Investigating critical frequency bands and channels for eeg-based emotion recognition with deep neural networks. IEEE Transactions on Auton. Mental Dev. 7, 162–175, https://doi.org/10.1109/TAMD.2015.2431497 (2015).
Abadi, M. K. et al. DECAF: meg-based multimodal database for decoding affective physiological responses. IEEE Transactions on Affect. Comput. 6, 209–222, https://doi.org/10.1109/TAFFC.2015.2392932 (2015).
Katsigiannis, S. & Ramzan, N. DREAMER: a database for emotion recognition through eeg and ecg signals from wireless low-cost off-the-shelf devices. IEEE J. Biomed. Heal. Informatics 22, 98–107, https://doi.org/10.1109/JBHI.2017.2688239 (2018).
Schmidt, P., Reiss, A., Duerichen, R., Marberger, C. & Van Laerhoven, K. Introducing WESAD, a multimodal dataset for wearable stress and affect detection. In Proceedings of the 20th ACM International Conference on Multimodal Interaction, ICMI’ 18, 400–408, https://doi.org/10.1145/3242969.3242985 (Association for Computing Machinery, 2018).
Zheng, W.-L., Liu, W., Lu, Y., Lu, B.-L. & Cichocki, A. Emotionmeter: a multimodal framework for recognizing human emotions. IEEE Transactions on Cybern. 49, 1110–1122, https://doi.org/10.1109/TCYB.2018.2797176 (2019).
Li, T.-H., Liu, W., Zheng, W.-L. & Lu, B.-L. Classification of five emotions from eeg and eye movement signals: discrimination ability and stability over time. In 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER), 607–610, https://doi.org/10.1109/NER.2019.8716943 (2019).
Miranda-Correa, J. A., Abadi, M. K., Sebe, N. & Patras, I. AMIGOS: a dataset for affect, personality and mood research on individuals and groups. IEEE Transactions on Affect. Comput. 12, 479–493, https://doi.org/10.1109/TAFFC.2018.2884461 (2021).
Hovsepian, K. et al. cStress: towards a gold standard for continuous stress assessment in the mobile environment. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp’ 15, 493–504, https://doi.org/10.1145/2750858.2807526 (Association for Computing Machinery, 2015).
King, Z. D. et al. Micro-Stress EMA: a passive sensing framework for detecting in-the-wild stress in pregnant mothers. Proc. ACM on Interactive, Mobile, Wearable Ubiquitous Technol. 3, https://doi.org/10.1145/3351249 (2019).
Park, C. Y. et al. K-EmoCon, a multimodal sensor dataset for continuous emotion recognition in naturalistic conversations. Sci. Data 7, 1–16, https://doi.org/10.1038/s41597-020-00630-y (2020).
Hektner, J. M., Schmidt, J. A. & Csikszentmihalyi, M. Experience sampling method: measuring the quality of everyday life (Sage Publications, Inc, 2006).
Wang, R. et al. StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp’ 14, 3–14, https://doi.org/10.1145/2632048.2632054 (Association for Computing Machinery, 2014).
Mattingly, S. M. et al. The Tesserae project: large-scale, longitudinal, in situ, multimodal sensing of information workers. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, CHI EA’ 19, 1–8, https://doi.org/10.1145/3290607.3299041 (Association for Computing Machinery, 2019).
John, O. P., Donahue, E. M. & Kentle, R. L. Big five inventory. J. Pers. Soc. Psychol. https://doi.org/10.1037/t07550-000 (1991).
Cobb-Clark, D. A. & Schurer, S. The stability of big-five personality traits. Econ. Lett. 115, 11–15, https://doi.org/10.1016/j.econlet.2011.11.015 (2012).
Kim, J.-H., Kim, B.-H. & Ha, M.-S. Validation of a korean version of the big five inventory. J. Hum. Underst. Couns. 32, 47–65, https://kiss.kstudy.com/Detail/Ar?key=2980444 (2011).
Scollon, C., Kim-Prieto, C. & Diener, E. Experience sampling: promises and pitfalls, strengths and weaknesses. J. Happiness Stud. 39, 157–180, https://doi.org/10.1007/978-90-481-2354-4_8 (2009).
Eisele, G., Vachon, H., Myin-Germeys, I. & Viechtbauer, W. Reported affect changes as a function of response delay: findings from a pooled dataset of nine experience sampling studies. Front. Psychol. 12, https://doi.org/10.3389/fpsyg.2021.580684 (2021).
Watson, D. & Clark, L. A. The PANAS-x: manual for the positive and negative affect schedule – expanded form. Tech. Rep., The University of Iowa. https://doi.org/10.17077/48vt-m4t2 (1994).
Russell, J. A. A circumplex model of affect. J. personality social psychology 39, 1161–1178, https://doi.org/10.1037/h0077714 (1980).
LiKamWa, R., Liu, Y., Lane, N. D. & Zhong, L. MoodScope: building a mood sensor from smartphone usage patterns. In Proceeding of the 11th annual international conference on Mobile systems, applications, and services, MobiSys’ 13, 389–402, https://doi.org/10.1145/2462456.2464449 (Association for Computing Machinery, 2013).
Mehrotra, A., Tsapeli, F., Hendley, R. & Musolesi, M. MyTraces: investigating correlation and causation between users’ emotional states and mobile phone interaction. Proc. ACM on Interactive, Mobile, Wearable Ubiquitous Technol. 1, https://doi.org/10.1145/3130948 (2017).
Cohen, S. Perceived stress in a probability sample of the united states. The social psychology health 31–67 (1988).
Schmidt, P., Durichen, R., Reiss, A., Van Laerhoven, K. & Plotz, T. Multi-target affect detection in the wild: an exploratory study. In Proceedings of the 23rd International Symposium on Wearable Computers, ISWC’ 19, 211–219, https://doi.org/10.1145/3341163.3347741 (Association for Computing Machinery, 2019).
Mark, G., Iqbal, S., Czerwinski, M. & Johns, P. Focused, aroused, but so distractible: temporal perspectives on multitasking and communications. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, CSCW’ 15, 903–916, https://doi.org/10.1145/2675133.2675221 (Association for Computing Machinery, 2015).
Mark, G., Iqbal, S. T., Czerwinski, M. & Johns, P. Bored mondays and focused afternoons: the rhythm of attention and online activity in the workplace. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’ 14, 3025–3034, https://doi.org/10.1145/2556288.2557204 (Association for Computing Machinery, 2014).
Pielot, M., Dingler, T., Pedro, J. S. & Oliver, N. When attention is not scarce – detecting boredom from mobile phone usage. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp’ 15, 825–836, https://doi.org/10.1145/2750858.2804252 (Association for Computing Machinery, 2015).
Choi, W., Park, S., Kim, D., Lim, Y.-K. & Lee, U. Multi-stage receptivity model for mobile just-in-time health intervention. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, https://doi.org/10.1145/3328910 (2019).
Turner, L. D., Allen, S. M. & Whitaker, R. M. Interruptibility prediction for ubiquitous systems: conventions and new directions from a growing field. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp’ 15, 801–812, https://doi.org/10.1145/2750858.2807514 (Association for Computing Machinery, 2015).
Mark, G., Gudith, D. & Klocke, U. The cost of interrupted work: more speed and stress. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’ 08, 107–110, https://doi.org/10.1145/1357054.1357072 (Association for Computing Machinery, 2008).
Bailey, B. P. & Konstan, J. A. On the need for attention-aware systems: measuring effects of interruption on task performance, error rate, and affective state. Comput. Hum. Behav. 22, 685–708, https://doi.org/10.1016/j.chb.2005.12.009 (2006).
Bailey, B. P., Konstan, J. A. & Carlis, J. V. The effects of interruptions on task performance, annoyance, and anxiety in the user interface. IFIP TC13 International Conference on Human-Computer Interaction 1, 593–601 (2001).
Park, S.-J., Choi, H.-R., Choi, J.-H., Kim, K.-W. & Hong, J.-P. Reliability and validity of the korean version of the patient health questionnaire-9 (phq-9). Anxiety mood 6, 119–124,https://koreascience.kr/article/JAKO201025247234261.page (2010).
Kroenke, K., Spitzer, R. L. & Williams, J. B. W. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Medicine 16, 606–613, https://doi.org/10.1046/j.1525-1497.2001.016009606.x (2001).
Lee, J. et al. The reliability and validity studies of the korean version of the perceived stress scale. Korean J. Psychosom. Medicine 20, 127–134, https://www.koreamed.org/SearchBasic.php?RID=1985570 (2012).
Goldberg, D. P. & Hillier, V. F. A scaled version of the general health questionnaire. Psychol. Medicine 9, 139–145, https://doi.org/10.1017/S0033291700021644 (1979).
Park, J.-I., Kim, Y. J. & Cho, M. J. Factor structure of the 12-item general health questionnaire in the korean general adult population. J. Korean Neuropsychiatr. Assoc. 51, 178–184, https://doi.org/10.4306/jknpa.2012.51.4.178 (2012).
Kang, S. et al. K-emophone, a mobile and wearable dataset with in-situ emotion, stress, and attention labels. Zenodo https://doi.org/10.5281/zenodo.7606611 (2022).
Bakdash, J. Z. & Marusich, L. R. Repeated measures correlation. Front. Psychol. 8, https://doi.org/10.3389/fpsyg.2017.00456 (2017).
Hastie, T., Tibshirani, R. & Friedman, J. The elements of statistical learning: data mining, inference, and prediction, vol. 2 (Springer Science & Business Media, 2009).
Hughes, G. F. On the mean accuracy of statistical pattern recognizers. IEEE Transactions on Inf. Theory 14, 55–63, https://doi.org/10.1109/TIT.1968.1054102 (1968).
Breiman, L. Random forests. Mach. Learn. 45, 5–32, https://doi.org/10.1023/A:1010933404324 (2001).
Chen, T. & Guestrin, C. Xgboost: a scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’ 16, 785–794, https://doi.org/10.1145/2939672.2939785 (Association for Computing Machinery, 2016).
Pielot, M. et al. Beyond interruptibility: predicting opportune moments to engage mobile phone users. Proc. ACM on Interactive, Mobile, Wearable Ubiquitous Technol. 1, https://doi.org/10.1145/3130956 (2017).
Sano, A., Johns, P. & Czerwinski, M. Designing opportune stress intervention delivery timing using multi-modal data. In 2017 Seventh International Conference on Affective Computing and Intelligent Interaction, ACII, 346–353, https://doi.org/10.1109/acii.2017.8273623 (IEEE, 2017).
Kunzler, F. et al. Exploring the state-of-receptivity for mhealth interventions. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, https://doi.org/10.1145/3369805 (2020).
Elhai, J. D. et al. Depression and emotion regulation predict objective smartphone use measured over one week. Pers. Individ. Differ. 133, 21–28, https://doi.org/10.1016/j.paid.2017.04.051 (2018).
Yuan, F., Gao, X. & Lindqvist, J. How busy are you? predicting the interruptibility intensity of mobile users. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI’ 17, 5346–5360, https://doi.org/10.1145/3025453.3025946 (Association for Computing Machinery, 2017).
Sano, A. et al. Recognizing academic performance, sleep quality, stress level, and mental health using personality traits, wearable sensors and mobile phones. In 2015 IEEE 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN), 1–6, https://doi.org/10.1109/bsn.2015.7299420 (IEEE, 2015).
Kang, S., Park, C. Y., Kim, A., Cha, N. & Lee, U. Understanding emotion changes in mobile experience sampling. In CHI Conference on Human Factors in Computing Systems, https://doi.org/10.1145/3491102.3501944 (Association for Computing Machinery, 2022).
Ekman, P. An argument for basic emotions. Cogn. Emot. 6, 169–200, https://doi.org/10.1080/02699939208411068 (1992).
Bradley, M. M. & Lang, P. J. Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25, 49–59, https://doi.org/10.1016/0005-7916(94)90063-9 (1994).
Pollak, J. P., Adams, P. & Gay, G. PAM: a photographic affect meter for frequent, in situ measurement of affect. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’ 11, 725–734, https://doi.org/10.1145/1978942.1979047 (Association for Computing Machinery, 2011).
Russell, D. W. UCLA loneliness scale (version 3): reliability, validity, and factor structure. J. Pers. Assess. 66, 20–40, https://doi.org/10.1207/s15327752jpa6601_2 (1996).
Diener, E. et al. New well-being measures: short scales to assess flourishing and positive and negative feelings. Soc. Indic. Res. 97, 143–156, https://doi.org/10.1007/s11205-009-9493-y (2010).
Bogomolov, A., Lepri, B., Ferron, M., Pianesi, F. & Pentland, A. Daily stress recognition from mobile phone data, weather conditions and individual traits. In Proceedings of the 22nd ACM international conference on Multimedia, MM’ 14, 477–486, https://doi.org/10.1145/2647868.2654933 (Association for Computing Machinery, 2014).
Plarre, K. et al. Continuous inference of psychological stress from sensory measurements collected in the natural environment. Proceedings of 10th ACM/IEEE Internatinal Conference on Information Processing in Sensor Networks, IPSN’ 11, 97–108 (2011).
Exler, A., Schankin, A., Klebsattel, C. & Beigl, M. A wearable system for mood assessment considering smartphone features and data from mobile ECGs. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, UbiComp’ 16, 1153–1161, https://doi.org/10.1145/2968219.2968302 (Association for Computing Machinery, 2016).
Wilhelm, P. & Schoebi, D. Assessing mood in daily life: structural validity, sensitivity to change, and reliability of a short-scale to measure three basic dimensions of mood. Eur. J. Psychol. Assess. 23, 258–267, https://doi.org/10.1027/1015-5759.23.4.258 (2007).
Watson, D., Clark, L. A. & Tellegen, A. Development and validation of brief measures of positive and negative affect: the panas scales. J. personality social psychology 54, 1063–1070, https://doi.org/10.1037/0022-3514.54.6.1063 (1988).
Helton, W. S. & Naswall, K. Short stress state questionnaire: factor structure and state change assessment. Eur. J. Psychol. Assess. 31, 20–30, https://doi.org/10.1027/1015-5759/a000200 (2015).
Spielberger, C., Gorsuch, R., Lushene, R., Vagg, P. & Jacobs, G. Manual for the state-trait anxiety inventory, vol. 4 (Palo Alto, CA: Consulting Psychologists Press, 1983).
Buysse, D. J., Reynolds, C. F., Monk, T. H., Berman, S. R. & Kupfer, D. J. The pittsburgh sleep quality index: a new instrument for psychiatric practice and research. Psychiatry Res. 28, 193–213, https://doi.org/10.1016/0165-1781(89)90047-4 (1989).
Davey, H. M., Barratt, A. L., Butow, P. N. & Deeks, J. J. A one-item question with a likert or visual analog scale adequately measured current anxiety. J. Clin. Epidemiol. 60, 356–360, https://doi.org/10.1016/j.jclinepi.2006.07.015 (2007).
Williams, L. J. & Anderson, S. E. Job satisfaction and organizational commitment as predictors of organizational citizenship and in-role behaviors. J. Manag. 17, 601–617, https://doi.org/10.1177/014920639101700305 (1991).
Griffin, M. A., Neal, A. & Parker, S. K. A new model of work role performance: positive behavior in uncertain and interdependent contexts. Acad. Manag. J. 50, 327–347, https://doi.org/10.5465/amj.2007.24634438 (2007).
Fox, S., Spector, P. E., Goh, A., Bruursema, K. & Kessler, S. R. The deviant citizen: measuring potential positive relations between counterproductive work behaviour and organizational citizenship behaviour. J. Occup. Organ. Psychol. 85, 199–220, https://doi.org/10.1111/j.2044-8325.2011.02032.x (2012).
Bennett, R. J. & Robinson, S. L. Development of a measure of workplace deviance. J. applied psychology 85, 349, https://doi.org/10.1037/0021-9010.85.3.349 (2000).
Saunders, J. B., Aasland, O. G., Babor, T. F., Fuente, J. R. D. L. & Grant, M. Development of the alcohol use disorders identification test (audit): who collaborative project on early detection of persons with harmful alcohol consumption-ii. Addict. 88, 791–804, https://doi.org/10.1111/j.1360-0443.1993.tb02093.x (1993).
Craig, C. L. et al. International physical activity questionnaire: 12-country reliability and validity. Medicine & Sci. Sports & Exerc. 35, 1381–1395, https://doi.org/10.1249/01.MSS.0000078924.61453.FB (2003).
Palipudi, K. M. et al. Methodology of the global adult tobacco survey — 2008–2010. Glob. Heal. Promot. 23, 3–23, https://doi.org/10.1177/1757975913499800 (2016).
Kaya, F., Delen, E. & Bulut, O. Test review: shipley-2 manual. J. Psychoeduc. Assess. 30, 593–597, https://doi.org/10.1177/0734282912440852 (2012).
Ocumpaugh, J., Baker, R. S. & Rodrigo, M. M. T. Baker rodrigo ocumpaugh monitoring protocol (BROMP) 2.0 technical and training manual. Tech. Rep., New York, NY and Manila, Philippines: Teachers College, Columbia University and Ateneo Laboratory for the Learning Sciences (2015).
Batista, G. E., Wang, X. & Keogh, E. J. A complexity-invariant distance measure for time series. In Proceedings of the 2011 SIAM International Conference on Data Mining, 699–710, https://doi.org/10.1137/1.9781611972818.60. SIAM (Society for Industrial and Applied Mathematics, 2011).
