M3B (Multi-Modal Meeting Behavior) Corpus is the dataset that includes sensor data for Micro Behavior in group discussion, survey data about satisfaction and engagement, and labeling data for the gesture and behavior (nodding, talking, smile).
We removed or processed the data which identify individuals (face images and voice).
Outline of Experiment
The flow of the experiment is as follows.
- 4 participants discuss the topic which is narrowed to 2 answers. The topic is simple and easy to participate fairly (ex.: Which do you like dogs or cats?, Which do you like summer or winter?)
- The discussion is finished, each participant answers the survey about satisfaction and engagement.
- Participants do 1 and 2 task two times, each participant labels their own behavior referring to the video.
Summary of the dataset
Detail of Experiment
The detail of the experiment is as follows.
- Period: May 7th – Jun 5th, 2019
- Participants: NAIST Ubiquitous Computing System Lab’s students
- Male 17 (10 participated 2 times), Female 4 (2 participated 2 times)
- Age: 22-25
- Native Language: Japanese
It takes 120 hours to make the dataset. Most of it was spent on labeling work.
The following three types of sensors are used.
- RICOH THETA V
- Record micro behaviors (expressions, gestures, etc)
- Converted to features with OpenFace and OpenPose
- Pupil Labs Eye Tracker
- Record the eye movements of the wearer as a video and CSV
- Calculate gaze from eye movement and record it as coordinates in the landscape
- Fixed to the grip of Pupil Labs
- Record head acceleration and angular acceleration as CSV
Some sensor data is missed due to device malfunction. Details are described in the data set file.
After the discussion, each participant answered the surveys about the change of opinion, engagement and so on. The answer point is 1 to 5.
|A1||Before the discussion, is your opinion the former(1) or the latter(5)?|
|A2||After the discussion, is your opinion the former(1) or the latter(5)?|
|B1||I was satisfied with the discussion.|
|B2||I could talk my own opinion.|
|B3||I heard what people with the same opinion say.|
|B4||I heard what people with the opposite opinion say.|
|B5||The discussion was enjoyable.|
|B6||The group often cut off my speech.|
|B7||I would like to have a discussion with this group in the future.|
|B8||I have an attention for the 360-degree camera.|
Each participant labeled their own behaviors for head and face. The contents of labeling are as shown in the table below. At that time, they also labeled the reason for the gesture.
|smile||response||without special meaning generated by the listener|
|agree||smile with consent|
|interesting||smile occurs when the discussion is interesting|
|nodding||response||without special meaning generated by the listener|
|agree||nodding with consent|
|talk||description||explain his/her opinion|
|objection||deny the opinion of the other|
|agree||agree with the other opinion|
|say||give the right to speak to an other person|
You can use this data set for non-profit and research purposes only. When presenting research results using datasets at academic conferences, please cite the following paper:
Yusuke Soneda, Yuki Matsuda, Yutaka Arakawa, Keiichi Yasumoto: “M3B Corpus: Multi-Modal Meeting Behavior Corpus for Group Meeting Assessment ”, UbiComp/ISWC ’19 Adjunct, September 9–13, 2019, London, United Kingdom.
Contact about Corpus
Please contact one of the following e-mail addresses, and we will give you M3B Corpus. When you send e-mail, please refer to the following for the mail format.
The way to use our corpus：
M3B Corpus is the result of joint research between the Kyushu University Humanophilic System Laboratory (Yutaka Arakawa Laboratory) and the Nara Institute of Science and Technology Ubiquitous Computing System Laboratory (Keiichi Yasumoto Laboratory).
The study received ethics approval (approval no:2018-I- 28) after review by the research ethics committee at Nara Institute of Science and Technology.