M3B Corpus

M3B (Multi-Modal Meeting Behavior) Corpus is the dataset that includes sensor data for Micro Behavior in group discussion, survey data about satisfaction and engagement, and labeling data for the gesture and behavior (nodding, talking, smile).

We removed or processed the data which identify individuals (face images and voice).

Outline of Experiment

The flow of the experiment is as follows.

  1. 4 participants discuss the topic which is narrowed to 2 answers. The topic is simple and easy to participate fairly (ex.: Which do you like dogs or cats?, Which do you like summer or winter?)
  2. The discussion is finished, each participant answers the survey about satisfaction and engagement.
  3. Participants do 1 and 2 task two times, each participant labels their own behavior referring to the video.

Summary of the dataset

Detail of Experiment

The detail of the experiment is as follows.

  • Period: May 7th – Jun 5th, 2019
  • Participants: NAIST Ubiquitous Computing System Lab’s students
  • Male 17 (10 participated 2 times), Female 4 (2 participated 2 times)
  • Age: 22-25
  • Native Language: Japanese

It takes 120 hours to make the dataset. Most of it was spent on labeling work.

About Sensor

The following three types of sensors are used.

  • RICOH THETA V
    • Record micro behaviors (expressions, gestures, etc)
    • Converted to features with OpenFace and OpenPose
  • Pupil Labs Eye Tracker
    • Record the eye movements of the wearer as a video and CSV
    • Calculate gaze from eye movement and record it as coordinates in the landscape
  • LPMS-B2
    • Fixed to the grip of Pupil Labs
    • Record head acceleration and angular acceleration as CSV

Some sensor data is missed due to device malfunction. Details are described in the data set file.

About Survey

After the discussion, each participant answered the surveys about the change of opinion, engagement and so on. The answer point is 1 to 5.

IDDescription
A1 Before the discussion, is your opinion the former(1) or the latter(5)?
A2 After the discussion, is your opinion the former(1) or the latter(5)?
IDDescription
B1I was satisfied with the discussion.
B2I could talk my own opinion.
B3I heard what people with the same opinion say.
B4I heard what people with the opposite opinion say.
B5The discussion was enjoyable.
B6The group often cut off my speech.
B7I would like to have a discussion with this group in the future.
B8I have an attention for the 360-degree camera.

About Labeling

Each participant labeled their own behaviors for head and face. The contents of labeling are as shown in the table below. At that time, they also labeled the reason for the gesture.

Label nameCategoryDetail
smileresponsewithout special meaning generated by the listener
agreesmile with consent
interestingsmile occurs when the discussion is interesting
sympathyseeking empathy
noddingresponsewithout special meaning generated by the listener
agreenodding with consent
talkdescriptionexplain his/her opinion
objectiondeny the opinion of the other
agreeagree with the other opinion
saygive the right to speak to an other person

Terms of use

You can use this data set for non-profit and research purposes only. When presenting research results using datasets at academic conferences, please cite the following paper:

Yusuke Soneda, Yuki Matsuda, Yutaka Arakawa, Keiichi Yasumoto: “M3B Corpus: Multi-Modal Meeting Behavior Corpus for Group Meeting Assessment ”, UbiComp/ISWC ’19 Adjunct, September 9–13, 2019, London, United Kingdom.

Contact about Corpus

Please contact one of the following e-mail addresses, and we will give you M3B Corpus. When you send e-mail, please refer to the following for the mail format.

e-mail addresses:
soneda.yusuke.su2@is.naist.jp
yukimat@is.naist.jp

——————
Name:
Belongs:
E-mail address:
The way to use our corpus:
——————

Data provider

M3B Corpus is the result of joint research between the Kyushu University Humanophilic System Laboratory (Yutaka Arakawa Laboratory) and the Nara Institute of Science and Technology Ubiquitous Computing System Laboratory (Keiichi Yasumoto Laboratory).

Ethics approval

The study received ethics approval (approval no:2018-I- 28) after review by the research ethics committee at Nara Institute of Science and Technology.