Cornell Language and Technology

exploring how technologies affect the way we talk, think and understand each other

Sunday, May 07, 2006

Presentations - Gender & Emotion

GENDER

You guys did a good job on your presentation. The “bunny ears” thing is pretty funny, and I will be interesting to see what other words you guys may find when doing a similar word analysis. It’s pretty amazing you guys found differences in men and women despite participants not even realizing they were there! It brings up a very interesting debate (and further research) about what’s causing this phenomenon linguistically, as well as what implications this may have for the recitation of instructions. I think it would be interesting to see if the linguistic analysis program could detect any of the gender linguistic differences. It would also be interesting to see if these results can be generalized in other mediums — namely FTF and over the phone. Knowing what you know now, it would be cool to see how adding the added cue of voice plays a role, as well as the non-verbal cues of FTF.

EMOTION

I was interested by your study because it seemed so intuitive, and I wanted to know if your hypotheses would prove true. Even though it don’t come out the way you wanted, I think that’s great too because you learned about another variable that might affect the process, which (I think, at least) is cool too.

I liked your presentation. It was clear overall and I enjoyed it (the color coding was a nice touch!). Good work relating things to Jeff’s study. I think people wanting to be amicable upon a first meeting as a face-related procedure definitely would explain your data. I know I would never want to express my sad emotions – especially upon first meeting someone – as I wouldn’t want one instance of my emotions to represent me as a person in someone else’s mind.

Friday, May 05, 2006

Presentation Comments

#1 - Facebook Common Ground

I was interested in your project because it seemed like one of the few studies of the new phenomenon of large groups (myspace, friendster, etc.) online. Great job on your speech and with finding results! First, I liked the diagrams that you used in your presentation - they were cute, well-drawn, and made your concepts easier to understand. One question I had about your project was why the participants did not have any obligation to click on the facebook link in the email you sent them. As you also stated in your presentation, perhaps if you explicitly told the participants to click on the link, you would have obtained more significant results. Also, it seemed like you stated your interpretations of the results as if they were facts; I think you should have emphasized that these were only speculations on your part. However, overall I was excited to hear your interpretations and I thought that given the limitations that you stated, the study turned out very well. Thanks for a ground-breaking study in this field!

#2 - Deception

I was impressed with how well you executed your project - I would imagine it is very difficult to deceive people in studies such as this one. It was also really interesting that your study was a kind of continuation of Professor Hancock's study. I thought that everyone in your group spoke clearly and well, giving your presentation a professional feeling. Also, the division of your results and interpretations into Q1, Q2, etc. helped a lot with clarity. One thing I didn't understand was that your two hypotheses seemed to be contradicting one another. I was also wondering if your choice to put an icebreaker at the beginning of the conversation was based on previous research or your own speculations. Overall, great interpretation of your results and an exciting study!

Thursday, May 04, 2006

Comments

Facebook Common Ground:

I really liked the general idea of seeing how people would act when they had knowledge but might not think it polite to put forth their knowledge/common ground directly. The different ways people approached it, from acting like they hadn't seen the Facebook to casually asking questions that would lean towards information they had gleaned from Facebook, was fascinating in what is says about how people determine the most polite or best way to act in conversation. Really interestin stuff. Great work.

Emotion:

Really interesting project, guys. I find the idea of emotion as a contagion and the differences in how accurately people can detect others' emotions in different mediums really fascinating. I agree with the general sentiment that it would be interesting to do the experiment with friends, and see if they fare better. All around, great idea and great analysis.

Comment #2: Emotions Group

Wonderful presentation! I was intrigued by your experiment the moment you decided to do it. I was curious to see whether or not, and to what degree you can truly induce emotion (such as sadness) in participants. It's so fascinating finding out that when the experimental group watched Sophie's Choice, they actually felt a lot more sad than the control group. I thought that the participants would try to repress their emotions since they're participating in an experiment and don't want to seem affected by it. But I'm not surprised at the actual result. However I still think that the induced emotion, however real it might be, is still not strong enough than emotions that people feel in their everyday lives, that is, how they feel when something happens around them. I think that, in order for someone to truly be saddened, they have to either experience something that they can relate to, or something like a sad movie (in which they have to sit through the whole thing and not just watch part of it). Only then will their emotions be strong. Therefore, one possible reason for your result (i.e. dyads couldn't detect their counterpart's emotions accurately) might be that the induced emotions weren't strong enough for others to detect.

Also, someone mentioned that friends might be better at detecting each other's emotions. I think
this is very true. If you know someone, you'll know how they act when they're sad--the subtle facial muscle movements, or the look in their eyes. But if you don't know someone, then you might not be able to pick up these subtle elements that indicate the emotional state that your partner is in.

Comment #1: Grounding Group

Great job on your presentation guys!
I think it's interesting/surprising that you found in your results that your participants were better at assessing each other's knowledge of the poem in an online medium rather than a face-to-face medium. One would think that, if you can see the person that you're talking to, you'd be better at gauging their understanding of the poem because of the abundance of visual/audio cues (i.e. from facial expressions to tone of voice). More generally, I think face-to-face overrules most other communication techniques in terms of aiding in someone's understanding of something. However, as Will mentioned today, we might have overlooked the social aspect of this. That is, when two strangers are put together in a room and are required to interact with each other on a specific topic, not only might it be uncomfortable, but i'd imagine it'd also be awkward. And because of this, the intended effect of having the dyads have a conversation (i.e. so that they can gauge how well each other understood the poem) may not be achieved. Of course, this might not be the case at all. I think with any experiment, it's important to have a large sample size. But given the time limit, I think you guys did great! :D

Comments

Grounding:

Wow, you definitely got some really interesting results. I think the fact that the participants thought they influenced their partner more over AIM than FtF is explainable - there are fewer visual cues that might suggest otherwise (they can't see their partners shake their heads, etc). But the fact that they were more successful in guessing each other's knowledge over aim is quite unintuitive. You'll have to closely analyze the transcripts to see why the conversations were so different. Maybe there was a lot of superfluous material in FtF that distracted from the utterances about the poem. Great job overall!

Emotion:

It's great that your participants were not suspicious of the setup of the experiment and that mood induction worked as planned. They fact that emotion could not be detected by the participants' partners, even though there were clear language differences, is very intriguing but not necessarily surprising given the content of your transcripts. Indeed, for example, most people would probably not associate the use of past tense with being in a sad state. Also, I'm sure that the fact that the two were strangers had an impact - I find it fairly easy to detect the mood of someone I know well over AIM. Nice presentation and good luck on the rest of your analysis!

Wednesday, May 03, 2006

Presentation Comments #2

Deception/Motivation group:
Your study is fascinating. I think that even though many of your results weren't significant, that still says a lot. Kailyn's comment makes an excellent point that we are simply really bad at detecting deception, in any situation, and we also are accustomed to believing that someone is telling us the truth. I think this definitely contributes to the finding that each group had about the same level of detection. I also think that your finding of a much higher word count in the high motivation condition is really interesting, and unexpected. I would love for you (or anyone) to dig deeper into this, and to try to find out why this actually happens. Great job guys.

Presentation Comments #1

To the gender group:
I really enjoyed your presentation, and hearing your findings. I was amazed that despite the fact that you found so many linguistic differences between men and women, we (as raters) were simply unable to do any better than chance at guessing the gender of the author. I would be really interested to see if, given these linguistic diferences, something like a automated computer program could do better at guessing the gender of the author, as a computer could just focus on these things while we are distracted by so many factors and are unaware of what is truly a difference between men and women's writing styles. Really neat results!

Tuesday, April 25, 2006

Facebook Intro - Results

Usage of Facebook

The study found widespread usage of Facebook. Not only did all of the surveyed students possess an account on Facebook, but 92% also stated that they had accessed it at least once in the past month; 35% of the students admitted to using it daily. Only two of the students, however, had ever used Facebook to meet a stranger before.


Knowledge Introduction Process

A statistical analysis revealed significant differences in all conditions, indicating that the average number of Facebook-related, highlighted utterances varied based on whether or not the participant had prior access to their partner's Facebook profile. In other words, if both participants had access to each other's Facebook profiles, students tended to highlight significantly more utterances than either of the individuals in the one-way Facebook condition. This finding can be taken as evidence that Facebook-inspired common ground was introduced into the conversations. Somewhat surprisingly, the individuals in the one-way condition without Facebook access, even after they were told that their partners had looked at their Facebook profiles, tended not to believe that Facebook had contributed at all to their conversation. It is also notable that, across conditions, questions were often interpreted differently by the participants. For example, in the mutual-Facebook condition, the question "what is your major?" was highlighted as a Facebook-related utterance, while in the one-way Facebook condition, the same question was often not highlighted by either of the individuals.

Our coding revealed that the common ground-related information typically introduced in the conversations tended to be academic, personal, or about interests or friends. The participants tended to introduce this information by asking or answering probes, by making explicit references to Facebook, or by making casual references to Facebook-obtained information. While casual references were fairly common, what was particularly surprising was the discovery that 76% of dyads engaged in some form of probing behavior, while only 20% made explicit references to Facebook. One possible explanation for this finding rests in Clark's equity principle; in our experiments, equity appeared to be disturbed either by the participants possessing nonequivalent information about each other or by the participants not knowing how acceptable it would be to introduce Facebook-obtained information into the conversation. As such, it is likely that the observed probing behavior may be the method that the speakers presuppose for maintaining equity with their addressees (Clark 295).
If so, our research may indicate that communicators to some degree use Facebook-obtained information to inform and improve their social interactions.

Of the dyads that used probes to introduce some kind of information, X% of them ended up explicitly mentioning Facebook somewhere in their conversations. Y% of the dyads that used casual references ended up using explicit references as well. The overall number of highlighted references of those whom explicitly mention Facebook was much higher than those whom exclusively used either a probing strategy or casual mentions, suggesting perhaps that after hesitations about the equity-disturbing effects of mentioning Facebook as a source of information have been allayed, the participants were better able to reflect and attribute certain utterances to Facebook.

Gender: Results

Questionnaires

Our questionnaire contains several questions, each one dealing with a specific aspect of the transcript. We will compute the average score for each question by the gender of the instruction givers and determine whether or not there is a significant difference. For example, it would be interesting if male-given instructions were rated an average of 2 on a 1-7 scale measuring clarity, while directions given by females were rated an average of 5 in that particular aspect.

We also want to see if the gender of the rater has any effect on the scores. Thus, we will compute the average scores for each question across the 4 possible pairings of instructor and rater (male-male, male-female, female-male, female-female). For example, do females think instructions given by other women are more understandable than those given by men?

One of the questions asks whether the rater thinks the giver is male or female. We would like to determine if there is any correlation between the perceived understandability of the directions and the apparent gender of the instructor. For instance, are clear-rated instructions assumed to be written by men? Furthermore, is this assumption correct?

Coding

To start with, it will be necessary to come up with an overview of the
language of the transcripts. The basic measures mentioned below
(i.e., response length) will serve to give a quantitative overview.
Particular aspects of IM speech can be described by references to
papers which describe it in depth.

Analysis of the language of the transcripts is straightforward, but
discovering meaningful trends will require taking a number of
measurements, the majority of which will probably not be meaningful.
All quantitative measures will be tested for significance and
correlation with actual gender, a clarity metric, and perceived
gender, the latter two being derived from questionnaire responses.

Basic measures are average response length (in characters, words, and
"sentences"), if only to establish a baseline.

Then we will calculate and tabluate the relative frequency of the use
of:

- words of a particular syntactic category (noun, verb/predicate,
descriptor)
- use of a particular mode of address
- use of personal pronouns
- particular constructions (passive voice [and other marked syntactic
constructions], indirect speech [which is more a semantic
distinction])
- use of particular semantic classes of verb (looking for "action"
verbs or verbs of movement)

If any other patterns in word choice or speech become apparent, we
will check them for significance, as well.

Some weaker measures which still might be useful are:

- lexical differences: does one gender use a given term to describe
something which the other doesn't? (This is hampered by our small
sample sizes and a lack of responses which use very similar
language.)
- use of a particular mood or tense (This is also hampered by our
small selection size and the fact that there is probably
insufficient variation to obtain a significant measure.)

We can also do a speech act analysis of the transcripts and then see
if the prevalence of certain speech acts in the instructions mean
anything, but that will probably not be terribly informative because
everyone is giving the same sort of speech acts. If we do not find
much in analysis of the basic language used, we will do a speech act
analysis similar to that in the Nastri paper.

It would then be possible to check for correlation of certain speech
act patterns with one of the three measures mentioned above (actual
gender, perceived clarity, and perceived gender), but there may be too
much uniformity in the transcripts to get a significant result other
than, "More words (speech acts) means more clear."

Monday, April 24, 2006

#11: Private vs. Public – Results

For the results section, our group is going to analyze the three sets of data obtained from our experiment: task messages, historical messages, and questionnaires. We will be focusing on the language content, syntax, and choice of communication used in the messages.

Task and Historical Messages
Our group will be dividing the messages into two categories: wall posts and private messages. Then, we are going to compare the language usage and content of each group. Specifically, we will be using software that counts and reports statistics on the number of times certain words and punctuation appear in each list. Preliminary analysis indicates that private messages contain more “I” words while wall posts contain more “you” words. This may suggest that PMs focus more on the sender while wall messages focus more on the receiver. With personal messages, there is a one-to-one communication whereas with walling there are numerous people who can view the message. Thus, in such a public setting the sender may want to maintain face and focus more on the receiver. We also found that there are more misspelled words and use of slang in wall posts. This observation is interesting because common sense suggests that public posts would be written more carefully than private messages. However, perhaps this is not the case due to the informal nature of walling.

We also plan to examine the data content to see whether there is a correlation between the type of message and the choice of communication used. Our group expects wall posts to be more social-oriented and positive, and PMs to be more task-oriented and negative/neutral. For our six tasks, we designed the statements so that three of them were task-related and positive, while the other three were social-related and negative/neutral. From the data obtained so far, the choices of communication used for five out of the six tasks confirms our predictions. Thus, wall posts in general seem to be more informal and casual than private messages.

Questionnaire
For each set of statements in the questionnaire, we will be evaluating which of the potential reasons listed were favored the most. Then we will relate those preferred statements back to the type of communication that was used for the particular task and see whether it matches our predictions. For example, one question for Task 1 was to circle to what degree the participant agreed with the following statement: “I did not want other people to see it (the message they wrote for Task 1)”. If they had initially chosen to use private messaging for Task 1 and also strongly agreed with the statement, then we can infer that audience plays a significant role in the choice of private versus public communication.

Tuesday, April 18, 2006

Emotions Group - Methods

Participants

We plan to recruit a total of 60 students from the Communications courses that offer class credit for participation in experiments. All the participants will be between the ages of 18 and 25. 30 people will be randomly assigned to the control group, and 30 people will be randomly assigned to the experimental group. In this way, both the control and experimental group will consist of 15 dyads. Each dyad will be of the same sex: males will be paired together and females will be paired together.

Our study was conducted on the second floor of Kennedy Hall in Professor Hancock’s laboratory. The two members of a dyad were asked to arrive in Kennedy Hall at separate locations so that they cannot communicate prior to the experiment. Additionally, they were shown to different rooms in the laboratory so that they could not see their partner during the study.

Procedure

The subjects were first told that they were involved in a study that will test their ability to analyze movies. In the control group, both people were shown a neutral clip from the documentary Mammoths of the Ice Age. In the experimental group, one person was shown the same neutral clip from Mammoths of the Ice Age, while the other person was shown a sad clip from the movie Sophie’s Choice. Afterwards, they were asked to fill out a brief questionnaire about the movie and a PANAS emotion scale. We then deceived the subjects by asking them to participate in a second study that will analyze how people interact when they meet someone new online. They were given the task of getting to know their partner for twenty minutes through the medium of instant messaging. In addition, we requested that they not mention the previous study to their partner, as conversation about the movie could affect the mood of both subjects. Following this, both members of the dyad were given a questionnaire that included questions asking them to rate their own mood as well as that of their partner. The subjects also filled out another PANAS emotion scale. After the experiment, we debriefed the subjects on the true purposes of our study.

Materials

As our objective was to use films to induce certain emotions, we chose two different films for the study. The film intended to induce sadness was entitled Sophie’s Choice; the neutral film was a documentary entitled Mammoths of the Ice Age. Our group also used instruction sheets, which we did not show the participants. In addition, we used two computer terminals, one for each participant. We used the basic psychological scale, PANAS, twice in the experiment in order to assess the mood of each participant after the movie and after the conversation. Accompanying each part of our experiment (the film and the online chat) was a questionnaire. One questionnaire asked about the effects of movies on viewers; the second questionnaire asked about the feelings of each participant and his partner after conversing through instant messaging. We expect that both the scales and questionnaires will be crucial to gauge the emotions of our participants.

Since our experiment was divided into two sections, we used two consent forms to aid in deceiving the participants. As a distraction for the single experiment we were running, the participants were told that one form was for Professor Shapiro and the other was for Professor Hancock. Lastly, we had debriefing forms that the participants received after completing the experiment.

Coding

Following the experiment, we will use the Linguistic Inquiry and Word Count program to analyze the emotion expressed through instant messaging. This program will help us determine the 1) emoticons and other CMC conventions, 2) length of responses, 3) use of affect language (divided into subcategories of positive and negative), 4) pronoun use, and 5) the number of negations and assents in the conversations. By hand, we can determine the 1) use of punctuation and 2) content of the conversations. In addition to the transcripts, we will use the questionnaires from our experiment to determine the mood of the neutral participant and to determine whether he could perceive the emotions of his partner.

#10 — Grounding/FOOK

Method


Participants


In our experiments, n students (y males, x females; average age z) from Communication courses at Cornell University participated in the study in exchange for course credit.


Materials & Procedure


At the start of each experiment, participants were given a brief overview about the premise of the experiment. They were told they would be participating in a study on short-term memory to prevent any expectations of the experiment from affecting the outcome. After this overview, each participant was asked to complete a consent form and a demographics form, the latter being used to collect statistics on age and gender.


The participants in each dyad were then separated; each participant was given an excerpt from a T.S. Eliot poem, "Burnt Norton", to read. Afterwards, they were asked to answer a short questionnaire that related to what they did and did not understand in the poem, and how they interpreted it. The final section of the questionnaire tested their recognition of public figures in a variety of fields as diverse as physics, art, entertainment, and crime. After each participant answered their questionnaire, the dyad was brought together either in an AIM
conversation or in a face-to-face setting and was asked to have a short conversation with each other about what they had read. After completing the conversation, the participants were again separated and asked to answer a second questionnaire. Many of the questions on this questionnaire corresponded directly to specific questions on the first questionnaire, but this time asked how they believed their partner in
the dyad had understood the poem. Similar to the first questionnaire, the second questionnaire included a public figure recognition section, which asked if a participant thought their partner would recognize the same people the participant was asked about in the first questionnaire.


Once both participants finished their questionnaires, they were brought together again for a short debriefing, during which they were told about the true premise of the experiment. Each participant received a copy of a standard debriefing document, as well as a copy of the consent form that they had signed before starting the experiment.


Coding


Once both questionnaires had been collected from all participants, the data from each dyad was entered into a spreadsheet and correlated. We plan to perform four types of correlations with these data:



  1. For each participant, compare his/her first questionnaire and second questionnaire;

  2. For each dyad, compare their first questionnaires;

  3. For each dyad, compare their second questionnaires;

  4. For each dyad, compare one’s first questionnaire to the other's second questionnaire, and vice versa.


Since many of the questions on the first questionnaire correspond to questions on the second questionnaire, these will be the focus of our numerical correlations. All of these corresponding questions have numerical values ranging from 1 to 7, except for the yes/no questions in the person recognition section. Our correlations will measure numerically how accurately participants intuited each other's knowledge and feelings, as well as how close each particpant's knowledge and feelings were to the knowledge and feelings of their dyad partner.