Gender: Results

Questionnaires

Our questionnaire contains several questions, each one dealing with a specific aspect of the transcript. We will compute the average score for each question by the gender of the instruction givers and determine whether or not there is a significant difference. For example, it would be interesting if male-given instructions were rated an average of 2 on a 1-7 scale measuring clarity, while directions given by females were rated an average of 5 in that particular aspect.

We also want to see if the gender of the rater has any effect on the scores. Thus, we will compute the average scores for each question across the 4 possible pairings of instructor and rater (male-male, male-female, female-male, female-female). For example, do females think instructions given by other women are more understandable than those given by men?

One of the questions asks whether the rater thinks the giver is male or female. We would like to determine if there is any correlation between the perceived understandability of the directions and the apparent gender of the instructor. For instance, are clear-rated instructions assumed to be written by men? Furthermore, is this assumption correct?

Coding

To start with, it will be necessary to come up with an overview of the
language of the transcripts.  The basic measures mentioned below
(i.e., response length) will serve to give a quantitative overview.
Particular aspects of IM speech can be described by references to
papers which describe it in depth.

Analysis of the language of the transcripts is straightforward, but
discovering meaningful trends will require taking a number of
measurements, the majority of which will probably not be meaningful.
All quantitative measures will be tested for significance and
correlation with actual gender, a clarity metric, and perceived
gender, the latter two being derived from questionnaire responses.

Basic measures are average response length (in characters, words, and
"sentences"), if only to establish a baseline.

Then we will calculate and tabluate the relative frequency of the use
of:

 - words of a particular syntactic category (noun, verb/predicate,
descriptor)
 - use of a particular mode of address
 - use of personal pronouns
 - particular constructions (passive voice [and other marked syntactic
   constructions], indirect speech [which is more a semantic
   distinction])
 - use of particular semantic classes of verb (looking for "action"
   verbs or verbs of movement)

If any other patterns in word choice or speech become apparent, we
will check them for significance, as well.

Some weaker measures which still might be useful are:

 - lexical differences: does one gender use a given term to describe
   something which the other doesn't?  (This is hampered by our small
   sample sizes and a lack of responses which use very similar
   language.)
 - use of a particular mood or tense (This is also hampered by our
   small selection size and the fact that there is probably
   insufficient variation to obtain a significant measure.)

We can also do a speech act analysis of the transcripts and then see
if the prevalence of certain speech acts in the instructions mean
anything, but that will probably not be terribly informative because
everyone is giving the same sort of speech acts.  If we do not find
much in analysis of the basic language used, we will do a speech act
analysis similar to that in the Nastri paper.

It would then be possible to check for correlation of certain speech
act patterns with one of the three measures mentioned above (actual
gender, perceived clarity, and perceived gender), but there may be too
much uniformity in the transcripts to get a significant result other
than, "More words (speech acts) means more clear."