9 What other gender disparities do we observe in oral sessions?

Author

Rebecca S. Chen & Petroula Botsidou

Note: this analysis is only part of the Appendix of the manuscript

What other gender disparities can we observe in oral sessions?

More specifically, we also collected data on whether positive appraisal was given by a questioner, whether a person asked a question without raising their hand or being chosen by the session host to do so (a jumper), whether a speaker talked longer than their allocated time, and if a question was criticizing the speaker. Depending on the question, we investigated the effect of the question asker’s gender, speaker’s gender, host’s gender while correcting for appropriate confounders such as talk number.

For all analysis, we only excluded questions that were follow-up questions, and those asked by the host. This means that these data include manipulated and unmanipulated sessions, as we do not expect our manipulation to interfere with what a questioner says exactly.

9.1 Positive appraisal

Positive appraisal was defined as any positive words towards the speaker, but excluding when simply “Thank you for your talk” was said. However, if this comment included a compliment such as “Thank you for your nice talk”, this was counted as positive appraisal.

We used binomial GLMMs to address the effect of the gender of the questioner and gender of the speaker, and corrected for the question number within that Q&A session. We assessed the fit of each variable using a likelihood ratio test and included only the variables that explained significant variation in the final model.

9.1.1 Receiving

# explore the data 
table(data_analysis$compliment, data_analysis$questioner_gender) %>% kbl() %>%
  kable_classic_2()

	F	M
No	275	325
Yes	216	214

table(data_analysis$compliment, data_analysis$speaker_gender) %>% kbl() %>%
  kable_classic_2()

	F	M
No	360	235
Yes	274	147

# build initial models

## speaker gender

model_compliment_s_gender <- glmer(compliment ~ speaker_gender + (1|session_id/talk_id), data = subset(data_analysis, !is.na(speaker_gender)), family = "binomial") 

model_compliment_s_gender_null <- glmer(compliment ~ 1 + (1|session_id/talk_id), data = subset(data_analysis, !is.na(speaker_gender)), family = "binomial") 

anova(model_compliment_s_gender, model_compliment_s_gender_null) # not significant

Data: subset(data_analysis, !is.na(speaker_gender))
Models:
model_compliment_s_gender_null: compliment ~ 1 + (1 | session_id/talk_id)
model_compliment_s_gender: compliment ~ speaker_gender + (1 | session_id/talk_id)
                               npar    AIC    BIC  logLik deviance  Chisq Df
model_compliment_s_gender_null    3 1351.1 1365.8 -672.53   1345.1          
model_compliment_s_gender         4 1351.0 1370.6 -671.47   1343.0 2.1136  1
                               Pr(>Chisq)
model_compliment_s_gender_null           
model_compliment_s_gender           0.146

summary(glmer(compliment ~ question_nr + (1|session_id/talk_id), data = subset(data_analysis, !is.na(speaker_gender)), family = "binomial"))

Generalized linear mixed model fit by maximum likelihood (Laplace
  Approximation) [glmerMod]
 Family: binomial  ( logit )
Formula: compliment ~ question_nr + (1 | session_id/talk_id)
   Data: subset(data_analysis, !is.na(speaker_gender))

     AIC      BIC   logLik deviance df.resid 
  1292.7   1312.4   -642.4   1284.7     1012 

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-1.5752 -0.7722 -0.4796  0.9020  3.7109 

Random effects:
 Groups             Name        Variance  Std.Dev. 
 talk_id:session_id (Intercept) 1.030e-09 0.0000321
 session_id         (Intercept) 5.363e-01 0.7323121
Number of obs: 1016, groups:  talk_id:session_id, 328; session_id, 64

Fixed effects:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  0.56082    0.16375   3.425 0.000615 ***
question_nr -0.34057    0.04676  -7.284 3.24e-13 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
            (Intr)
question_nr -0.705
optimizer (Nelder_Mead) convergence code: 0 (OK)
boundary (singular) fit: see help('isSingular')

9.1.2 Giving

## questioner gender

model_compliment_q_gender <- glmer(compliment ~ questioner_gender + (1|session_id/talk_id), data = subset(data_analysis, !is.na(questioner_gender)), family = "binomial") 

model_compliment_q_gender_null <- glmer(compliment ~ 1+ (1|session_id/talk_id), data = subset(data_analysis, !is.na(questioner_gender)), family = "binomial") 

anova(model_compliment_q_gender, model_compliment_q_gender_null) # not significant

Data: subset(data_analysis, !is.na(questioner_gender))
Models:
model_compliment_q_gender_null: compliment ~ 1 + (1 | session_id/talk_id)
model_compliment_q_gender: compliment ~ questioner_gender + (1 | session_id/talk_id)
                               npar    AIC    BIC  logLik deviance  Chisq Df
model_compliment_q_gender_null    3 1364.9 1379.7 -679.45   1358.9          
model_compliment_q_gender         4 1365.5 1385.2 -678.73   1357.5 1.4431  1
                               Pr(>Chisq)
model_compliment_q_gender_null           
model_compliment_q_gender          0.2296

These results show is that the likelihood of a questioner giving a words of positive appraisal is not affected by perceived gender.

9.2 Jumpers

Next, we asked whether men or women are more likely to ask a question without being allocated to do so (i.e. chosen by the session host to ask your question). Since this might have to do with the perceived ‘authority’ of the session host, we controlled for the gender of the session host as well.

We used binomial GLMMs to address the effect of the gender of the questioner, and corrected for question number and gender of the session host. We assessed the fit of each variable using a likelihood ratio test and included only the variables that explained significant variation in the final model.

# recode NA to 'no jumper' = 0
jumperdata <- data_analysis %>% mutate(jumper = as.factor(case_when(
  is.na(jumper) ~ "0",
  jumper == "1" ~ "1" )))

# explore the data 
table(jumperdata$jumper, jumperdata$questioner_gender) %>% kbl() %>%
  kable_classic_2()

	F	M
0	486	526
1	5	13

# build initial models

## question nr

model_jumper_q_nr <- glmer(jumper ~ question_nr + (1|session_id/talk_id), data = subset(jumperdata, !is.na(question_nr)), family = "binomial") 

model_jumper_q_nr_null <- glmer(jumper ~ 1 + (1|session_id/talk_id), data = subset(jumperdata, !is.na(question_nr)), family = "binomial") 

anova(model_jumper_q_nr, model_jumper_q_nr_null) # not significant

Data: subset(jumperdata, !is.na(question_nr))
Models:
model_jumper_q_nr_null: jumper ~ 1 + (1 | session_id/talk_id)
model_jumper_q_nr: jumper ~ question_nr + (1 | session_id/talk_id)
                       npar    AIC    BIC  logLik deviance  Chisq Df Pr(>Chisq)
model_jumper_q_nr_null    3 188.24 203.13 -91.122   182.24                     
model_jumper_q_nr         4 188.12 207.97 -90.062   180.12 2.1197  1     0.1454

## questioner gender

model_jumper_q_gender <- glmer(jumper ~ questioner_gender + (1|session_id/talk_id), data = subset(jumperdata, !is.na(questioner_gender)), family = "binomial") 

model_jumper_q_gender_null <- glmer(jumper ~ 1 + (1|session_id/talk_id), data = subset(jumperdata, !is.na(questioner_gender)), family = "binomial") 

anova(model_jumper_q_gender, model_jumper_q_gender_null) # almost significant

Data: subset(jumperdata, !is.na(questioner_gender))
Models:
model_jumper_q_gender_null: jumper ~ 1 + (1 | session_id/talk_id)
model_jumper_q_gender: jumper ~ questioner_gender + (1 | session_id/talk_id)
                           npar    AIC    BIC  logLik deviance  Chisq Df
model_jumper_q_gender_null    3 187.37 202.19 -90.687   181.37          
model_jumper_q_gender         4 186.35 206.10 -89.173   178.35 3.0267  1
                           Pr(>Chisq)  
model_jumper_q_gender_null             
model_jumper_q_gender          0.0819 .
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

summary(model_jumper_q_gender)

Generalized linear mixed model fit by maximum likelihood (Laplace
  Approximation) [glmerMod]
 Family: binomial  ( logit )
Formula: jumper ~ questioner_gender + (1 | session_id/talk_id)
   Data: subset(jumperdata, !is.na(questioner_gender))

     AIC      BIC   logLik deviance df.resid 
   186.3    206.1    -89.2    178.3     1026 

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-0.1572 -0.1572 -0.1572 -0.1014  9.8590 

Random effects:
 Groups             Name        Variance Std.Dev.
 talk_id:session_id (Intercept) 0        0       
 session_id         (Intercept) 0        0       
Number of obs: 1030, groups:  talk_id:session_id, 339; session_id, 64

Fixed effects:
                   Estimate Std. Error z value Pr(>|z|)    
(Intercept)         -4.5768     0.4495 -10.182   <2e-16 ***
questioner_genderM   0.8764     0.5300   1.654   0.0982 .  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
            (Intr)
qstnr_gndrM -0.848
optimizer (Nelder_Mead) convergence code: 0 (OK)
boundary (singular) fit: see help('isSingular')

## host gender
 
model_jumper_h_gender <- glmer(jumper ~ host_1_gender + (1|session_id/talk_id), data = subset(jumperdata, !is.na(host_1_gender)), family = "binomial") 

model_jumper_h_gender_null <- glmer(jumper ~ 1 + (1|session_id/talk_id), data = subset(jumperdata, !is.na(host_1_gender)), family = "binomial") 

anova(model_jumper_h_gender, model_jumper_h_gender_null) # not significant

Data: subset(jumperdata, !is.na(host_1_gender))
Models:
model_jumper_h_gender_null: jumper ~ 1 + (1 | session_id/talk_id)
model_jumper_h_gender: jumper ~ host_1_gender + (1 | session_id/talk_id)
                           npar    AIC    BIC  logLik deviance  Chisq Df
model_jumper_h_gender_null    3 169.48 184.15 -81.741   163.48          
model_jumper_h_gender         4 171.16 190.72 -81.579   163.16 0.3243  1
                           Pr(>Chisq)
model_jumper_h_gender_null           
model_jumper_h_gender           0.569

# nothing is significant, trend for gender questioner

These results show is that the likelihood of a person jumping a question not significantly affected by any of the variables. There was however a tendency for men to be more likely to jump a question compared to women. Note that the inference of these models is however limited, since jumpers were rare (N = 18).

9.3 Speaker over time

Next, we investigated the probability that a speaker talks for longer than their allocated speaking time is affected by speaker gender. We did not expect that any other confounding variables would explain variation in speaking overtime, since this is something that was prepared by only the speaker.

# have to summarize by talk not per question
data_overtime <- data_analysis %>% select(talk_id, session_id, overtime, speaker_gender, speaker_career_short) %>% unique()

# N = 
nrow(data_overtime)

[1] 340

# explore data
table(data_overtime$overtime, data_overtime$speaker_gender) %>% kbl() %>%
  kable_classic_2()

	F	M
N	181	97
Y	29	21

## speaker gender
model_overtime_s_gender <- glmer(overtime ~ speaker_gender + (1|session_id/talk_id), data = subset(data_overtime, !is.na(speaker_gender)), family = "binomial") 

model_overtime_s_gender_null <- glmer(overtime ~ 1 + (1|session_id/talk_id), data = subset(data_overtime, !is.na(speaker_gender)), family = "binomial") 

anova(model_overtime_s_gender, model_overtime_s_gender_null) # not significant

Data: subset(data_overtime, !is.na(speaker_gender))
Models:
model_overtime_s_gender_null: overtime ~ 1 + (1 | session_id/talk_id)
model_overtime_s_gender: overtime ~ speaker_gender + (1 | session_id/talk_id)
                             npar    AIC    BIC  logLik deviance  Chisq Df
model_overtime_s_gender_null    3 160.29 171.67 -77.145   154.29          
model_overtime_s_gender         4 162.21 177.38 -77.105   154.21 0.0795  1
                             Pr(>Chisq)
model_overtime_s_gender_null           
model_overtime_s_gender           0.778

The model output shows that gender did not affect the probability to speak overtime.

9.4 Critical question

Next, we assessed whether the likelihood of receiving a ‘critical’ question is affected by the gender of the questioner or the gender of the speaker.

9.4.1 Receiving

# explore data
table(data_analysis$question_type_e, data_analysis$questioner_gender) %>% kbl() %>%  kable_classic_2()

	F	M
0	458	499
1	19	23

table(data_analysis$question_type_e, data_analysis$speaker_gender) %>% kbl() %>%
  kable_classic_2()

	F	M
0	593	349
1	19	23

# build initial models

## speaker gender
model_critical_s_gender <- glmer(question_type_e ~ speaker_gender + (1|session_id/talk_id), data = subset(data_analysis, !is.na(speaker_gender)), family = "binomial") 

model_critical_s_gender_null <- glmer(question_type_e ~ 1 + (1|session_id/talk_id), data = subset(data_analysis, !is.na(speaker_gender)), family = "binomial") 

anova(model_critical_s_gender, model_critical_s_gender_null) # not significant, sort of trend

Data: subset(data_analysis, !is.na(speaker_gender))
Models:
model_critical_s_gender_null: question_type_e ~ 1 + (1 | session_id/talk_id)
model_critical_s_gender: question_type_e ~ speaker_gender + (1 | session_id/talk_id)
                             npar    AIC    BIC  logLik deviance  Chisq Df
model_critical_s_gender_null    3 317.43 332.11 -155.72   311.43          
model_critical_s_gender         4 318.13 337.69 -155.06   310.12 1.3065  1
                             Pr(>Chisq)
model_critical_s_gender_null           
model_critical_s_gender           0.253

9.4.2 Giving

## questioner gender
model_critical_q_gender <- glmer(question_type_e ~ questioner_gender + (1|session_id/talk_id), data = subset(data_analysis, !is.na(questioner_gender)), family = "binomial") 

model_critical_q_gender_null <- glmer(question_type_e ~ 1 + (1|session_id/talk_id), data = subset(data_analysis, !is.na(questioner_gender)), family = "binomial") 

anova(model_critical_q_gender, model_critical_q_gender_null) # not significant

Data: subset(data_analysis, !is.na(questioner_gender))
Models:
model_critical_q_gender_null: question_type_e ~ 1 + (1 | session_id/talk_id)
model_critical_q_gender: question_type_e ~ questioner_gender + (1 | session_id/talk_id)
                             npar    AIC    BIC  logLik deviance Chisq Df
model_critical_q_gender_null    3 317.28 332.00 -155.64   311.28         
model_critical_q_gender         4 318.42 338.04 -155.21   310.42 0.863  1
                             Pr(>Chisq)
model_critical_q_gender_null           
model_critical_q_gender          0.3529

These results show is that the likelihood of a person asking or receiving a critical question was not affected by the gender of the questioner nor speaker. Note that the inference of these models is however limited, since critical questions were rare and subject to observer bias (N = 43).