5  Factors that affect the gender disparity

Author

Rebecca S. Chen

Why do women ask less questions than men do?

The results from the previous sections show that women ask less questions than men do, and this is because they raise their hands less often. We next aimed to address why we observe this, and how we can encourage women to ask more questions. Could female representation encourage women to participate more, or does it depend more on the setting?

5.1 Survey data

5.1.1 Conditions

More specifically, we asked in the survey to indicate on a 7-point Likert scale to what extent people agree with five statements. The statements were phrased as follows: I feel more comfortable asking a question if: 1) the presenter is of my own gender, 2) there is representation of my gender in the audience, 3) the audience size is smaller, 4) I know the speaker, or 5) the host is of their my own gender.

5.1.2 Motivations and hestitations

The post-congress survey additionally included questions on what aspect(s) motivate(s) people from asking questions at the Behaviour 2023 and in general (hereafter referred to as “motivations”), and what aspect(s) make(s) people more hesitant towards asking a question (hereafter referred to as “hesitations”). We can therefore get a deeper understanding into the internal factors associated with question asking behaviour that affect women more than men. The motivations to ask a question that were most commonly selected by respondents included “Interest in the topic” (87.6%), “Gaining deeper understanding” (72.5%), and “Relevance for my own research” (50%) (see Methods for all motivations included in the survey). The most commonly selected hesitations included “I would rather ask my question after the session, 1-1 with the speaker” (45.3%), “Afraid I misunderstood the content of the presentation” (40.4%) and “I did not think my question was relevant/important” (39.6%).

5.1.3 Analysis of gender effects

We built five ordinal GLMs, with the Likert-scale response to each of the five conditions as the dependent variable (1: strongly disagree, 7: strongly agree) and self-reported gender identity as the independent variable, while correcting for variation in career stages. We also built 19 binomial GLMs, one per motivation/hesitation, with the binomial response whether the motivation or hesitation was ticked (1) or not (0) as the dependent variable, and self-reported gender as the independent variable, while controlling for career stage. Statistical significance was inferred through an LRT.

We first analyzed which conditions, motivations and hesitations are more often indicated to be of importance by women compared to men, and subsequently which factors best predict the probability that a person asked a question during the congress.

# rearrange genders
survey$gender <- factor(survey$gender, levels = c("Male", "Female", "Non-binary"))

# first ask: gender bias in feeling uncomfortable in asking a question?
survey$comfort_asking_rating <- as.factor(survey$comfort_asking_rating)

comfort_gender <- MASS::polr(comfort_asking_rating ~ gender + career_3cat, data = subset(survey, !is.na(gender))) 

coeftest(comfort_gender)

t test of coefficients:

                       Estimate Std. Error t value  Pr(>|t|)    
genderFemale           -1.26188    0.21061 -5.9917 5.006e-09 ***
genderNon-binary       -1.60412    0.67979 -2.3597   0.01882 *  
career_3catLate career  2.47523    0.32986  7.5038 4.829e-13 ***
career_3catMid career   1.04273    0.20670  5.0447 7.192e-07 ***
career_3catOther        0.56917    0.47113  1.2081   0.22780    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
comfort_gender_null <- MASS::polr(comfort_asking_rating ~ career_3cat, data = subset(survey, !is.na(gender))) 

m_comfortout <- collect_out(model = comfort_gender, null = comfort_gender_null, name = "comfort_ask_question_gender", n_factors = 4, type = "likert", save = "yes",  dir = "../results/survey") 

m_comfortout %>% t() %>% kbl() %>%
  kable_classic_2()
model_name comfort_ask_question_gender
AIC 1360.136
n_obs 374
lrt_pval 0
lrt_chisq 38.188
intercept_12 -2.774
intercept_23 -1.616
intercept_34 -0.875
intercept_45 -0.074
intercept_56 0.945
intercept_67 2.295
n_factors 4
est_genderFemale -1.262
lowerCI_genderFemale -1.676
higherCI_genderFemale -0.848
se_genderFemale 0.211
tval_genderFemale -5.992
pval_genderFemale 0
est_genderNon-binary -1.604
lowerCI_genderNon-binary -2.941
higherCI_genderNon-binary -0.267
se_genderNon-binary 0.68
tval_genderNon-binary -2.36
pval_genderNon-binary 0.019
est_career_3catLate career 2.475
lowerCI_career_3catLate career 1.827
higherCI_career_3catLate career 3.124
se_career_3catLate career 0.33
tval_career_3catLate career 7.504
pval_career_3catLate career 0
est_career_3catMid career 1.043
lowerCI_career_3catMid career 0.636
higherCI_career_3catMid career 1.449
se_career_3catMid career 0.207
tval_career_3catMid career 5.045
pval_career_3catMid career 0 
# if you're less comfortable asking a question, is it less likely you ask one?
survey$comfort_asking_rating <- as.numeric(as.character(survey$comfort_asking_rating))

ask_comfort <- glm(ask_questions ~ comfort_asking_rating + gender + career_3cat,
                   family = binomial, data = subset(survey, !is.na(gender) & !is.na(career_3cat) & !is.na(comfort_asking_rating))) 

ask_comfort_null <- glm(ask_questions ~ gender + career_3cat, family = binomial,
                        data = subset(survey, !is.na(gender) & !is.na(career_3cat) & !is.na(comfort_asking_rating))) 

m_comfort_ask_out <- collect_out(model = ask_comfort, null = ask_comfort_null, name = "comfort_ask_question", n_factors = 5 , type="survey", save = "yes",  dir = "../results/survey") 

m_comfort_ask_out %>% t() %>% kbl() %>%
  kable_classic_2()
model_name comfort_ask_question
AIC 371.653
n_obs 372
lrt_pval 0
lrt_chisq 86.582
intercept_estimate -2.781
intercept_estimate_prop 0.058
intercept_pval 0
intercept_ci_lower -3.695
intercept_ci_higher -1.866
n_factors 5
est_comfort_asking_rating 0.733
lowerCI_comfort_asking_rating 0.555
higherCI_comfort_asking_rating 0.911
pval_comfort_asking_rating 0
zval_comfort_asking_rating 8.07
est_genderFemale 0.259
lowerCI_genderFemale -0.341
higherCI_genderFemale 0.86
pval_genderFemale 0.397
zval_genderFemale 0.846
est_genderNon-binary 2.686
lowerCI_genderNon-binary 0.372
higherCI_genderNon-binary 4.999
pval_genderNon-binary 0.023
zval_genderNon-binary 2.275
est_career_3catLate career 1.732
lowerCI_career_3catLate career 0.414
higherCI_career_3catLate career 3.05
pval_career_3catLate career 0.01
zval_career_3catLate career 2.575
est_career_3catMid career 0.336
lowerCI_career_3catMid career -0.22
higherCI_career_3catMid career 0.892
pval_career_3catMid career 0.237
zval_career_3catMid career 1.183 
# indeed due to gender bias in being comfortable asking questions

### isolate reasons
# take out reasons that were ticked less than 15 times
n_ticked <- data.frame()

for (i in 1:ncol(survey)){
  if (grepl("reason_question|reason_noquestion", colnames(survey)[i]) == TRUE) {
    sub <- survey[,i]
    sum <- as.data.frame(summary(as.factor(sub)))
    ticked <- sum["1","summary(as.factor(sub))"]
    row <- data.frame(i = i,
                      reason = colnames(survey)[i],
                      ticked = ticked)
    n_ticked <- rbind(n_ticked, row)
  }
}

ticked_enough <- subset(n_ticked, ticked >= 15)

# isolate questions about conditions, motivations and hesitations
conditions <- grep("more_comfort", colnames(survey))
reasons <- c(ticked_enough$i)
reasons_conditions <- c(conditions, reasons)

# take out 'other' reasons
other <- grep("_other", colnames(survey)[reasons_conditions]) 
reasons_conditions <- reasons_conditions[-other] #exclude 'other' categories

# take out not chosen
notchosen <- grep("reason_noquestion_notchosen", colnames(survey)[reasons_conditions]) 
reasons_conditions <- reasons_conditions[-notchosen] #exclude 

output_reasons <- data.frame()

for (i in reasons_conditions){
  ## split conditions between motivation/hestitations 
  if (grepl("more_comfort", colnames(survey)[i]) == TRUE) {
    # null model
    formula_null <- as.formula(paste0(colnames(survey)[i], "~ career_3cat"))
    null <- MASS::polr(formula_null, data = subset(survey, !is.na(gender))) #polr for ordinal GLMs
    
    # alternative model with gender
    formula_alt <- formula(paste0(colnames(survey)[i], "~ gender + career_3cat"))  
    model <- MASS::polr(formula_alt, data = subset(survey, !is.na(gender)))
    
    sum <- as.data.frame(coeftest(model)[,])
    confint <- as.data.frame(confint(model))
    
    # lrt
    lrt <- anova(null, model, test = "Chisq")
    lrt_pval <- lrt$`Pr(Chi)`[2]
    lrt_chisq <- lrt$`LR stat.`[2]
    
    out <- data.frame(response = colnames(survey)[i],
                      lrt_chisq = lrt_chisq,
                      lrt_pval = lrt_pval,
                      coef_female = sum["genderFemale","Estimate"],
                      pval_female = sum["genderFemale","Pr(>|t|)"],
                      lower_female = confint["genderFemale","2.5 %"],
                      upper_female = confint["genderFemale","97.5 %"],
                      se_female = sum["genderFemale", "Std. Error"],
                      zval_tval_female = sum["genderFemale", "t value"],
                      coef_non_binary = sum["genderNon-binary","Estimate"],
                      pval_non_binary = sum["genderNon-binary","Pr(>|t|)"],
                      lower_non_binary = confint["genderNon-binary","2.5 %"],
                      upper_non_binary = confint["genderNon-binary","97.5 %"],
                      se_non_binary = sum["genderNon-binary", "Std. Error"],
                      zval_tval_non_binary = sum["genderNon-binary", "t value"],
                      coef_mid = sum["career_3catMid","Estimate"],
                      pval_mid = sum["career_3catMid","Pr(>|t|)"],
                      se_mid = sum["career_3catMid", "Std. Error"],
                      zval_tval_mid = sum["career_3catMid", "t value"],
                      coef_late = sum["career_3catLate","Estimate"],
                      pval_late = sum["career_3catLate","Pr(>|t|)"],
                       se_late = sum["career_3catLate", "Std. Error"],
                      zval_tval_late = sum["career_3catLate", "t value"],
                      coef_other = sum["career_3catOther","Estimate"],
                      pval_other = sum["career_3catOther","Pr(>|t|)"],
                       se_other = sum["career_3catOther", "Std. Error"],
                     zval_tval_other = sum["career_3catOther", "t value"])
    
    
    output_reasons <- rbind(output_reasons, out)
    
    }
  if (grepl("reason", colnames(survey)[i]) == TRUE) {
    # null model
    formula_null <- as.formula(paste0(colnames(survey)[i], "~ career_3cat"))
    null <- glm(formula_null, data = subset(survey, !is.na(gender)), family = "binomial")
    # alternative model with gender
    formula_alt <- formula(paste0(colnames(survey)[i], "~ gender + career_3cat"))  
    model <- glm(formula_alt, data = survey, family = "binomial")
    sum <- summary(model)$coefficients
    confint <- (as.data.frame(summ(model, confint=TRUE)$coeftable))
    
    # lrt
    lrt <- anova(null, model, test = "Chisq")
    lrt_pval <- lrt$`Pr(>Chi)`[2]
    lrt_chisq <- lrt$`Deviance`[2]
    
    # collect all output
    out <- data.frame(response = colnames(survey)[i],
                      lrt_chisq = lrt_chisq,
                      lrt_pval = lrt_pval,
                      coef_female = sum["genderFemale","Estimate"],
                      pval_female = sum["genderFemale","Pr(>|z|)"],
                      lower_female = confint["genderFemale","2.5%"],
                      upper_female = confint["genderFemale","97.5%"],
                      se_female = sum["genderFemale", "Std. Error"],
                      zval_tval_female = sum["genderFemale", "z value"],
                      coef_non_binary = sum["genderNon-binary","Estimate"],
                      pval_non_binary = sum["genderNon-binary","Pr(>|z|)"],
                      lower_non_binary = confint["genderNon-binary","2.5%"],
                      upper_non_binary = confint["genderNon-binary","97.5%"],
                      se_non_binary = sum["genderNon-binary", "Std. Error"],
                      zval_tval_non_binary = sum["genderNon-binary", "z value"],
                      coef_mid = sum["career_3catMid career","Estimate"],
                      se_mid = sum["career_3catMid career","Std. Error"],
                      zval_tval_mid = sum["career_3catMid career","z value"],
                      pval_mid = sum["career_3catMid career","Pr(>|z|)"],
                      coef_late = sum["career_3catLate career","Estimate"],
                      se_late = sum["career_3catLate career","Std. Error"],
                      zval_tval_late = sum["career_3catLate career","z value"],
                      pval_late = sum["career_3catLate career","Pr(>|z|)"],
                      coef_other = sum["career_3catOther","Estimate"],
                      se_other = sum["career_3catOther","Std. Error"],
                      zval_tval_other = sum["career_3catOther","z value"],
                      pval_other = sum["career_3catOther","Pr(>|z|)"]
                      )
    
    output_reasons <- rbind(output_reasons, out)
    }}
  

# adjust p-values for multiple testing

output_reasons$lrt_qval <- p.adjust(output_reasons$lrt_pval, method = "fdr", n = nrow(output_reasons))

# round to 3 digits
output_reasons <- output_reasons %>% mutate_if(is.numeric, round, digits=3)

# save output in results
write.csv(output_reasons, file = "../results/question-asking/question_asking_why_survey_result.csv", quote=F, row.names = F)

output_reasons %>% arrange(lrt_qval) %>% kbl() %>%
  kable_classic_2()
response lrt_chisq lrt_pval coef_female pval_female lower_female upper_female se_female zval_tval_female coef_non_binary pval_non_binary lower_non_binary upper_non_binary se_non_binary zval_tval_non_binary coef_mid pval_mid se_mid zval_tval_mid coef_late pval_late se_late zval_tval_late coef_other pval_other se_other zval_tval_other lrt_qval
more_comfort_asking_owngender_speaker_rating 36.304 0.000 1.222 0.000 0.785 1.673 0.226 5.402 2.441 0.000 1.080 3.781 0.681 3.583 -0.345 0.104 0.211 -1.631 -0.617 0.062 0.330 -1.869 -0.222 0.657 0.500 -0.444 0.000
more_comfort_asking_owngender_audience_rating 41.061 0.000 1.327 0.000 0.903 1.764 0.219 6.054 1.902 0.004 0.585 3.201 0.659 2.888 -0.114 0.583 0.208 -0.549 -0.453 0.175 0.333 -1.358 -0.161 0.730 0.466 -0.345 0.000
more_comfort_asking_owngender_host_rating 19.640 0.000 0.921 0.000 0.487 1.368 0.224 4.109 1.578 0.019 0.255 2.923 0.672 2.349 -0.215 0.318 0.215 -1.000 -0.707 0.033 0.330 -2.141 0.119 0.818 0.515 0.231 0.000
more_comfort_asking_smallaudience_rating 15.805 0.000 0.791 0.000 0.389 1.196 0.206 3.843 -0.074 0.913 -1.419 1.250 0.673 -0.109 -0.295 0.145 0.202 -1.460 -0.588 0.058 0.309 -1.901 -0.370 0.445 0.484 -0.765 0.002
reason_noquestion_phrasing 11.191 0.004 0.900 0.002 0.343 1.458 0.285 3.164 1.026 0.205 -0.562 2.614 0.810 1.266 -0.570 0.024 0.253 -2.251 -1.588 0.002 0.503 -3.156 -0.987 0.141 0.670 -1.474 0.015
reason_noquestion_confidence 7.642 0.022 0.784 0.012 0.176 1.392 0.310 2.526 -0.233 0.835 -2.422 1.957 1.117 -0.208 -0.420 0.119 0.269 -1.559 -2.914 0.005 1.026 -2.840 -0.590 0.379 0.671 -0.879 0.073
reason_noquestion_intimidated_audience 6.270 0.043 0.760 0.021 0.115 1.405 0.329 2.309 -0.107 0.925 -2.326 2.111 1.132 -0.095 -1.211 0.000 0.322 -3.762 -1.197 0.018 0.505 -2.369 -1.075 0.169 0.781 -1.376 0.124
reason_question_voiceheard 5.946 0.051 -0.601 0.193 -1.505 0.304 0.461 -1.302 1.787 0.057 -0.052 3.626 0.938 1.904 0.856 0.094 0.511 1.673 0.703 0.334 0.727 0.967 1.564 0.072 0.869 1.800 0.128
reason_noquestion_private 5.367 0.068 0.297 0.201 -0.159 0.753 0.233 1.277 -1.553 0.158 -3.707 0.601 1.099 -1.413 -0.047 0.837 0.229 -0.206 -1.106 0.004 0.379 -2.918 0.394 0.473 0.550 0.717 0.152
reason_noquestion_intimidated_setting 4.871 0.088 0.719 0.126 -0.203 1.641 0.470 1.528 2.002 0.036 0.129 3.876 0.956 2.094 -0.974 0.044 0.483 -2.016 0.293 0.563 0.505 0.579 0.698 0.315 0.694 1.006 0.175
reason_question_interest 4.050 0.132 -0.529 0.166 -1.275 0.218 0.381 -1.387 14.303 0.987 -1753.050 1781.656 901.727 0.016 -0.529 0.107 0.329 -1.610 1.731 0.096 1.040 1.665 0.646 0.543 1.064 0.608 0.217
reason_noquestion_relevance 3.894 0.143 -0.264 0.255 -0.719 0.190 0.232 -1.139 -1.688 0.125 -3.844 0.469 1.100 -1.534 -0.329 0.160 0.234 -1.403 -0.710 0.051 0.363 -1.955 0.636 0.248 0.550 1.156 0.217
reason_noquestion_notime 3.771 0.152 -0.403 0.108 -0.895 0.088 0.251 -1.608 0.620 0.438 -0.948 2.188 0.800 0.775 0.554 0.034 0.262 2.116 1.204 0.001 0.353 3.406 0.374 0.543 0.615 0.608 0.217
reason_noquestion_introvert 4.035 0.133 0.515 0.072 -0.047 1.077 0.287 1.797 -0.426 0.702 -2.604 1.752 1.111 -0.383 -0.347 0.192 0.267 -1.303 -0.961 0.041 0.471 -2.042 -1.024 0.189 0.780 -1.312 0.217
more_comfort_asking_know_speaker_rating 2.759 0.252 0.304 0.123 -0.081 0.691 0.197 1.545 -0.187 0.778 -1.512 1.138 0.665 -0.282 -0.334 0.094 0.199 -1.679 -0.836 0.006 0.304 -2.752 -0.776 0.119 0.497 -1.562 0.318
reason_noquestion_clever 2.737 0.254 0.418 0.107 -0.091 0.928 0.260 1.610 0.516 0.523 -1.067 2.099 0.808 0.639 -0.815 0.001 0.250 -3.258 -1.789 0.000 0.499 -3.584 -0.449 0.432 0.571 -0.787 0.318
reason_question_relevance 1.289 0.525 0.072 0.753 -0.375 0.518 0.228 0.314 0.929 0.280 -0.757 2.615 0.860 1.080 0.096 0.673 0.228 0.422 -0.052 0.875 0.334 -0.157 -1.029 0.088 0.604 -1.705 0.618
reason_question_appreciate 0.984 0.612 -0.243 0.365 -0.769 0.283 0.268 -0.907 -0.597 0.589 -2.763 1.568 1.105 -0.541 0.144 0.608 0.280 0.514 0.496 0.192 0.380 1.304 0.418 0.497 0.615 0.679 0.679
reason_question_understand 0.841 0.657 -0.049 0.849 -0.553 0.455 0.257 -0.190 -0.748 0.348 -2.310 0.814 0.797 -0.938 0.207 0.426 0.260 0.796 -0.445 0.204 0.351 -1.271 0.934 0.230 0.778 1.200 0.691
reason_noquestion_misunderstand 0.363 0.834 0.037 0.879 -0.435 0.508 0.240 0.152 0.484 0.550 -1.101 2.069 0.809 0.598 -0.958 0.000 0.240 -3.995 -1.496 0.000 0.402 -3.718 0.019 0.972 0.540 0.036 0.834

Including gender as an explanatory variable included the fit of the model for most conditions. In order of highest to lowest effects, women are more comfortable asking questions they are more comfortable asking questions if: women are represented in the audience, the speaker is a woman, the host is a woman, and when the audience is smaller. Women are not more comfortable asking a question if they know the speaker. Despite the low sample size of non-binary respondents, also people with non-binary gender identities (N = 7) are more comfortable asking questions if their own genders identity is represented in the audience, if the speakers is of their own gender identity and if the host is of their own gender identity. Based on self-reports, female and non-binary representation in various contexts at the congress appears to make women and non-binary researchers more comfortable in asking questions, respectively.

Although no motivations were significantly affected by gender, there were two motivations that were (almost) significantly affected by gender after correcting for multiple testing: “Afraid I would not be able to phrase/articulate my question well”, and “I did not have the confidence” (LRT FDR-corrected q-value < 0.1). These two motivations were also affected by career stage, where early career researchers are more likely to tick the phrasing/articulation-related and confidence-related hesitation compared to late career researchers.

5.1.4 Analysis of motivations and hesitations that predict question asking

Next, we fitted 19 more binomial GLMs to test which one of the motivations and hesitations was a statistically significant predictor for the probability of a person asking a question during the congress. The dependent variable in these models was therefore the response to the question “Did you ask one or more questions during Q&A sessions?” (1 for yes, 0 for no) and the response variable the binomial response whether the motivation or hesitation was ticked (1) or not (0), while again controlling for career stage.

# loop over reasons again but without comfort

# take out 'other'
other <- grep("_other", colnames(survey)[reasons]) 
reasons <- reasons[-other]

# take out not chosen
notchosen <- grep("reason_noquestion_notchosen", colnames(survey)[reasons]) 
reasons <- reasons[-notchosen] #exclude 

output_asking <- data.frame()

for (i in reasons){
  subset <- survey %>% filter(!is.na(.[[i]]) & !is.na(colnames(survey)[i]) &
                     !is.na(gender) & !is.na(career_3cat) &
                     !is.na(ask_questions))


  # null model
  formula_null <- as.formula(paste0("ask_questions ~ gender +career_3cat"))
  null <- glm(formula_null, data = subset, family = "binomial")
  # alternative model with gender
  formula_alt <- as.formula(paste0("ask_questions ~ ", 
                                   colnames(survey)[i], "+ gender + career_3cat")) 
  model <- glm(formula_alt, data = subset, family = "binomial")
  
  # summary
  sum <- summary(model)$coefficients
  confint <- (as.data.frame(summ(model, confint=TRUE)$coeftable))
  
  # lrt
  lrt <- anova(null, model, test = "Chisq")
  lrt_sig <- lrt$`Pr(>Chi)`[2]
  lrt_chisq <- lrt$`Deviance`[2]
  
  out <- data.frame(predictor = colnames(survey)[i],
                    lrt_chisq = lrt_chisq,
                    lrt_sig = lrt_sig,
                    coef_reason = sum[2,"Estimate"],
                    se_reason = sum[2,"Std. Error"],
                    z_reason = sum[2, "z value"],
                    lower_reason = confint[2,"2.5%"],
                    upper_reason = confint[2,"97.5%"],
                    pval_reason = sum[2,"Pr(>|z|)"],
                    coef_female = sum["genderFemale","Estimate"],
                    pval_female = sum["genderFemale","Pr(>|z|)"],
                    coef_non_binary = sum["genderNon-binary","Estimate"],
                    pval_non_binary = sum["genderNon-binary","Pr(>|z|)"],
                    coef_mid = sum["career_3catMid career","Estimate"],
                    pval_mid = sum["career_3catMid career","Pr(>|z|)"],
                    coef_late = sum["career_3catLate career","Estimate"],
                    pval_late = sum["career_3catLate career","Pr(>|z|)"],
                    coef_other = sum["career_3catOther","Estimate"],
                    pval_other = sum["career_3catOther","Pr(>|z|)"]
                    
  )
  
  output_asking <- rbind(output_asking, out)
 
}

## add a column if this variable has a significant effect of gender
sig_gender <- subset(output_reasons, lrt_qval < 0.05)

output_asking <- output_asking %>% mutate(gender_effect = case_when(
  output_asking$predictor %in% sig_gender$response ~ "gender_effect",
  TRUE ~ "no_gender_effect"
))

# add a column with the number of people saying this factor was important
n_responses <- data.frame()
for (i in reasons){
  summary <- as.data.frame(table(survey[,i], survey$gender))
  response_yes_female <- subset(summary, Var1 == "1" & Var2 == "Female")$Freq
  response_yes_male <- subset(summary, Var1 == "1" & Var2 == "Male")$Freq
  response_yes_non_binary <- subset(summary, Var1 == "1" & Var2 == "Non-binary")$Freq
  
  predictor <- data.frame(predictor = colnames(survey)[i],
                          n_yes_female = response_yes_female,
                          n_yes_male = response_yes_male,
                          n_yes_non_binary = response_yes_non_binary)
  n_responses <- rbind(n_responses, predictor)
}

output_asking <- left_join(output_asking, n_responses, by = "predictor")

# round
output_asking <- output_asking %>% mutate_if(is.numeric, round, digits=3)

# subset significant predictors of question asking

output_asking$lrt_qval <- p.adjust(output_asking$lrt_sig,
                                   method="fdr",
                                   n=nrow(output_asking))

output_asking %>% arrange(lrt_qval) %>% kbl() %>%
  kable_classic_2()
predictor lrt_chisq lrt_sig coef_reason se_reason z_reason lower_reason upper_reason pval_reason coef_female pval_female coef_non_binary pval_non_binary coef_mid pval_mid coef_late pval_late coef_other pval_other gender_effect n_yes_female n_yes_male n_yes_non_binary lrt_qval
reason_noquestion_notime 12.415 0.000 0.979 0.290 3.382 0.412 1.547 0.001 -0.435 0.094 1.195 0.287 0.733 0.003 2.488 0 1.060 0.085 no_gender_effect 65 38 3 0.0000000
reason_noquestion_introvert 23.122 0.000 -1.270 0.269 -4.715 -1.798 -0.742 0.000 -0.412 0.117 1.168 0.293 0.762 0.002 2.569 0 0.929 0.136 no_gender_effect 68 20 1 0.0000000
reason_noquestion_private 19.842 0.000 -1.038 0.237 -4.383 -1.502 -0.574 0.000 -0.434 0.096 0.928 0.403 0.822 0.001 2.474 0 1.224 0.048 no_gender_effect 125 46 1 0.0000000
reason_question_interest 10.324 0.001 1.073 0.340 3.153 0.406 1.741 0.002 -0.473 0.069 1.156 0.298 0.902 0.000 2.583 0 1.052 0.086 no_gender_effect 220 102 7 0.0037500
reason_question_voiceheard 7.029 0.008 1.679 0.763 2.200 0.183 3.175 0.028 -0.495 0.055 0.971 0.390 0.747 0.002 2.651 0 0.993 0.107 no_gender_effect 12 9 2 0.0171429
reason_noquestion_intimidated_audience 7.328 0.007 -0.766 0.284 -2.696 -1.323 -0.209 0.007 -0.428 0.098 1.277 0.253 0.670 0.007 2.546 0 0.975 0.112 no_gender_effect 59 14 1 0.0171429
reason_noquestion_confidence 6.979 0.008 -0.698 0.265 -2.636 -1.216 -0.179 0.008 -0.422 0.103 1.268 0.257 0.760 0.002 2.470 0 1.035 0.092 no_gender_effect 68 16 1 0.0171429
reason_noquestion_clever 5.329 0.021 -0.559 0.242 -2.311 -1.033 -0.085 0.021 -0.464 0.072 1.377 0.219 0.714 0.004 2.492 0 1.056 0.085 no_gender_effect 89 29 3 0.0393750
reason_question_understand 3.936 0.047 0.511 0.258 1.983 0.006 1.016 0.047 -0.515 0.044 1.399 0.212 0.790 0.001 2.746 0 1.020 0.094 no_gender_effect 186 82 4 0.0783333
reason_question_appreciate 2.247 0.134 0.426 0.288 1.479 -0.138 0.990 0.139 -0.501 0.050 1.295 0.243 0.791 0.001 2.638 0 1.059 0.081 no_gender_effect 53 28 1 0.2010000
reason_noquestion_phrasing 1.803 0.179 -0.335 0.249 -1.346 -0.822 0.153 0.178 -0.452 0.081 1.361 0.223 0.759 0.002 2.573 0 1.028 0.092 gender_effect 88 20 3 0.2440909
reason_noquestion_relevance 0.848 0.357 0.215 0.234 0.919 -0.243 0.673 0.358 -0.503 0.049 1.331 0.231 0.816 0.001 2.701 0 1.054 0.083 no_gender_effect 100 50 1 0.4462500
reason_noquestion_intimidated_setting 0.184 0.668 -0.171 0.398 -0.430 -0.952 0.609 0.667 -0.506 0.048 1.311 0.240 0.787 0.001 2.667 0 1.106 0.069 no_gender_effect 27 6 2 0.7707692
reason_question_relevance 0.028 0.866 -0.038 0.229 -0.168 -0.486 0.409 0.866 -0.514 0.044 1.272 0.252 0.798 0.001 2.661 0 1.079 0.076 no_gender_effect 129 55 5 0.8660000
reason_noquestion_misunderstand 0.042 0.838 -0.048 0.235 -0.205 -0.510 0.413 0.838 -0.515 0.043 1.271 0.252 0.786 0.001 2.646 0 1.090 0.073 no_gender_effect 105 44 4 0.8660000 
write.csv(output_asking, file = "../results/question-asking/model_qa_conditions_motivations_gender_effect.csv",
          quote=F, row.names = F)

5.2 Observational data

Next, we investigated if we observe this same pattern in the observational data on question asking behaviour. We introduced a series of potential explanatory variables in the model described in the previous section that identified the gender disparity in question asking (Section 3.1). More specifically, we built five binomial GLMMs with one of the following five factors as an independent variable, 1) speaker gender, 2) gender proportion of the audience, 3) total audience size, 4) room size, or 5) host gender. These five factors therefore represent the same five conditions that were investigated above using the survey data, respectively.

Because of the result of the previous section, namely that the gender disparity in question asking arises due to women raising their hands less often (Section 4.2), we only modelled the effect of these five factors on the number of hands that were raised as a dependent variable. We again accounted for the gender proportion of the audience and the nonindependence of talks within a session. We assess significance of each factor through a likelihood ratio test.

# speaker gender
data_hands$speaker_gender <- factor(data_hands$speaker_gender, levels=c("M", "F"))

m_speaker_gender <- glmer(cbind(hands_women, hands_men) ~ speaker_gender + (1|session_id/talk_id), family = "binomial", offset=boot::logit(audience_women_prop), data = subset(data_hands, !is.na(speaker_gender)))

summary(m_speaker_gender)
Generalized linear mixed model fit by maximum likelihood (Laplace
  Approximation) [glmerMod]
 Family: binomial  ( logit )
Formula: 
cbind(hands_women, hands_men) ~ speaker_gender + (1 | session_id/talk_id)
   Data: subset(data_hands, !is.na(speaker_gender))
 Offset: boot::logit(audience_women_prop)

     AIC      BIC   logLik deviance df.resid 
   574.9    590.1   -283.4    566.9      327 

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-1.57063 -0.93767 -0.03456  0.91578  1.79208 

Random effects:
 Groups             Name        Variance Std.Dev.
 talk_id:session_id (Intercept) 0.0541   0.2326  
 session_id         (Intercept) 0.1135   0.3368  
Number of obs: 331, groups:  talk_id:session_id, 121; session_id, 24

Fixed effects:
                 Estimate Std. Error z value Pr(>|z|)    
(Intercept)     -0.568417   0.164963  -3.446  0.00057 ***
speaker_genderF -0.003861   0.197053  -0.020  0.98437    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
            (Intr)
spekr_gndrF -0.697
m_speaker_gender_null <- glmer(cbind(hands_women, hands_men) ~ 1 + (1|session_id/talk_id), family = "binomial", offset=boot::logit(audience_women_prop), data = subset(data_hands, !is.na(speaker_gender)))

## helper function to collect output
m_speaker_gender_out <- collect_out(model = m_speaker_gender, null = m_speaker_gender_null, name="qa_speaker_gender", n_factors=1, type="qa", save = "yes", dir = "../results/question-asking/")

m_speaker_gender_out %>% t() %>% kbl() %>%
  kable_classic_2()
model_name qa_speaker_gender
AIC 574.872
n_obs 331
lrt_pval 0.984
lrt_chisq 0
intercept_estimate -0.568
intercept_estimate_prop 0.362
intercept_pval 0.001
intercept_ci_lower -0.892
intercept_ci_higher -0.245
n_factors 1
est_speaker_genderF -0.004
lowerCI_speaker_genderF -0.39
higherCI_speaker_genderF 0.382
pval_speaker_genderF 0.984
zval_speaker_genderF -0.02 
# not significant

# audience gender proportion

m_audience_gender <- glmer(cbind(hands_women, hands_men) ~ audience_women_prop + (1|session_id/talk_id), family = "binomial", offset=boot::logit(audience_women_prop), data = subset(data_hands, !is.na(audience_women_prop)))

summary(m_audience_gender)
Generalized linear mixed model fit by maximum likelihood (Laplace
  Approximation) [glmerMod]
 Family: binomial  ( logit )
Formula: 
cbind(hands_women, hands_men) ~ audience_women_prop + (1 | session_id/talk_id)
   Data: subset(data_hands, !is.na(audience_women_prop))
 Offset: boot::logit(audience_women_prop)

     AIC      BIC   logLik deviance df.resid 
   606.4    621.8   -299.2    598.4      345 

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-1.56732 -0.95301 -0.01054  0.94821  1.63888 

Random effects:
 Groups             Name        Variance Std.Dev.
 talk_id:session_id (Intercept) 0.08359  0.2891  
 session_id         (Intercept) 0.05665  0.2380  
Number of obs: 349, groups:  talk_id:session_id, 127; session_id, 24

Fixed effects:
                    Estimate Std. Error z value Pr(>|z|)
(Intercept)           0.5228     0.7734   0.676    0.499
audience_women_prop  -1.7424     1.2090  -1.441    0.150

Correlation of Fixed Effects:
            (Intr)
adnc_wmn_pr -0.991
m_audience_gender_null <- glmer(cbind(hands_women, hands_men) ~ 1 + (1|session_id/talk_id), family = "binomial", offset=boot::logit(audience_women_prop), data = subset(data_hands, !is.na(audience_women_prop)))

## helper function to collect output
m_audience_gender_out <- collect_out(model = m_audience_gender, null = m_audience_gender_null, name="qa_audience_gender", n_factors=1, type="qa", save = "yes", dir = "../results/question-asking/")

m_audience_gender_out %>% t() %>% kbl() %>%
  kable_classic_2()
model_name qa_audience_gender
AIC 606.38
n_obs 349
lrt_pval 0.154
lrt_chisq 2.028
intercept_estimate 0.523
intercept_estimate_prop 0.628
intercept_pval 0.499
intercept_ci_lower -0.993
intercept_ci_higher 2.039
n_factors 1
est_audience_women_prop -1.742
lowerCI_audience_women_prop -4.112
higherCI_audience_women_prop 0.627
pval_audience_women_prop 0.15
zval_audience_women_prop -1.441 
# almost significant

# host gender
data_hands$host_1_gender <- factor(data_hands$host_1_gender, levels=c("M", "F"))

m_host_gender <- glmer(cbind(hands_women, hands_men) ~ host_1_gender + (1|session_id/talk_id), family = "binomial", offset=boot::logit(audience_women_prop), data = subset(data_hands, !is.na(host_1_gender)))

summary(m_host_gender)
Generalized linear mixed model fit by maximum likelihood (Laplace
  Approximation) [glmerMod]
 Family: binomial  ( logit )
Formula: 
cbind(hands_women, hands_men) ~ host_1_gender + (1 | session_id/talk_id)
   Data: subset(data_hands, !is.na(host_1_gender))
 Offset: boot::logit(audience_women_prop)

     AIC      BIC   logLik deviance df.resid 
   546.4    561.5   -269.2    538.4      312 

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-1.6184 -0.9117 -0.1024  0.9602  1.9401 

Random effects:
 Groups             Name        Variance Std.Dev.
 talk_id:session_id (Intercept) 0.03963  0.1991  
 session_id         (Intercept) 0.09504  0.3083  
Number of obs: 316, groups:  talk_id:session_id, 115; session_id, 23

Fixed effects:
               Estimate Std. Error z value Pr(>|z|)    
(Intercept)     -0.8204     0.1948  -4.212 2.54e-05 ***
host_1_genderF   0.2963     0.2444   1.213    0.225    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
            (Intr)
hst_1_gndrF -0.795
m_host_gender_null <- glmer(cbind(hands_women, hands_men) ~ 1 + (1|session_id/talk_id), family = "binomial", offset=boot::logit(audience_women_prop), data = subset(data_hands, !is.na(host_1_gender)))

## helper function to collect output
m_host_gender_out <- collect_out(model = m_host_gender, null = m_host_gender_null, name="qa_host_gender", n_factors=1, type="qa", save = "yes", dir = "../results/question-asking/")

m_host_gender_out %>% t() %>% kbl() %>%
  kable_classic_2()
model_name qa_host_gender
AIC 546.441
n_obs 316
lrt_pval 0.217
lrt_chisq 1.522
intercept_estimate -0.82
intercept_estimate_prop 0.306
intercept_pval 0
intercept_ci_lower -1.202
intercept_ci_higher -0.439
n_factors 1
est_host_1_genderF 0.296
lowerCI_host_1_genderF -0.183
higherCI_host_1_genderF 0.775
pval_host_1_genderF 0.225
zval_host_1_genderF 1.213 
# not significant 

# audience size

m_audience_size <- glmer(cbind(hands_women, hands_men) ~ audience_total + (1|session_id/talk_id), family = "binomial", offset=boot::logit(audience_women_prop), data = subset(data_hands, !is.na(audience_total)))

summary(m_audience_size)
Generalized linear mixed model fit by maximum likelihood (Laplace
  Approximation) [glmerMod]
 Family: binomial  ( logit )
Formula: 
cbind(hands_women, hands_men) ~ audience_total + (1 | session_id/talk_id)
   Data: subset(data_hands, !is.na(audience_total))
 Offset: boot::logit(audience_women_prop)

     AIC      BIC   logLik deviance df.resid 
   608.4    623.8   -300.2    600.4      345 

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-1.50007 -0.94290 -0.00701  0.92802  1.81278 

Random effects:
 Groups             Name        Variance Std.Dev.
 talk_id:session_id (Intercept) 0.10857  0.3295  
 session_id         (Intercept) 0.05458  0.2336  
Number of obs: 349, groups:  talk_id:session_id, 127; session_id, 24

Fixed effects:
                Estimate Std. Error z value Pr(>|z|)  
(Intercept)    -0.524947   0.268331  -1.956   0.0504 .
audience_total -0.001031   0.004318  -0.239   0.8113  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
            (Intr)
audienc_ttl -0.916
m_audience_size_null <- glmer(cbind(hands_women, hands_men) ~ 1 + (1|session_id/talk_id), family = "binomial", offset=boot::logit(audience_women_prop), data = subset(data_hands, !is.na(audience_total)))

## helper function to collect output
m_audience_size_out <- collect_out(model = m_audience_size, null = m_audience_size_null, name="qa_audience_size", n_factors=1, type="qa", save = "yes", dir = "../results/question-asking/")

m_audience_size_out %>% t() %>% kbl() %>%
  kable_classic_2()
model_name qa_audience_size
AIC 608.351
n_obs 349
lrt_pval 0.811
lrt_chisq 0.057
intercept_estimate -0.525
intercept_estimate_prop 0.372
intercept_pval 0.05
intercept_ci_lower -1.051
intercept_ci_higher 0.001
n_factors 1
est_audience_total -0.001
lowerCI_audience_total -0.009
higherCI_audience_total 0.007
pval_audience_total 0.811
zval_audience_total -0.239 
# not significant 

# room size
data_hands$room_size <- factor(data_hands$room_size, levels = c( "Large","Small", "Medium"))
                               
m_room_size <- glmer(cbind(hands_women, hands_men) ~ room_size + (1|session_id/talk_id), family = "binomial", offset=boot::logit(audience_women_prop), data = subset(data_hands, !is.na(room_size)))

summary(m_room_size)
Generalized linear mixed model fit by maximum likelihood (Laplace
  Approximation) [glmerMod]
 Family: binomial  ( logit )
Formula: cbind(hands_women, hands_men) ~ room_size + (1 | session_id/talk_id)
   Data: subset(data_hands, !is.na(room_size))
 Offset: boot::logit(audience_women_prop)

     AIC      BIC   logLik deviance df.resid 
   609.0    628.2   -299.5    599.0      344 

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-1.5545 -0.9263 -0.0012  0.9380  1.7043 

Random effects:
 Groups             Name        Variance Std.Dev.
 talk_id:session_id (Intercept) 0.11381  0.3374  
 session_id         (Intercept) 0.02748  0.1658  
Number of obs: 349, groups:  talk_id:session_id, 127; session_id, 24

Fixed effects:
                Estimate Std. Error z value Pr(>|z|)   
(Intercept)      -0.5521     0.1725  -3.201  0.00137 **
room_sizeSmall   -0.1511     0.2313  -0.653  0.51356   
room_sizeMedium   0.1678     0.2680   0.626  0.53117   
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
            (Intr) rm_szS
room_szSmll -0.745       
room_sizMdm -0.626  0.471
m_room_size_null <- glmer(cbind(hands_women, hands_men) ~ 1 + (1|session_id/talk_id), family = "binomial", offset=boot::logit(audience_women_prop), data = subset(data_hands, !is.na(room_size)))

## helper function to collect output
m_room_size_out <- collect_out(model = m_room_size, null = m_room_size_null, name="qa_room_size", n_factors=2, type="qa", save = "yes", dir = "../results/question-asking/")

m_room_size_out %>% t() %>% kbl() %>%
  kable_classic_2
model_name qa_room_size
AIC 608.974
n_obs 349
lrt_pval 0.488
lrt_chisq 1.434
intercept_estimate -0.552
intercept_estimate_prop 0.365
intercept_pval 0.001
intercept_ci_lower -0.89
intercept_ci_higher -0.214
n_factors 2
est_room_sizeSmall -0.151
lowerCI_room_sizeSmall -0.605
higherCI_room_sizeSmall 0.302
pval_room_sizeSmall 0.514
zval_room_sizeSmall -0.653
est_room_sizeMedium 0.168
lowerCI_room_sizeMedium -0.357
higherCI_room_sizeMedium 0.693
pval_room_sizeMedium 0.531
zval_room_sizeMedium 0.626 
# not significant

The results based on the observational data contrast our findings from the survey. None of the five factors significantly improved the fit of the models, indicating that they do not significantly affect the probability that a woman raises her hand (LRT p-values > 0.05). The only variable that got close to being significant was the gender proportion of the audience, with an LRT p-value of 0.154, where a higher number of women in the audience was associated with less women raising their hand (estimate = -1.742), opposite of expectations.

Therefore, although self-reports indicate that women feel more comfortable asking questions in certain situations, observational data collected at the congress indicate that in practice, these conditions are in fact not associated with the probability that a woman asks a question.