Why It’s Important to Analyze Research Data by Sex with Nicole Woitowich, PhD

Erin Spain, MS: For years, much of biomedical research, especially animal research, relied primarily on male subjects, often missing critical differences in how diseases and medications affect women. In 2016, a major NIH policy shifted the rules requiring scientists to consider sex as a biological variable in federally funded research with the goal of creating more rigorous science and precise medicine for everyone. But nearly a decade later, has science truly changed? Joining me to discuss that question is Nicole Woitowich, the executive director of NUCATS, the Clinical and Translational Sciences Institute at Northwestern University. She's the corresponding author of a study published in Nature Communications Medicine, examining how well biomedical research is incorporating sex differences into study design and analysis. Welcome, Niki. 

Nicole Woitowich, PhD: Thanks so much for having me, Erin. 

Erin Spain, MS: Some listeners may not know that Northwestern University actually played a pretty important role in this 2016 NIH policy around sex as a biological variable in biomedical research. Can you tell me about that and your role in this movement back in 2016? 

Nicole Woitowich, PhD: Faculty at Northwestern have been engaged in this area for quite some time, and we really had a large mass of faculty who advocated for this policy to be created. In 2016 when the policy was announced, this was a really good opportunity for combining some of my interests in the science of biomedical science as well as science policy to really start looking at implementation over time. And some of the work that we've done started really early on in 2016, asking study section members how they felt about the policy, how it was being evaluated in the study section. And then later on, continuing to track progress towards policy implementation to some of the work that we'll talk about today. 

Erin Spain, MS: At the time when this was implemented in 2016, there was this sense that this sex inclusion could revolutionize research and help advance precision medicine. And as you said, you and your team, you've wanted to see now a decade later how much progress has been made. Why is it important to do this kind of research and follow up after such a big policy change? 

Nicole Woitowich, PhD: So I think it's great that we're including both sexes in research, and that's becoming more common, but without analyzing our data by sex, that really limits our scientific impact and generalizability of our findings for the health of all people. One of the things I find fascinating about this is that as we're evaluating this policy, we're looking at in real time how research practices and behaviors shift. So we are looking at how scientists ask their questions, how they design their studies. Evaluate and report their data. And some good news that we found is that over time, more and more scientists and investigators are including both sexes in their research. As you might know, historically, a lot of biomedical research was done primarily in males, and that includes male cells, male animals, and really in the past, a lot of clinical data was also based on male data. So this really was a kind of behavioral cultural shift to really emphasize the need for research studies that included both sexes. What we found is that over the past 10 years, we have seen an improvement in sex inclusion. More than half of NIH studies include both sexes now, which is great. However, there is another part that we really need to talk about. And that's that, while scientists are including both sexes more frequently, they're not necessarily analyzing data by sex. And that's, I think, a really big issue. 

Erin Spain, MS: Now you published about this in Nature Communications Medicine and you and your team looked at over 500 NIH funded papers that revealed this gap. Why is this distinction inclusion versus analysis so critical to understand? 

Nicole Woitowich, PhD: Here's the thing, when you include both sexes, but you don't analyze your data by sex, you can't really move towards this ideal of precision medicine. And what I mean by that is that there may be sex differences between groups that are being masked by including and analyzing your data just as one full set. When you break down data by sex, you may be able to pick out these nuances. There may be sex differences in the topic or disease area you're studying that are being left unreported. And so then in the future when somebody wants to repeat your study, they don't know if this work has already been looked at and there are no sex differences. Or if they haven't been analyzed in general. So those are two distinct problems, which lead to more studies being needed and more federal funding to support this type of work. I really think it's an economic issue as much as it's a scientific issue. 

Erin Spain, MS: What are some of the consequences when researchers combine the male and female data together without looking at the sex differences separately? You said it's hard to replicate these studies, but what are some of the other consequences? 

Nicole Woitowich, PhD: Again, we don't know if there are ways we can develop better treatments and therapies for all people. Because if we're pooling our data together, as I mentioned, there are ways in which differences between sexes might arise, but we're not actually seeing that. And where this has played out more broadly is in women's health because historically, a lot of our research is focused on male subjects. There's still so many gaps in our understanding of women's health. And while it can be said that by studying both sexes, analyzing data by sex would surely help improve our knowledge in women's health. I also wanna emphasize that it's really just about improving health for everyone. 'Cause we can create better medicines, better treatments, and better therapies for everyone when we can really move towards the ideal of personalized medicine. 

Erin Spain, MS: Is there an example you can give where it does make a difference when we analyze by sex? 

Nicole Woitowich, PhD: Right, there's a study I often talk about as an example, and when we think about sex differences, often, many times people think about classical differences in reproductive health, but that's not the case. There are sex differences across our body systems. A handful of years back, a group was looking at sex differences in glioblastoma treatment. So that's a very deadly form of brain cancer and treatments are very limited. And so in this article they said we decided to break down our data by sex. And they found that women responded better to treatment than men. And they went on to develop more questions and ideas to study how and why that was happening. So looking again from the clinical data they were seeing going down into the molecular mechanisms of how this therapy was working. And they found some pretty significant sex differences. And so then the authors concluded, Hey, we need to learn more about this so that we can make treatment more efficacious for everyone, not just one group. So that's just one example of a treatment area where we can learn a lot more by analyzing our data by sex. 

Erin Spain, MS: One thing that stood out is how inconsistent the implementation still is. Even among studies that included both sexes, many didn't report sample sizes by sex or explain why they skipped the sex-based analysis. What impact does that have on the rigor and reproducibility of many studies taking place today? 

Nicole Woitowich, PhD: One of my biggest pet peeves of doing this research is finding out that there are a handful of investigators who do not report the sex of their subjects at all. So that's one huge big red flag when we can't even find out the sex of their research subjects. In our study, we found that 13% of studies failed to report the sex of their subjects at all, which is a huge issue. Typically, this occurred in animal-based studies where they would state things such as we used mice or we used rats and that doesn't help us when we go to replicate a study. Another area where there were studied flaws, I would say is they would not report the sample size by sex. And so that was in 17% of studies we evaluated, so that's when they would say, we used mice of both sexes. Again, when I go to repeat a study, did you use 15 male mice and two female mice? And then I'm somebody trying to repeat your work and I'm getting different data and I can't figure out, is that because I'm using a different composition of a sample size? Is it because of my reagents? Is it because of something entirely different? I then have to repeat this work and repeat this work. It's not rigorous science and it's really hindering reproducibility. And another issue is it's not transparent. So these I think should just be mandatory reporting requirements that aren't being met. 

Erin Spain, MS: What is it about the animal studies? Why do you think it is that the animal studies continue to lag behind human studies in this area? 

Nicole Woitowich, PhD: So typically human studies were reporting the sex of their subjects. Were more likely to analyze data by sex. And I think that really when you're in a clinical space, you're seeing people and you can understand how this might impact people differently. Yet when you're working in the basic sciences and I'm a biochemist by training. I worked with mice, I worked with rats. I think you lose that connection of, we really should be looking at this in both sexes. 'Cause even in this early stage of discovery, this is what leads to new treatments and therapies, and that will eventually translate into the clinic. And if we are not looking at this in the early stages of discovery, we are hindering rigor and reproducibility. But then again, it becomes another economic issue when drugs fail once they reach human studies. And it takes over 15 years for a new drug or therapy on average to go to market. With millions of dollars being invested in that process. If we can do better science at the bench side, then I think we can make more treatments for people more quickly by cutting out some of those steps where we have to go back and repeat our work. 

Erin Spain, MS: Why was it so common that male animals, male mice, for example, were typically used in research studies before 2016? 

Nicole Woitowich, PhD: So there is a common misconception that there is hormonal variation in using female animals. And quite frankly, even humans in research studies and investigators wanted to reduce any extraneous signals in their data. And so to avoid the hormonal variation, they just use males. And we've shown over the past decade plus that this is a really poor excuse to exclude females from research. A study a handful of years back actually looked at that and really tried to debunk this myth and showed that in neuroscience studies there really wasn't a difference when including females. Now I think there is something to be said about hormonal variation and in fact, if you are so concerned that there are going to be hormonal variations in your research, maybe you should be looking at it ' cause that would potentially indicate that there is some sort of sex difference at play. 

Erin Spain, MS: Something very interesting in this study is you found that teams led by women were more likely to analyze data by sex. What does this reveal about, the importance of diversity in scientific leadership? 

Nicole Woitowich, PhD: One of the things we found in this study was women first and last authors were twice as likely to analyze their data by sex compared to men. And we've seen this happen in other studies as well. I think it has to do with the fact that women are more aware of their own physiology. They're aware of the gaps that exist in women's health, right? So they're acutely familiar with this issue. That's not to discredit our male colleagues who work in these spaces as well. But I think it's a personal issue that it's just understood, and I think this is an area where we could really benefit from some more research to figure out why women analyze their data in this way more frequently than men. Is it because of that just social cultural awareness? That would be my guess. , women are still underrepresented in the biomedical research enterprise, and we see that again within our own study, that the percentage of male first and last authors is still higher than the percentage of articles authored by women first and last authors. So I think this will also be a driver of sex inclusion and sex-based analyses as women still rise to leadership positions within the biomedical research enterprise I think so too will sex-based analyses and reporting as that shift in the composition of our teams happens. 

Erin Spain, MS: So you've done similar work looking at the publication record and looking at sex inclusion and sex-based analysis from publications. But this study was the first that tied it to NIH-funded grants and you were actually able to look at how different NIH-funded institutes and centers relate back to this policy. What did you find? 

Nicole Woitowich, PhD: We found some really interesting differences across NIH institutes and centers that really, I think, shows how the different practices and types of research investigators conduct might also influence how sex is included and or analyzed and reported on in their data. So some of our results varied widely by NIH institute or center. For example, only 26% of NCI National Cancer Institute studies analyze their data by sex. And we know there are sex differences in cancer diagnosis, progression, response to treatment. So I think this is an area and a call to action where we can really improve upon. 

Erin Spain, MS: As you were going through and collecting this data and looking at this over the past 10 years, were you surprised by these results? 

Nicole Woitowich, PhD: It takes time to make significant changes in the way that we do things. So I think I am not entirely discouraged. When you implement a new policy, there's always going to be changes to the status quo. And I think the improvements we've seen in sex inclusion are great, and my hope is that eventually sex-based analyses will follow and this will become a more routine practice. One of the areas I think we can improve on is education and training. I have colleagues—Donna Maney from Emory, she has been looking at how scientists analyze their data by sex and her and her team have found that they often do not use the right biostatistical tests and methods. So even when they report sex differences, that might not actually be statistically accurate. So clearly I think there's a need for increased education and training in how to do it. So that's another thing. Maybe we just don't have enough skills and expertise in this area. I just wanna give a shout out. I always collaborate with the Biostatistics Collaboration Center because they're really the experts and can help us design more rigorous testing and methods at the onset. So shout out to those colleagues who help me because I don't have that training. I leverage the expertise of others who do. But if you are working solo and you don't know how to do this, there's a lot of opportunities for error or just not doing it correctly. And so maybe that's a reason why we haven't seen a greater uptake in that. Another area is just, I think, practice. So a lot of the way we're trained in the lab is based on the way things have been done. And this protocol has worked for me. It worked for my colleague. I'm gonna keep doing it this way. We just continue to do the same thing and sometimes change can be difficult, and so I hope that we'll get better in the future. I do think that there is a great deal of opportunity by studying sex differences, analyzing our data by sex. There's still so much we don't know that this area holds a lot of promise for research, new avenues of investigation. So I like to encourage trainees, early career folks, to really consider, have you looked at how your topic area of interest, if there are sex differences, and ask yourself, is it because nobody's explored that yet? Or is it truly that there aren't, but really you have to go in and dig for that information. 

Erin Spain, MS: What role do publications and journals have, do you think, in helping to implement this policy?  

Nicole Woitowich, PhD: The fact that you've got studies that don't even say the sex of their subjects at all, those are getting published in peer-reviewed journals. So it's up to the editors, the reviewers, which I don't wanna put any additional burden on reviewers, but, really, this needs to be evaluated more critically at the publication stage, and there are reporting guidelines such as the SAGER reporting guidelines where this is meant to be captured. Not all journals use those guidelines and perhaps some do. And even though those reporting guidelines are in place, it's not actually making it to print. So you've got a couple issues that you're facing there, but I do think as we're seeing with some other policy changes coming from the NIH, things that impact publishing tend to get a more robust reaction because we still rely on publications as our academic currency. And I think if you were to put in more stringent measures at that juncture to say, are you actually doing this? And if so, provide the most basic information such as the number of subjects by sex, that would be really a great start. 

Erin Spain, MS: Your work as executive director of the NUCATS Institute really does focus on helping scientists learn new skills and also translate science into real-world impact. Tell me about what's happening at NUCATS to help close this gap. Is there anything specifically happening for your scientists to make sex-inclusive science part of the standard practice at Northwestern? 

Nicole Woitowich, PhD: I think this is an area where we collaborate with our colleagues in the Biostatistics Collaboration Center on providing guidance on how to do this from a best practices standpoint We talk about this in education and training. Our grant mechanisms that we offer through NUCATS also asks investigators to consider how sex is a biological variable is being considered in their proposals. So again, by embedding those, not just in NIH funding mechanisms, but in our internal mechanisms so that hopefully folks are starting to see this as a common practice to use when designing their studies. 

Erin Spain, MS: So it's been 10 years since this policy went into effect. What do you hope will happen in the next 10 years? Where would you like to see the status of this policy in 2036? 

Nicole Woitowich, PhD: As we continue to track this over time, I would love to see sex inclusion increase more. Recognizing, and this is something we haven't quite touched on, that not every study will be sex inclusive by the nature of the study themself, whether they're focusing on a sex specific condition or not. But I would love to see more sex inclusion happening, especially in those using animal models. That's where sex inclusion still lags significantly behind, as well as in reporting. And we showed in our most recent study, huge differences between the studies involving human subjects versus those involving animal models. This is an area where we can have improvement, especially working with colleagues in those areas. I would also love for colleagues who use primary tissue samples being derived from human participants to report the sex of their subjects because that also plays a big role and we saw a lot of cases where that wasn't happening. I would also love to see sex-based analysis and reporting improve, even if folks don't conduct a robust sex-based analysis within their data set, at least, if they could say why. That would help a ton with reproducibility. And we did see that, but only in 4% of studies in this sample that they provided a justification as to why they did or did not conduct analyses. And at least if you have that information, you know a bit more about the context in which you can reproduce that work. 

Erin Spain, MS: And so an example of that would be, as you said, maybe they're studying prostate cancer, so they were only using male mice. Would that be an example of why they would exclude the analysis? 

Nicole Woitowich, PhD: Yes. And that's again, a justifiable condition. And I think the NIH policy is clear on that, that there are cases where single-sex studies are okay, so this isn't a policy intended to demand that investigators use both sexes, but really just think deeply about how sex could influence their work. 

Erin Spain, MS: What's the reaction been like from your peers, the scientific community, media? What's been said about your findings here? 

Nicole Woitowich, PhD: It gained a lot of traction in other areas where scientists are getting their news, so that's great that they're seeing this as an issue that continues to be elevated so that we could talk about it. I was really excited to see that my work also got picked up on Instagram, which is something I had never seen an influencer cover my research before. So that was awesome because I think a lot of us are getting our information on social media these days, for better or for worse, but there are some really good science communicators out there that are trying to share this work in a way that's really trying to engage a broader audience. And I think that is great. This is then highlighting this issue that we're not doing the best science if we cannot consider everyone in the population. 

Erin Spain, MS: This is your area of expertise. You're gonna keep looking at this, so what's next for your research in this area? 

Nicole Woitowich, PhD: For me, I really like to keep tracking and evaluating this policy over time, so I think you'll see more work coming up in the next handful of years. We've done some retrospective work looking at 10 years of progress. I think we're coming up to our 20-year milestone soon. So, keep track and evaluate that. I'm interested in looking at the ways in which different scientific disciplines within biomedicine focus on this. So, as I mentioned we've already seen that there are differences in the basic sciences and how discipline-specific norms can really shift how one considers sex in their work, so I think that's gonna be another area where I'd like to dig in. To this issue on the investigator aspect of why scientists choose to do different things, choose to ask different questions, and really focusing on why do women do this differently and really getting some more concrete data versus anecdotal data on the differences in reporting. 

Erin Spain, MS: Thank you so much for coming on the show today and sharing the results of this new study and how things stand since 2016, and we look forward to your continued research on this topic. 

Nicole Woitowich, PhD: Erin, thank you so much for having me, and I appreciate the opportunity to speak with you. 

Erin Spain, MS: Thanks for listening. Please click the bell to receive notifications about our latest episodes and follow us on social media @NUFeinbergMed to stay up to date with our latest research findings. 

Physicians who listen to this podcast may claim continuing medical education credit after listening to an episode of this program.

Target Audience

Academic/Research, Multiple specialties

Learning Objectives

At the conclusion of this activity, participants will be able to:

1. Identify the research interests and initiatives of Feinberg faculty.
2. Discuss new updates in clinical and translational research.

Accreditation Statement

The Northwestern University Feinberg School of Medicine is accredited by the Accreditation Council for Continuing Medical Education (ACCME) to provide continuing medical education for physicians.

Credit Designation Statement

The Northwestern University Feinberg School of Medicine designates this Enduring Material for a maximum of 0.25 AMA PRA Category 1 Credit(s)™. Physicians should claim only the credit commensurate with the extent of their participation in the activity.

American Board of Surgery Continuous Certification Program

Successful completion of this CME activity enables the learner to earn credit toward the CME requirement(s) of the American Board of Surgery’s Continuous Certification program. It is the CME activity provider's responsibility to submit learner completion information to ACCME for the purpose of granting ABS credit.

Disclosure Statement

Nicole Woitowich, PhD, has nothing to disclose. Course director, Robert Rosa, MD, has nothing to disclose. Planning committee member, Erin Spain, has nothing to disclose. FSM’s CME Leadership, Review Committee, and Staff have no relevant financial relationships with ineligible companies to disclose.