Laura Kier, VP of Growth at Centaur Labs

How can medical professionals employ AI to enhance their decision-making process? Laura Kier, VP of Growth at Centaur Labs, discusses her role in healthcare data annotation and explains how the DiagnosUs app taps into a vast network of experts to generate quick and accurate data. She shares a story of how a global medical device manufacturer spent a year annotating data that the app was able to annotate in 21 days. She envisions a future with an AI-driven ecosystem that works with professionals to improve patient care.

Laura Kier is a data expert and technology optimist. She serves as VP of Growth at Centaur Labs, a data labeling platform for companies and researchers on the cutting edge of AI and data analytics, with a focus on life sciences, medical data, and scientific research.

In her current role, Laura works across product, customer success and sales teams to build technology for the medical AI ecosystem, reflecting her passion for finding technical, data-driven solutions to improve the lives of people and society.

Previously, Laura worked in product and operations roles in cleantech, creating infrastructure and policy programs to help people reduce their energy consumption and utilities procure cleaner energy sources. She holds a BA and BE in Engineering and Public Policy from Dartmouth College and an MBA from MIT Sloan School of Management.

Show Notes

• Laura Kier shares her background in technology and AI solutions. [03:35]

• How can the wisdom of crowds aggregate data more efficiently? [06:00]

• A vision of a healthcare future using data-driven AI to provide better patient care. [07:14]

• What are potential barriers to AI implementation in healthcare? [11:02]

• How does the medical app DiagnosUs gather data? [12:10]

• An example of how to train an algorithm to predict medical patterns. [17:28]

• What type of customers are leveraging this technology? [19:08]

• What are the challenges to AI deployment? [21:12]

• Why is making data-driven decisions important for healthcare? [25:15]

• What should we be cautious about with this technology? [27:57]

• Will the data annotation requirements of AI plateau? [31:12]

Transcript

Laura Kier  00:09

One company we worked with, a global medical device manufacturer, spent a year annotating data that we annotated in 21 days. Our speed comes from this network of experts that we have leveraged through our app called DiagnosUs. So we have 1000s of medical students and medical professionals on this app that essentially compete to annotate data most correctly. And what that does is we are able to, at the press of a button, contact this wide network and get multiple opinions on every single medical case.

Bisi Williams  00:58

Hi, my name is Bisi Williams, and you're listening to Health2049. Today we're going to talk about healthcare and data annotation. Approximately 30% of the world's data volume is being generated by the healthcare industry. But it's mostly unstructured right now. If you organized it, it could be broadly used in AI to create learning models for better health outcomes. Here's a stat that blows my mind: data scientists spend up to 80% of their time preparing data, that's collecting it, organizing it and then building training models. Imagine if they could repurpose that time to do what they do best, creating and deploying models to produce the best outcomes across the healthcare space. Today's guest is a self-proclaimed data nerd and technology optimist. Laura Kier is an engineer and holds an MBA from MIT. She serves as VP of Growth at Centaur Labs, a data labeling platform for companies and researchers on the cutting edge of AI and data analytics with a focus on life sciences, medical data and scientific research. It is my pleasure to welcome Laura Kier to Health2049. Laura, welcome to the show.

Laura Kier  03:22

Great to be here. Thanks so much for having me.

Bisi Williams  03:24

It's a pleasure. I'm so excited to talk to you today. There's so much that I'm eager to learn. But first, if you could just tell us a bit about your fascinating background.

Laura Kier  03:35

Sure. So my passion throughout my entire career has been in leveraging technology to impact people's lives and critical global challenges. So specifically, I've always been interested in this from a systems perspective, within highly regulated industries, like clean tech and healthcare. It always struck me that the most boring problems were the most painful, and at the same time, the hardest to overcome. So I started my career focusing on this in the context of clean tech. I was working at a company called Energy Hub, which is a subsidiary of Alarm.com and what we were building was technology to help people use less energy in their homes. And I always like to talk about that as well as trying to improve the electric grid, not the cutting edge technology that is wind, solar and wave energy and things that are maybe a little more sexy, but potentially less impactful to the entire electric system. In grad school at MIT, I took the famous computer science course artificial intelligence and during that class, it struck me that the biggest barrier to AI deployment was not computing power or creative AI approaches, but again, all this unsexy stuff. How do you embed AI solutions into your daily lives? And specifically, what we do at Centaur but how do you get enough training data to build AI? At MIT, I met Erik Duhaime who's the CEO of Centaur Labs. He was getting his PhD at MIT's Center for Collective Intelligence. His research focused on the wisdom of crowds, so how multiple opinions tethered together are smarter than any single expert opinion. You can think about it like you go to a town fair and they're asking everyone, how many jellybeans are in the jelly bean box? Or how much does the cow weigh? And what happens is the most accurate prediction is actually the aggregate of every one submission, no single submission.

Bisi Williams  05:58

Is that a fact, is that really true?

Laura Kier  06:00

Yeah, so that's the idea of generally the wisdom of crowds, how you can aggregate opinions in an intelligent way, will always lead to a more accurate result. And he applied it to the healthcare space and his dissertation in regards to skin imaging. So essentially, looking at how, if you aggregate the opinions of let's say, 10 medical students, you're gonna get a more accurate prediction of is there cancer in the skin image or any other type of pathology than any single 10 year board certified dermatologist?

Bisi Williams  06:44

So what you've just done with that, and brilliant analogy in a sentence or two, you've actually explained what AI does, and how it works in very human terms, which I know we'll get to a little bit later in our discussion about how you think about AI. What is your vision for health and wellness in the year 2049?

Laura Kier  07:14

Yeah, so I really love this question. I think, obviously, in my day to day, but I think all of our day to day is we always think about the short term challenges to health care, drug prices, staffing costs, clinical burnout, my husband's a doctor, so clinical burnout is something we talk about a lot. But despite what makes these headlines now, the whole industry is about people, and how we can keep people healthy and well. At Centaur Labs, we envision a healthcare ecosystem that leverages these insights generated by AI at every turn, to provide better patient care. So you can think about it from the beginning, how people manage their health with fitness and nutrition apps, to how a provider is making clinical decisions. There are all these AI decision support tools, which is a lot of what we do, and help a lot of our customers to know how hospital systems or providers can elevate high risk situations. So making sure you can identify stroke faster, that has a real impact on people's lives, to how medications are developed. So thinking about how do we find the right patients for clinical trials? And how do we look through pathology slides to find different indications of cancer? So we see at Centaur Labs, the fact that AI can be deployed in every single decision and turn in the patient journey. Importantly, our vision is very much contingent on the fact that all these use cases really demonstrate how AI analytics are most powerful when they are working with the humans of health care, doctors, patients, providers and not replacing them. I think there's a lot of conversation about doctors being worried that AI is replacing their jobs. But when you think about those examples I just listed, it's all about how AI can supplement and make their lives better. So the foundation of the company was built on the idea that humans working with AI produce the best outcomes across the healthcare space. I like to tell the story of the founding of the name of the company, which comes from Centaur Chess. So, world class chess champion, Garry Kasparov invented it after he was defeated by IBM's Deep Blue and that was the computer chess program and Centaur Chess is a form of chess where players play with the computer chess program and the idea is they can make the best decisions by marrying, processing data and analytical power of a computer with the creativity and ingenuity of humans, and we really see that happening in the same manner in the healthcare space with doctors leveraging AI at the bedside and scientists leveraging AI at the bench.

Bisi Williams  10:21

I love that. And so now my secret's out, too, because when I'm playing chess with my kids on my phone, I actually run a parallel program to make sure that I'm doing it at the same time. So I don't know if that's cheating. But I like to see how to work with those algorithms and try to make the next move. So I love that your vision has the data, the crowdsourcing the AI working in tandem with humans to do this parallel process. And I'm curious, why are you confident that your idea and our vision can be achieved within 30 years?

Laura Kier  11:02

Yeah, so I think the two major barriers to this vision are the development of AI and the adoption of AI. So on the development of AI, that's a lot of what we're focused on at Centaur Labs, what we're trying to do is help our customers organize their data and structure it such that they can develop these models. You referenced earlier, the fact that 80 plus percent of data scientists spend their time on this collection of data, organizing it and then building the training data. We want to help replace that step and get these products to market faster. One company we worked with, a global medical device manufacturer, spent a year annotating data that we annotated in 21 days.

Bisi Williams  12:02

Wait, how did you take a year of data and get it annotated in 21 days? That's amazing.

Laura Kier  12:10

Yeah, so our speed really comes from this network of experts that we have leveraged through our app called DiagnosUs. So we have 1000s of medical students and medical professionals on this app that essentially compete to annotate data most correctly. And what that does is, we are able to at the press of a button, contact this wide network and get multiple opinions on every single medical case. What's also unique about our process is we have a method of measuring performance. So not only are the people on the app doing the annotations, but we are mixing in cases where we know the answer and cases where we don't know the answer and saying, Okay, we now know that this individual is performing very well, we're going to reward this person. And we actually are not rewarding those who are not performing well. Our CEO actually always makes this joke where if you're a doctor, and you're doing annotation, you might be doing it on a Saturday night and drink a glass of wine and your performance goes down and there's no way to know because all we do is just trust what these doctors are saying, but we don't actually have a method for measuring their performance. So that's sort of what we were able to do. And if you want me to go into it, there's a cool kind of analogy to reCAPTCHA actually.

Bisi Williams  14:04

Yeah, tell me more. I mean, I find this fascinating, the gamification, that you've got 1000s of doctors and students in the system. This is the whole guess the weight of the cow analogy.

Laura Kier  14:16

Exactly, so I'll do a quick tangent here. Do you ever get that pop up that says click all of the stop signs to show your data robot?

Bisi Williams  14:33

Yes, can you make that go away?

Laura Kier  14:38

So what they're doing there is actually helping you annotate for self-driving cars. Which is crazy, I'll explain it a little more. By you going and saying, I see a stop sign here, I don't see a stop sign here, you're actually looking at pieces of data and tagging them. And what they are doing as part of that process is they are putting in some pictures where they know it's a stop sign. And they know it's not a stop side and some pictures where they don't. So for the ones that they do, they're deciding whether or not you're a robot. And for the ones that they don't, they are collecting your work. Our process is similar. What we're doing is we're mixing in some images, let's say we're looking at a bunch of skin images and trying to find melanoma. We're mixing in some where we know that there's melanoma or not melanoma. And what happens is, we can basically tell, are you good at detecting melanoma? If so, we're gonna collect your work. If not, we're gonna throw it out.

Bisi Williams  16:11

That's awesome. It's gonna make me think about that. Now, I have to bring my A game every time I sign up for something. But seriously, I think that's fascinating how you use double blinds, you use multiple ways of testing, and that I feel should bring an enormous level of comfort to everyone. I mean, I love my stuff quintupled tested, if you know what I mean. And I think that in these life and death situations, the work that you're doing is really quite remarkable with this annotation and incredibly exciting. I love that you use this gamification and that your researchers are engaged. Now you've got this treasure trove of information. From my perspective, that's a higher order of design. You are curating and cultivating this material to give you best in class for clarity. That's what we want. It's just clarity. So now tell me how that gets translated, you've got great information sets. How do doctors use that? Who's using your technology?

Laura Kier  17:28

Yeah, great question. So I'll walk you through an example of a company we work with. So the company is called Eko, they have digital stethoscopes where they're able to automatically record heart sounds and lung sounds. What they're trying to do is build on top of that, a software layer that's going to help doctors make the decisions while they're working with patients. So better able to detect different pathologies. The software could say, Hey, we detect a heart murmur or we detect a cough, and that would be really helpful for them at the point of care. So what they're doing is they now have a bunch of data and they need to train and create an algorithm. In order to do that, they need examples of what is a heart murmur and what's not a heart murmur. So they'll send us this data, we will funnel it through our system and through our network, get multiple opinions, filter out those opinions that are not performing well and create a single aggregated opinion, then we can send that back to the customer and say, recording A we detected a heart murmur. And now they have these examples where they can train an algorithm that will predict these patterns.

Bisi Williams  18:57

I find that amazing and this is available today. It's just so incredible and so who's using it?

Laura Kier  19:08

What type of customers are leveraging this? A lot of AI companies, so we're working with this company called Eko. We're working with a company called Page that had the first FDA-approved pathology AI algorithm. So they're wonderful. We're working with a company called Volastra that is trying to use pathology slides as well, but to detect cancer. So a lot of these software AI companies. We're also working with a lot of global medical device manufacturers who are trying to layer this software on top of hardware. A lot of them might be just collecting this data through their hardware. My husband's a GI, you can think about someone who has an endoscopy scope, like those tools, the colonoscopy that no one wants to go to. But they're collecting a lot of the data at the point of care. So they want to have this software layer to help them. And then a lot of pharmaceutical insurance companies trying to do the same level of prediction, a lot of researchers as well. So it's really across the healthcare space that people are trying to use AI to make these predictions and support people through their healthcare journey.

Bisi Williams  19:26

I think that's amazing. When I think about all of these different parts of the body, we're very complex beings, obviously. But at what point does a synthesis happen? At what point do you have the whole corpus from the beginning and how do you imagine layering all of those pieces? So who is it that puts it all together or is that the future state?

Laura Kier  21:12

Yeah, I think part of what you're getting at is how many pieces there are to AI deployment, and how hard it is to align all these pieces and get them used by the people in the healthcare space. It is a real challenge. I think, specifically, there are a lot of companies right now. So we have a lot of companies developing AI and we have a lot of companies that are also trying to be like an aggregate AI solution and help with that deployment. So there's a company called DeepSea and they're trying to be a platform where a lot of different AI companies can work with them and they are in charge of getting it into hospital systems. There's a company called Lucem Health and they're a Mayo Clinic spin off. So Mayo Clinic is a large hospital system, but has a lot of community hospitals that are connected to them, so by sort of doing the vetting and integration of the AI such that there's just one platform for these hospitals systems to interact with, that's really helping get these into the day to day flow of healthcare delivery. I would say another large burden to AI deployment or large challenge is the reimbursement. Who's paying for this?

Bisi Williams  22:52

Who is paying for it?

Laura Kier  22:54

That reimbursement model hasn't totally been figured out yet. A lot of hospital systems are looking at reduced medical misdiagnosis. So can they save a lot of money by having these clinical decision support tools and not have as many lawsuits essentially. I also do see that there's a beginning already of billing codes that are going to let these physicians bill for deploying these solutions.

Bisi Williams  23:29

That's fascinating. So could you give an example, either now or in the future case of a billing code for this technology?

Laura Kier  23:41

Well, I think it would just be something like I have used this technology, in this particular case, and I am being reimbursed for it at the point of care. I think there are a few already in the GI space. But still, it's quite nascent and I think there's a long way to go and then thinking even more generally about regulation and how we can keep, yeah, I don't know this might be a little bit deeper into my knowledge.

Bisi Williams  24:13

Keep going, it's great. I think this is fascinating, because, honestly, I think the intersection where you're working at, where it's really difficult, regulation and policy, because we can dream and imagine all of these things and systematically get them. And what I love about what you're doing with your company is the way that you're thinking, you're sort of designing the space first so that it's not, you don't have an alien baby landing somewhere in the universe, so that it's actually connected. So there's wraparound support and there's a connection in terms of how you can use AI to assist with diagnosis and get better care. And I want to know from you,  why is your idea and vision important? How does it make the world a better place?

Laura Kier  24:20

Well, I mean, asking it another way, why is making data driven decisions important for our health care?

Bisi Williams  25:14

That's a good question.

Laura Kier  25:15

We can think about how data is already being leveraged in all these different recommendation engines in our day to day lives, like the TV shows we watch, where we order dinner from, what apartments come up and the StreetEasy app that I'm always using. And nothing's more important than our health. It's so important that we have all the information to make the best decisions possible. A lot of what I think about in our day to day at Centaur Labs is, it's so important because we collect these multiple opinions on each piece of data, we're able to also tell you the confidence level that we have in the assessment. So you can think about it like if someone just looks at a piece of data and says, Yes, this audio recording has a heart murmur. That's it, you have one person who says this audio recording has a heart murmur. But if we got 10 people's opinions, we can say, four out of 10 thought it was a heart murmur, or eight out of 10. And that's really important, because you can think about how that's embedded both into the AI development, like the level of certainty. But you can obviously think about the broader implications of that.

Bisi Williams  26:50

If you think about that, if we're not using data to the full extent possible, what are the consequences with that, in your opinion?

Laura Kier  26:58

Well, I think there are two things. One is that I think there are real people's lives at stake. So there's so much that goes into a medical decision, how do we not leverage all this information and provide the best care? I would also say, there's a democratization of data piece, and how to access that is super important, because if we aren't using all the data possible, if everyone in the world can leverage the different data, then they can also provide better care to their communities.

Bisi Williams  27:41

I think that's a great answer and I know this show is about optimism and possibility, but I have to ask this one good question. What, if anything, should we be cautious about in this space? 

Laura Kier  27:57

Oh, my gosh, what a good question. So a lot of our customers are applying for FDA approval. And what we've seen as they undergo this process is that the verification is different for each of their different applications. So they need to prove that this AI is working and going to be successful, if deployed. And to do so they need to prove that their AI is helping with detection of these diseases, or improving and doesn't have negative effects. But the reality is, a lot of them, each of them have applied with a different verification process. So looking at one company, they're saying, We proved that the AI was better than three board certified doctors, and that means we're better, whereas another company will apply and say, Well, we had two people look at it, and then one person arbitrate and they all have different processes. And I think clarity over what is the process for AI to get this FDA approval, could be really beneficial in streamlining and getting more AI out in the world, but also add clarity and probably produce safer and more impactful products.

Bisi Williams  29:47

I have to agree with you, Laura. It just seems to me when we think about innovation and regulation that somehow if they could be designed in tandem, because the innovators are really fast and they're super liquid. And the regulators are kind of crystals and they're slow. And if you can imagine, in this process if you're using this AI methodology, have multiple opinions on either side, that the regulators could use that it could have the benefit of that, too. So to take some of the angst out of their decision-making. At the same time, the innovators on the other side could do the same thing. I'm just thinking out loud, because they could then move smoothly through this process for something that we desperately need and clearly want. And that I think that co-design could actually expedite that process and take the pain out of the 80 hours of researching, just putting it all together, so that you can actually use the best minds to work on the innovations, the cool things that they love to do to make the world a better place. And my next question is, will the data annotation requirements of AI plateau?

Laura Kier  31:12

That's a great question. And I think it comes up a lot when we talk to investors and potential clients, because a lot of clients come to us say, Okay, we want you to annotate, let's say, 2000 images. After that, we're good. But the problem is, first of all, they realize, okay, I don't just want to know if there's melanoma, I want to know if there is this other type of cancer, then a new tool comes out that is collecting data in a different way. And it looks different and the AI needs to learn how to collect that data differently. Some self driving car analogies are, you can think about how the data from the streets of San Francisco is going to be different than Philadelphia, you're going to need to have diverse datasets in order to create a robust AI bottle. Skin is actually particularly interesting, because a lot of datasets come from these hospital systems where the population doesn't have a diverse skin color. And so we see a lot of challenges that these hospital systems or these AI companies have when they try to deploy their AI in new communities. Because the people that they're serving are different.

Bisi Williams  32:47

You raise an interesting point because we were just talking to some Gen Z folks, when they talk about the future of health and wellness, that diversity, they already just assume it's baked in. So you would realize that by building these models, that it would have the rainbow of skin colors, I guess, with the presentation of melanoma, and that really opens up a whole other thing, which is very cool. So I guess you're right, it's not going to plateau, is it? 

Laura Kier  33:22

Yeah, it's ever changing. I think it's definitely A, the fact that you're going to always want to uncover more from the data you have B, you're going to need to think about how the data is different and the different locations are going to need diversity. And then what's also sort of interesting is, it's really hard to continue to improve a model a lot of times, our customers will think about, okay, I only need 2000 images annotated and then they just don't get to the level of accuracy that they want to get to. So, a lot of times people talk about data being the new oil and it's very much the same in this space.

Bisi Williams  34:19

And I think you raise an interesting component to your world in your work and what you're laying forward is that we have to be continually learning. This is a process that's rather iterative and it's designed to be that. Eventually, I guess the cost of annotating this data, eventually you've got a body of knowledge and you build on that. And then at some point, there's room for the new and novel.

Laura Kier  34:53

Yeah, I think there's a challenge that our clients are still running into is just accessing enough data. I think sometimes we think about the 30 to 40 year vision and we forget that we still use the fax machine. There's so much data that still isn't digitized. I mean, when we're talking about AI development, it's the data annotation bottleneck, and it's also the data access problem. I'm always surprised how little we accomplish in five years and how much we accomplished in 10 years. I think the data digitization still has a way to go, though, obviously, we've made so many improvements. And I think a lot of this what's really exciting to our earlier part of our conversation about the hardware software solutions is that all new hardware solutions are natively capturing and digitizing this data.

Bisi Williams  36:07

Oh, that is music to my ears. I'm always concerned that we're going to be doing all this incredible stuff on an Atari 64. Could we not do those in tandem? I could talk to you about data and information all day long. And this is a subject that's dear to my heart. This has just been fascinating about this world of data annotation. I just can't thank you enough for joining me today on Health2049

Laura Kier  36:40

Thanks so much for having me. This was really fun.