Hey, everybody. Welcome back to the Elon Musk
Podcast. This is a show where we discuss
the critical crossroads that shape SpaceX, Tesla X, The
Boring Company, and Neuralink. I'm your host, Will Walden.
Good afternoon. Thank you for joining us for
NASA's first artificial intelligence Town Hall, I'm
Melissa Howell. Today's all about offering more
insight and how NASA is using AI to build on its foundation of
innovative technology and exploration and the role AI will
continue to play here at the agency as we look to the future.
After the discussion, we'll take questions from the audience
right here in the room and those joining us online at conference
dot IO. To kick off today, Town Hall,
I'd like to welcome Administrator Bill Nelson for
opening remarks. Hey everybody, When a new
generation of tools reaches NASA's doorstep, we don't just
plug in and suddenly go into a whole new bunch of things.
We study them, we iterate them, and we advance them for the
benefit of all. We work with partners across
industry, across universities, across our government, across
the world to find ways to improve them.
And so we do what NASA does best.
The technology that we have today is astounding.
Take for example, the phone in your pocket or even the phone
that you're using to watch this town hall.
It has over a hundred 100,000 times the processing power of
the Apollo Guidance computer. The technology, of course, that
took us in Apollo 11 and put Neil and Buzz on the surface of
the Moon. And we're talking about just the
phone that we use to send emails.
So imagine what we can do today as NASA dives into the next
generation of artificial intelligence.
We have used AI safely, sustainably, successfully for
decades. Yet today this technology is
transforming before our eyes. There is new promise in
artificial intelligence, for example, machine learning tools
like neural networks, deep learning, generative AI, and
modeling. And as this technology grows, so
too does our capacity to use it, to test it, to refine it, and to
integrate it into our work to try to benefit all of humankind.
When used right, AI accelerates the pace of discovery, and it
can support our missions. It can drive our research.
It can analyze our data. It can support our spacecraft,
aircraft, science, and a lot more.
And it can open new possibilities in our ability to
land on celestial bodies and to navigate to them, to peer into
the vast corners of the cosmos and even in the search for life.
And remember, that's a statutory requirement of NASA searching
for life. That's why we're digging on Mars
right now. That's why we're looking for
exoplanets. And it goes on and on.
That's why we're sampling that asteroid sample from Bennu.
Well, AI can make our work more efficient, but that's only if we
approach these new tools in the right way, with the same pillars
that have defined us since the beginning.
Safety, transparency, and reliability.
Those three things matter not only to NASA, but to the
president and to the vice president, and they are focused
on making sure that when our government uses AI, including
these emerging technologies, that we do so safely, securely,
transparently, and responsibly. So consistent with the
president's executive order on artificial intelligence that he
wrote last October, we recently announced our new chief AI
officer. You're going to get to meet
David. And I thank David and all of our
panelists who are going to share with you today about where we're
going with AI. When new tools arrive at our
doorstep, we test them, we improve them, and we deploy them
to hopefully better serve humankind.
There's a lot of risk with AI because if it's employed in ways
that are not for the betterment of humankind, then it could be
disastrous. But the way we're doing it is
the way that we employed technology for the very first
step on the moon 55 years ago. And that's how, using every tool
at our disposal, we're going to make leap after giant leap in
the decades to come. And that's going to be how we
lead. And so I want to bring up our
deputy administrator, Colonel Pam Melroy, a person you all
know whose resume far exceeds her as a space shuttle commander
and an Air Force test pilot, and on top of that, a great team
member and as we refer to ourselves as a crew, a great
crew member. Come on up, Pam.
Thank you so much, Sir. It's a not just an honor, it's a
pleasure to serve. So thank you.
I'm really excited about today. You can probably guess you, you
guys all know I'm really a nerd at heart.
So moments like this are very exciting for me.
So I just want to say thank you to everybody for joining us for
this town hall. Of course, we're gathering today
to discuss one of the most transformative and exciting
frontiers and technology, artificial intelligence.
At its core, AI is an advanced form of statistics and
probability that creates the capability for a computer system
to perform complex tasks that have traditionally required
human intelligence to integrate, from reasoning to decision
making and even cool things like creative endeavors and art.
At NASA, Unsurprisingly science and technology Organization,
this is not just a buzzword. We have already harnessed the
power of AI tools to benefit humanity by safely supporting
our missions and our research projects, analyzing data to
reveal underlying trends and patterns, and developing systems
that are capable of supporting spacecraft and aircraft
autonomously. Unsurprisingly, in a poll across
the federal government, NASA had more use cases than any other
federal agency. I can tell you, I want to give
kudos to our Associate Administrator for Science,
Doctor Nikki Fox. Five years ago, I heard her give
a talk about the use of AI in heliophysics data.
That blew my mind. It was awesome.
So we're already doing this, and we're doing a lot of it.
But as we push those boundaries and we continue to find exciting
new ways to do use AI, we have to recognize the importance of
responsible usage like we do with any other disruptive
technology. So as was said most recently by
Spider Man, with great power comes great responsibility.
So we're totally committed to that.
We have to ensure that what we are doing in our AI initiatives
are guided by robust governance and protective measures.
So our goal of course is to harness the benefits of AI for
the betterment of humanity. But we have to do the other
side, which is to safeguard against the potential risks,
which are unfortunately all too real from unintended bias and
data collection or things like data compression, which ends up
resulting in less than accurate answers.
The consequences can range from funny, think auto correct on
your phone to very serious impacting human health and
safety. The administrator mentioned the
president's executive order. So as hard as we're leaning in
on the technology, we must be steadfast and lean in to
formalize the processes and the protocols for AI usage to make
sure that we're taking advantage safely of AI innovations across
the agency. So having that governance
structure is critically important.
It provides us procedures and guidance from the agency.
And actually, when you put that kind of structure in, it can
actually empower you, our workforce, and unleash your
innovation in a way that we all feel comfortable.
We're doing it the right way. Our artificial Intelligence
Working Group has been championed by three powerhouse
leaders at the agency, our Chief Technologist, AC Terrania, Chief
Scientist Kate Calvin, and our CIO Jeff Seaton.
This working group has been looking at this executive order
and developing recommendations in response to the directive
that are very focused on NASA, on our mission, how to best
enable our mission. The other exciting step that we
took is we named David Salvagnini as our chief
artificial intelligence officer, and I think his phone started
ringing off the hook right away, maybe even before we announced.
This is really just to under score the commitment that having
a focus area and taking this aspect of governance seriously
is really important to the leadership team and it's
important to all of us. So just so exciting when you
think about the future and all the interesting use cases that
we have for AIA. Pivotal aspect of our approach
involves how AI can enhance collaboration within our
workforce. That's a really exciting
opportunity. Communities of practice, working
groups and other avenues. We aim to foster a culture of
knowledge sharing about AI and collective learning.
And thank you. We have seen a significant
increase in the number of AI trained employees.
Yes, we do have formal training for this and this number is
poised to grow. And I encourage those of you who
are listening who have not taken this training to look into it.
This is all part of this commitment to responsible AI.
And it goes actually beyond our internal operations.
We're looking at partnerships with regards to AI leaders in
the private sector and in academia, recognizing that this
is incredibly important to collaborate to drive cutting
edge developments that impact the largest problems of our
time. So our participation in
initiatives like the National Science Foundation's National
Artificial Intelligence Research Resource Pilot is just one great
example of how we're dedicated to also leveraging our partners
and collaboration to advance AI on a much broader scale,
impacting not just NASA's mission, but the whole country.
AI is going to help us in so many areas, analyzing
heliophysics and Earth science imagery, probing the depths of
space with our telescopes for new insights, having them work
together to schedule communications on our networks.
We all know our communications networks are so loaded with
fascinating information, but we can barely squeeze it all in.
AI can help with that and just all kinds of other exciting
things, helping crews in the future on the way to Mars.
And really just think about, we don't even know yet what new
insights we're gonna get by using these new techniques to
look at old data in new ways. Again, establishing a very
robust governance structure and empowering you.
We can harness that full potential to fuel our own
missions. Dr. Innovation and continue to
change the world. So as I close, I just want to
emphasize it is a powerful, ingenious and very exciting
tool. But if we don't manage it
responsibly, we're going to open ourselves up to a world of risk
that jeopardizes our credibility and our mission.
Fortunately, this is something we are very good at at NASA.
We challenge ourselves to manage risk effectively, to push
boundaries and be innovative, to boldly explore as a team and do
it with the best possible risk management practices.
That's exactly what we're going to do with AI.
So I'm excited for you to hear today from the panel who really
knows what's going on, Jeff, AC, Kate, and Dave about how we're
using AI at NASA, how the field is changing, and how you can
learn more and educate yourself more and hopefully take
advantage of this technology. So thanks for joining us today.
Thank you, Deputy Administrator Melroy and Administrator Nelson.
I hope you all are ready for what is going to be a great
discussion. We're going to go ahead and
welcome our panelists to the stage.
Joining us today for the discussion, we have Doctor Kate
Calvin, NASA's chief scientist, AC Sharnia.
He serves as the agency's chief technologist.
And then we also have Dave Salganini, NASA's first
artificial intelligence officer, and our Chief Information
Officer, Jeff Seaton. Go ahead and give them a round
of applause for joining us today.
Thank you all for being here. We're looking forward to having
what will be an exciting and important conversation around
artificial intelligence. I want to start by giving you a
chance to talk a little bit about yourselves and your role
as it connects to AI. So I'm Kate Calvin.
I'm NASA's Chief Scientist and senior climate advisor.
As Chief Scientist, my office is responsible for representing and
enabling science across the agency, including the use of AI
to advance science. All right, I am AC Trinia, NASA
Chief Technologist in the Office of Technology Policy and
Strategy here at NASA Headquarters.
My job is to help shine a light on how we can be more
innovative, specifically with emerging technologies like AI.
Good afternoon. My name is David Salvinini and I
started here at NASA last summer as the Chief Data Officer
supporting the requirements in the Evidence Act and have since
assumed the role of the Chief artificial intelligence
Officers. So in both of those roles, I
will be looking after NASA's journey as it relates to meeting
some of the requirements from both executive orders and also
the OMB guidance, but more importantly, doing what's right
for NASA and addressing our journey and facilitating our way
forward as it relates to the use of AI in responsible and ethical
manner. Great and good afternoon.
I'm Jeff Seaton, the Chief Information Officer for NASA.
And I've got a responsibility for providing tools and
capabilities that enable the the work of the agency, leveraging
IT technology and also ensuring the security of our data and
systems across the agency and AI falls into to that sphere as
well. Thank you all.
And I want to start first with you, Dave.
First of all, congratulations on your new role as Chief
Artificial Intelligence Officer here at NASA.
Would love to hear from you about what your role actually
entails and what you're hoping to see as we move forward with
with your new role. No, thank you very much for the
question. So just a little bit of context
about this, if you're not tracking and I'm sure many of
you already are, the regulatory guidance coming from the
administration is not new. In December of 2020, they
released the Responsible AI Executive Order, which created
the need for a responsible AI official within federal
organizations and that really went after the need to be
ethical, responsible, transparent, safe, and and it it
accounted for accountability as it related to how federal
organizations use AI. Fast forward from there, we have
a new executive order last October, which now UPS the ante
to some degree and says, Nope, a responsible AI official is not
enough. What we'd like as a chief
artificial intelligence officer. And that person is going to be
responsible for ensuring the safety and rights of U.S.
citizens and also really driving agencies towards more innovative
practices and risk management. And then since that was issued
in in October, OMB released a memorandum in 28, the 20th of
March, which offered a lot of additional detail for how
federal agencies ought to address that.
So in part, my role is addressing those requirements
from the executive order and the OMB memo.
So what does that mean? Well, that means, you know,
establishing my role, establishing a leader who's
ultimately responsible for this journey at NASA, and then really
establishing some of those mechanisms that director, Deputy
Director Melroy and Administrator Nelson talked
about as it relates to how we manage risk associated with AI,
but also I would say manage risk while also managing opportunity.
So what do I do there? So I create situational
awareness. I would say the role is largely
an orchestration coordination role.
I think of myself if if NASA were a Symphony, I'm the
conductor. And I'm harmonizing the various
different instruments and sections of the orchestra in a
way where we're all rowing in the same direction.
We're situationally aware of what the other is doing, and
we're capitalizing on that knowledge of each other's work
so that we can build upon it as it relates to our specific
mission area. So there's that role, there's
certainly the compliance role, there is the role of standing up
governance and doing so in a mission informed manner.
So it's not burdensome, but it's a value add to the organizations
that look to leverage it. There's a work forcing component
to this. In other words, dealing with,
well, how do we equip the workforce to address the change
in AIAI has been around for years, but AI has also changed
dramatically in the last 12 to 18 months.
And there's some responsibilities that we all
have as it relates to how we use these tools responsibly,
ethically, transparently and so on.
So that imparts the responsibility to make sure
we're reskilling and upskilling the workforce to, to be aware of
the change and understand their responsibilities.
So I think that's a good synopsis of the responsibilities
associated with the role. So I'll stop there.
And that really ties into what Pam and Bill mentioned about how
NASA has been using AI responsibly and effectively for
years. And Kate, I'd like to pull you
in here. Can you share how AI has
contributed and been applied to scientific research?
Absolutely. So scientists have been using AI
for a long time. And one of the things that AI is
really good at is analyzing large data sets, like the data
that we get from our Earth observing satellites or space
telescopes or our other science missions.
And I want to hone in on one one of the ways that we use AI
around with what scientists like to call anomaly detection or
change detection. Essentially, when you look
through a data and you look for something that looks distinct, a
specific feature, and once you've identified that feature,
you can count it, you can track it, you can avoid it, you can
seek it depending on what your goal is.
So some of the ways we've used AI, one is to track wildfire
smoke. So we can teach the computer,
this is what smoke from a wildfire looks like, and then
look through all of our satellite imagery and find other
wildfire smoke and track that. Another way we've used it is to
count trees. So if you teach the computer,
here's a tree, find all of the trees.
And this is really important to use AI in these because for the
wildfire smoke with wildfire and other disasters, what we really
want to be able to do is do things quickly.
People need that information urgently and the computer can do
it faster than a human can. In terms of the counting trees,
you know, it's not tractable for a person to count every tree
across the country or a continent.
So instead, we can have the computer help us and then we can
focus on what do we learn once we know how many trees.
So that's in looking at, you know, observational data sets.
But we can also use AI to help improve models that that do
weather and climate forecast in the future.
And I focused a little bit on Earth science applications just
because I think they're easier sometimes for people to relate
to. But we use this throughout
science. Pam mentioned heliophysics, but
also in the science mission director, we do things like
counting exoplanets. It's the same type of thing that
we do with counting trees. But now we're looking out in the
universe and seeing what else is out there.
And AC want to pull you in here and talk about the Artemis
missions. Can you talk about how AI is
supporting NASA's efforts to put men back on the moon and then
eventually put humans, bring humans to Mars?
Yeah, thanks. Kind of.
As Bill and Pam mentioned, I think artificial intelligence,
large language models can be used in combination with humans
and not to replace humans. We've used AI for many, many
years with humans to examine that data and to make us smarter
about the universe. Relative examples to your
question about how do we use artificial intelligence, large
language models and the like in terms of Artemis include helping
us observe the Moon and Mars. In terms of surface imagery,
surface features. We can look at how do we
leverage these kinds of technologies and human machine
integration interface to help humans more intuitively and
better work with machines. We can look at how we use this
technology to help us communicate with spacecraft at a
large distance to alleviate, accelerate mission operations,
not to replace decision making, but to accelerate our missions.
We can also look at machine vision technologies and
capabilities that AI enables to help our Rovers on the surface
of the Moon or Mars traverse farther and faster.
And finally, I'm also very excited with the potential for
AI agents, large language model agents to help us optimize,
understand, manage, observe these complicated interconnected
networks that we're establishing.
Thus, for instance, in the absence of crews, can these
technologies helps help us observe and maintain Rovers,
space stations, habitats, the like throughout the solar
system? And Jeff, you know, we hear a
lot about the concerns and the risks across the industry when
it comes to AI. Can you talk about the use of
the tool right here at NASA and what we're doing to kind of
eliminate that risk? Yeah.
But first, let me take a step back because you heard the
administrator, he said that it's a new generation of capability.
And it definitely is AI is, is not new.
We've been using it, as you've heard for years in our existing
missions. But there is something that's
new, right? And it's becoming more
accessible to, to all of us. And if you think about
generative AI, one of the questions is what is it?
Well, think about when you were a kid.
If you're like me, you like logic puzzles.
And I can remember these logic puzzles that say, here are 5
numbers, what's the 6th number in the series, right?
Generative AI is a little bit like that, except instead of
this simplistic series of five numbers, it's this trove of
information that we can now access because of the networks
that are connecting systems worldwide.
So instead of a simple series, you have this volume of
information that we can learn from that the computer
algorithms can learn from. And because of the progress
we've made over the last 20 years in natural language
processing and understanding, we're able to take human
language and prompts and ask questions and have mathematical
models. It's it's math, it's not magic
mathematical models that can then generate responses to those
questions. And so there's a huge promise in
that. But you talk about the some of
the challenges to the risks. And one thing I want to say is
that these are computers and programs and algorithms.
We have existing processes and approaches to appropriately
secure data computers, programs and algorithms.
So we're not going to create new processes and policies to
protect our systems and software.
We're going to use the ones that we already have.
Goes back to understanding the sensitivity of the data, goes
back to understanding where is that data going, how is it being
transmitted, What systems is it connecting to?
And we're going to be able to leverage those same processes to
make sure that you're able to access these new emerging
capabilities in a safe and secure and a responsible way.
So from 1 standpoint, there is a lot that's new.
From a security of the system standpoint, there's not a lot
that's new. We're just going to apply our
existing processes to to be able to do that and to make these
tools available to you ultimately.
And coming off of that with so many tools that that are going
to be available, Kate, the importance of having humans
actually review the work and the research that AI is is doing.
Can you talk a little bit about that?
Yeah, so human review of AI products is really, really
critical. AI can make mistakes, and we
have to look through it and check them and make sure that
it's robust. And we've been using AI and
science and engineering for a long time, but scientists and
engineers have processes to check the products.
So we do things like code review, where if you use
something to generate a computer code, you have processes for
reviewing that code, for checking that it's accurate, for
testing it. We use peer review in the
science so that we actually peer review the document and get
people looking at it and making sure that it's correct.
And we need to think about those same things when we're looking
at AI. There might be some cases where
you look at an AI product and it's clear that there's a
mistake in it. So Pam was talking about auto
correct earlier. So if you see it something
that's auto corrected and it's not the word you meant, you'll
know it, right? You know what you wanted to say,
you'll see that it's incorrect. But when you're generating a
whole paragraph that's maybe outside your expertise, you
still need that review because you have to check that it's
correct. AI only knows what it's been
trained on, and it could be trained on things that are out
of date. It's also combining multiple
data sources, and sometimes it does that in ways that are
really powerful and helpful. Sometimes it makes mistakes and
we have to check it. So human review is really
important. And you know, Jeff, we we
received a lot of interest in generative AII know you spoke a
little bit about what that is. Can you go in a little deeper
talk about NASA's current policy and what we can expect to see in
the future as these tools are evaluated for safety?
Sure. And following on to what Kate
said, I think it's good to maybe kind of create two different
buckets. One is the the ad that's built
into our missions where you have program and project processes
that are doing the review and evaluation of how we're
leveraging the the capabilities. And then we have what is really
emerging now. And these are the, the general
tools that will be accessible to to so many of us to maybe do
some things in ways we never even imagined possible.
And so I think I'll focus on that latter category right now.
And that's you. You probably, if you listen to
the radio, listen to podcast, you can't go 30 minutes without
hearing an advertisement of some company that's saying they're
now the AI company right there. Everybody is an AI company.
And we're going to see AI capabilities built into many of
the tools that we're already using today here at NASA.
We do a lot of our work from a kind of business product
activity standpoint with Microsoft tools.
We use others, but Microsoft is is definitely one of the primary
providers of some of our capabilities.
And you've probably heard of things like Copilot.
Well, Copilot is a term that's an umbrella for a number of
different products and offerings and integrations that Microsoft
is building in to its product suite.
Similarly, many of you might use products from Adobe to create
images or videos or audio, and they're building in AI
capabilities into their products.
Many companies are doing that and, and we use many, many
vendors. So one of the things that we are
looking to do is evaluate how those vendors are building these
capabilities into their products.
And it can be a little bit frustrating sometimes though,
because at home you might have access to some of these
capabilities today because they're in the generally
available products. And within the government, we
have a few more requirements that we have to to make sure are
met before we use them. One example would be in the the
general utilization, a provider might be using data centers that
are worldwide, but for the government, we have to use data
centers that are within the continental US typically.
And so we have to wait for providers to create a version of
products that do meet the federal government requirements.
So we're working with many different providers to
understand where they are, understand what capabilities
they are building in and how they are protecting those
capabilities to ensure that we're being responsible with the
data that we have, the data that we are working with.
And while all that is happening, there's safety, right, Dave?
That's a priority as well. Can you talk a little bit about
how the workforce can actually play a role to make sure that
safety is made a priority in an effective way?
Yes, certainly so and and I would say it's, it's beyond
safety, but just responsible use, right.
So as Jeff talked about and Kate talked about an AC talked about
the use of AI and the many opportunities you know, there
are. I, I almost wanna, I wish
artificial intelligence was not referred to as artificial
intelligence. I wish it was referred to as
assistive intelligence assistive.
In other words, it is my digital assistant.
It is that resource that I now have access to that can help me
in my decision process. And Kate talked about this quite
a bit. So I think it's incumbent upon
all of us. You know, the AI is not
accountable for the outcome. The person is the human is
right, So and the vendor who may offer an embedded AI capability
as part of a product suite is not accountable as well.
You know, the user is the person who leverages that tool is
responsible. So we have to 1st own that and
we have to understand what responsibilities come with us.
You know, I was thinking about this a little bit earlier and
you know, I was thinking about hurricanes and I was thinking
about weather forecasters that released as it relates to
hurricanes and think about the validation that goes into sort
of hurricane forecasting. You know, and you've all seen
the forecast where they'll show that the projected track and
they'll be multiple lines and each one of those lines is
represented by a model. Why do they represent multiple
lines? Because they're cross
correlating the outcomes and trying to ascertain what do we
reasonably believe is, you know, an accurate projection and can
we assert with confidence what that projection is?
Well, it's based on the aggregation of those outputs.
So again, human judgement, not AI doing our job for us.
So I think that's the important part.
So then, you know, how do we be safe about this?
We understand our responsibility as the ultimate accountable
person as it relates to the use, as it relates to our work
products. And then if we happen to use AI
as part of the generation of a work product, that's fine, but
just understand its capabilities and limitations.
Another thing I'd like to talk about is, is, you know, there's
this notion of hallucinations where maybe the AI kind of gives
you a false answer and maybe it generated that false answer, but
there's that can be somewhat obvious and more easily
detected. I would be more worried about
errors of omission. So what about the AI that gives
you an answer, but there was a whole bunch of data that it
actually didn't reference. And maybe if that data were
referenced in the response from the AI, the answer would have
been different. So again, it's incumbent upon us
as we think about our use of these tools.
This digital assistive technology is going to be in
every part of our work day. It's already in many parts of
our life. I drove to work this morning.
I used a navigation application that, you know, referenced
traffic, OK. And by the way, I actually
didn't take the recommended course.
I applied judgement and I said, well, I know the flow of
traffic. I know preferred courses and
routing and so on. I'm gonna take the second
course. I applied judgement.
So I think that's the key here. We're ultimately accountable.
This is assistive technology and you know, we're not outsourcing
our thinking to the AI. We're applying judgement and
using it as data points to enable us to make good
decisions. And I'm, I'm glad you touched on
AI and hallucination and fabricating sources because that
was a concern that we heard from a lot of, a lot of people here
at NASA and AC turning to you. I mean, what does growth in this
field look like when it comes to missions and innovation and
opportunities here at NASA? I I think there's a few aspects
to this. One is inspiration inspiring us.
Technologies of all types inspire us to think differently,
innovatively. I also think we should leverage
these tools to help all of our directorates, including mission
support in terms of helping us think about how do we accelerate
our day-to-day practices separate from science and
engineering. And then finally, in some sense
we are an engineering and science agency, also a data
agency. And how can we uncover
knowledge, new knowledge from these various data sets we have
historically we're collecting today.
Our missions of the future are going to provide even more data
downstream that we can leverage and analyze from both human and
robotic missions across the board.
We should be working with experts of both internally and
externally to help us manage and ride this wave of innovation.
And finally, I think about this town hall I was mentioning to
someone earlier this morning. These are the kinds of town
halls I came to NASA for. These are the kinds of town
halls NASA should be having in the 21st century in terms of
leveraging technologies, leading the way of leveraging our data,
showcasing how we use it to other government agencies as
best practices or the practices of of use leveraging this
technology. So it's a pretty exciting time
at the agency of how we use these safely, responsibly, but
innovatively to achieve our mission faster and more bolder,
I think in more bold fashions if I could.
Add a little bit to what AC said too, because he talked about
here in the mission support realm, applying some of these
technologies and I mentioned we use Microsoft and today if
you're in a Teams meeting, you can already get an automated
transcript generated, right? It's listening to the audio and
doing a pretty good job of transcribing that meeting.
Well, one of the things that I would love to see happen and I
hope happens in the near future is we can take that transcript
and we can say give me a one page summary of that one hour
meeting and automatically the system gives that to you, right.
So that OK and we'll have to evaluate and see is that really
valid. But hopefully as the technology
matures, it will be a pretty valid synopsis of an hour long
meeting that is generated almost in the blink of an eye.
We've already been working to automate many processes over the
last years. We've saved 10s of thousands of
hours using existing technologies to automate manual
processes and that work is ongoing.
The mission support director at the NSSC working with various
organizations. And now if we are able to apply
these, you know, evolving AI capabilities, I think we'll be
able to automate much more. And I don't know about you, but
I don't hear a lot of people saying I've got so much free
time Jeff, give me more to do what I hear is I'm overworked
help, right? And so to apply some of these
technologies to address some of the the mundane tasks, to free
us up to do other tasks that we would love to get to, but we
don't have time to because we're wrapped up in the bureaucracy or
the process, right? I think there's a lot of
opportunity there. So it's not about replacing the
people, it's about enabling us to do more than we can today.
And I'm really excited about that.
And I, I want to pull the, the audience into the conversation
here, but before we do, I just want to see if there's anything
else that you all wanted to touch on or mention.
And here I am, the Chief data Officer and now Chief Data and
AI officer here at NASA. And I've talked less about data
than anyone else on the panel. So I want to say that, you know,
in Jeff's case of, let's say, the transcript of a meeting,
it's a very, very narrow data set.
You know, that's a pretty easy use case when you think about
it. What's much more complicated is
a large corpus of holdings, where you're now asking
generative AI to comb through a large corpus of holdings and
come up with some kind of logical conclusion to be able to
do that effectively. There's a a practice of data
management that's actually critical to the success of the
algorithm. You know, Eloquent algorithms
can really go bad with poor data.
So we also have a responsibility when we're thinking about our
use of AI is understanding the data that enables the AI to give
us the answer that it's providing.
And is that data complete or not complete?
Is it reliably sourced or not necessarily reliably sourced?
So think about, you know, AI also, you know, do we understand
the data? Do we understand the origins of
the data? Do we understand or have high
confidence in the accuracy of the data and its completeness?
Because if if the answer to those questions isn't yes, then
you know our confidence in the AI outcome should should be
declined or diminished. Yeah, I just add one thing.
So the four of us have had a lot of opportunities to talk about
AI over the last six months. And I think all four of us sort
of we're really excited about the opportunities that the
things like Jeff said that AI can do to help make our jobs
easier so that we can focus on the things that are new, that
are innovative. And I think we really are.
We're here to help. Yeah.
And I guess to add on to that a little bit about maybe what you
can do. And you asked a policy question
early and I don't think I really answered it right, because last
year I put out an agency policy on generative AI and said, hey,
at this point we don't have approved tools that are in the
environment. So no, we shouldn't be
installing things and using them.
And we're going to get to that. And we're still, you know,
working in that direction. But one of the things that was
noted is, hey, on your personal computers with, you know,
publicly available data, you can, you know, access and, and
so that's one thing I'd like to say right now, as we're
continuing to move forward to bringing tools into the NASA
environment, I would encourage you to play right now.
You can on your home computer, actually on your NASA computer,
right? We're not blocking certain
sites. You can go to
copilot.microsoft.com. You can go to ChatGPT today and
you can use your NASA computer. It's not a block site.
And you could just type in queries and questions, you know,
I'm taking a vacation this summer.
What's a five day itinerary for Mount Rainier National Park and
see what it says, right? Start to experiment with some
personal questions and things. Just be aware that you send
something out. It's out.
It's not in your possession anymore.
That's why we're saying, hey, no NASA sensitive data because it,
it's outside of our control. But on my phone, alright, my
personal phone, I've got a couple of these apps installed
and I asked ChatGPT to explain generative AI.
So let's have a little fun. So it did good, good, good
explanation. Then it said, OK, tell it to me
as a Limerick. So here we go.
There once was an AI so keen it learned from all things it had
seen it wrote and it drew made new from the old 2A marvel of
tech quite serene. And I said, OK, how about let's
tighten it up a little bit. How about as a haiku?
AI learns from all, creates new from what it knows.
Art and words unfold. So I would just encourage you to
experiment on your own, you know, little, maybe a little bit
of work. Time to go to some of the
available sites and start playing with some of the
prompts. Cause one of the things that we
see is the questions that we ask actually have a pretty
significant influence on the results that we get out of these
generative AI tools. And so prompt engineering,
right? What are the questions that
we're asking is going to be something that's going to be
important as we move forward. So I would just encourage
everybody to get a little smarter by experimenting a
little bit using non sensitive data, right?
Because pretty soon we're gonna have tools at your disposal that
will be able to use in our everyday work with sensitive
data internal to the agency or in approved cloud capabilities.
And the more you know now, the more prepared you'll be for that
future that's on the horizon. And if I can add one more thing,
Jeff and I are part of the NASA 24 Technology Work stream.
You may have heard of NASA 2040 of how we look at how the agency
should operate in the future. And on the technology work
stream, we're thinking about how do we work digitally leveraging
data sets. AI and Jeff and I are committed
to leveraging that work stream that the agencies provided us
under the guise of 2040 to help David and others using these
technologies. So I think there's a very great
synergy and we recognize it of this AI initiative, the NASA
2040 initiative, and how we can leverage 2040 to accelerate
everything we've talked about today.
Thank you all. And we're we're not done yet.
Got a lot more. If you're in the room and you do
have a question, we have a microphone up front and we would
invite you to come up and ask that question.
So while we give folks a chance to do that, we also received a
lot of questions at conference dot IO.
So I'm going to go ahead and give you guys a few of those so
we can get started. One of the questions when it
comes to ensuring that AI is being is being used is done in
an equitable manner. How will NASA ensure the AI used
by the agency doesn't develop unforeseen biases?
I can I can start with that question.
So I love the bias question only because we often think about the
bias as a negative and and I would offer this as an analogy,
right? Think about think about an
autonomous vehicle and think about the algorithm having a
bias. Do you want the autonomous
algorithm that drives your car to be biased towards speed and
performance or towards safety? I think everyone would admit
safety, right? We want some margin, we want
some threshold. We want to know that we're going
to be safe in the vehicle. So first of all, we have to look
at bias as there's positive bias, right?
And it's a very well worded question because it says
unforeseen bias. So what about the bias that we
don't understand? And by the way, with AI, you can
adjust that bias. So I can crank up or down the
margin of safety, OK? I can be very deliberate about
it. And this applies to so many
different use cases. So bias is a good thing when
applied appropriately. Now, unforeseen bias could be
something like, you know, address an outcome related to
some of the concerns the administration has referenced as
it relates to privacy. But let's say equitable
underserved communities being underrepresented in the outcomes
of AI. How do we address this?
You know, another thing could be age discrimination.
Think about a data set where the response to people who are,
let's say, older is different than the response to questions
about people who are younger. You have to understand the data
and you have to be on the lookout for those biases and you
have to test. So again, I go back to sort of
the accountability and one of the things we have to be careful
about as we onboard various different AI technologies that
are coming from our vendor partners is do we really
understand how they work? And have we thoroughly tested
them for bias and have we again understand the data?
Is the data leading the algorithm toward a biased
outcome or not? So that really is what it comes
down to. So we have often thought about
onborn technology from a security perspective in cyber.
Is it safe from a cyber perspective?
Now we have to add a set of attributes around, OK, is the AI
protected against bias? There are things like model
drift where AI can actually drift based on its use over
time, and we have to be on the lookout for that as well.
So part of what we'll be doing, and you'll see announcements
soon is the Summer of AI, which is a training initiative where
everyone in NASA is going to have an opportunity to learn
more about AI. It's literally a campaign.
It's going to be kind of a surge, if you will, of training
opportunity. So stay tuned for those
announcements, but I would encourage you, you know,
participate in those courses and learn about bias and learn about
how you can prevent some of the bias that would be unforeseen.
Learn about the data that enables an AI to do what it
does. And how do you, if you want to
take age bias out of an, an algorithmic outcome from an AI,
maybe you take the age parameter out of the data set, but you
have to be thinking about that. Or if there's other ways where
you can weight the outcomes differently as it relates to age
in that particular data set, that would be an alternative as
well. But again, just exercising
judgment and understanding the technology and not just treating
it as a black box and just assuming, well, it's seems right
most of the time, it must always be right.
Well, not necessarily. We've also heard from you all
today about, you know, there are so many organizations and tools
out there. This next question, how will
NASA's AI tools differ from AI tools used by the public?
And is NASA looking to partner with any of the leading private
AI organizations? Maybe I'll start with that.
In terms of the organizations, yes, I mean, I mentioned we work
with many, many different vendors and providers of
products. And so there are conversations
ongoing with some of those providers already in terms of
what their plans are and how those can potentially roll out
into the the government NASA environment.
As I mentioned, it can be a little bit frustrating because
it takes longer for tools, the kind of publicly available
generally known tools to to get into our hands in the government
environment. We are working on that front
with many, many different providers.
So there's that. And then there's another piece
going more towards the the mission technical side of
things. And we do have established in at
a FISMA low level, right. In terms of the the data
sensitivity, we have generative AI large language model
capability that's being you know, tested out by some some of
our folks within the agency. And we anticipate that by mid to
late summer we'll have that environment rated at a FISMA
moderate level. So we can start leveraging some
sensitive internal data and experimenting.
And so that's another, I think value in terms of engagement,
getting engaged with the AI community that is being formed
and that's that's growing right to be aware of what are some of
the capabilities that are emerging also to have your voice
in, Hey, what are some of the things we should be doing?
Right, because this is not the four of us up on this stage are
not the the know it alls as far as what's happening with Jenner
today. We've got a lot of smart people
across this agency and I think together we can guide some of of
the investments we've got limited time, limited resources,
we all do. And so how do we apply those
most effectively to advance this, you know, kind of journey
that we're on that's going to take involvement from all of us?
And we have a question here in the room, Sir.
Hi Moon Kim from OCFO. Thanks for this and super
excited to use Jenny AI or AI going forward.
Coming from a budget perspective since I'm from OCFO.
AI is not cheap, right? It cost GPU clouds, cloud
environment is expensive. In the middle of a budget
constrained environment. Do you foresee any of the AI
tools being openly available for everybody at NASA despite how
much budget you have in your your division in your office or,
or and do you see any tools that might be just free, like not
free, but Excel, you know, all attached to every single
computer like Excel or office. Any plans to navigate around
this? Constrained budget please.
Good question. I guess I'll start see if you
all have anything you want to add to that.
Margaret, the CF OS in the room somewhere I think right.
I saw her walk in there she is up front.
That was a question that you planted right.
So, and but that's a a great question because we we all know
that budgets are tight and companies need to make money and
so they'll be looking to leverage these new capabilities
to make profit. Sure that's that's what they
should do. So that's a valid question.
What we need to ask ourselves is the value of these capabilities.
So to back to what AC mentioned about 20-40 in the tech work
stream. So one of the things we're
taking a look at in that work stream is, hey, where should we
be investing more in technologies that can actually
enable the NASA mission to be successful maybe more rapidly,
maybe in different ways. And so the, the investment
question is a very real one. I think as an agency, we do need
to invest more in our technology, foundational
capabilities and some of these advancing capabilities.
And so honestly, that's a leadership conversation that
we'll be having, right. So as we take a look at some of
the products that we'll be rolling out and we see the cost
models and my guess is the cost models are going to continue to
evolve. That's what's happened in, you
know, the cloud world for the last five to 10 years.
The business models have evolved and changed and we've had to try
to understand and adapt to them. That's going to happen in the
generative AI space as well. So yes, I think we need to
invest. Yes, I think there needs to be a
set of capabilities that are just broadly available to the
NASA workforce. Go back to my example about the
summation of a hour long teams meeting, you know, if that is a
capability that rolls out, I don't think that the three
people that can pay for that should have access to it and
everybody else not so much, right.
So I think there should there should be and there will be a
general level of capability that we see value in providing to to
the broad workforce. And then I think they'll be
communities that will say there's value in the investing
in this tool that the whole agency doesn't need, but this
community does. And so we'll have to take a look
and understand what's available to us and how we optimize the
investments we make. Yeah, just following on a Jeff
statements on 2040 to give you a little bit of insight.
You know, we're having conversations with senior
leadership on these technology investments of how do we work
digitally. So we've divided those future
investment opportunities in various strings from
cybersecurity to data management to AI and having conversations
with Jeff and David and others to say in those dreams, what are
some investments over the next few years we need to start
making. And so I think I'm committed
personally along with Jeff and others to make sure that gets a,
a proper voice within the 2040 environment and they're offering
that to us. So I think that's a very
positive sign of recognizing Jeff, Scott, Jeff's budget over
the next few years. But how can we enhance it?
How can we enhance directorate funding to enable these tools to
come online and all these various streams that are pretty
important to how do we work digitally.
So that's going that, that is going on in terms of those
budget conversations, I think and those investment
opportunities. Thank you, super excited.
I, I might share one other thought and that is I talked
earlier about, you know, shared situational awareness.
The thing is let's not duplicate effort.
So if we know that one organization is exploring an AI
capability to enable some part of their business, you know,
having that situational awareness and not having to
recreate that elsewhere across NASA will go a long way to
helping us be efficient about how we pursue this technology.
So, and I think there's a governance activity related to
that as well. So when we think about shared
awareness, getting us all rowing in the same direction as it
relates to our pursuit of AI as a, you know, Team NASA, not as
individual organizations. Yeah.
And I think that that underscores a good point Dave
that I want to mention cause Dave in his new role as Chief AI
Officer, he he came and working in my organization as the Chief
data Officer, both data and AI. I think that's an apartment
description of of Dave's role. He's trying to facilitate and
coordinate for the benefit of the agency, right.
So that awareness, that understanding, pulling people
together to have the shared conversations is I think, a key
piece. And then it goes back to what I
mentioned about community. Getting folks involved, being
part of those conversations and helping us to frame up where
we're going as an agency is an opportunity I think that many
have. And we we have another question
here in the room. Hi, Jenny Modder, Art Director
for NASA Science. I work a lot with the creative
community, and we've noticed that uptick in generative AI
imagery falsely attributed to NASA appearing online and in
stock libraries. And I'm just curious if there's
any plans for authenticating NASA imagery to protect our
credibility or if that's a worthwhile investment in time
and energies. Thank you.
Soy there is a concern and everyone has probably heard the
term deepfakes right where you see the falsification of imagery
and even the the fact that AI can create an image that
actually didn't occur, a representation of something that
didn't occur. The good news, at least right
now at this moment in time, is, is that AI is actually pretty
good at detecting that, better than maybe a human might be
because there is a sort of a footprint associated with that
type of that type of misuse, if you will, of AI.
There are ways in which we can validate our content.
And I think this gets to also being careful about our security
and where we post content. And you know, if you want to go
and find authoritative representation of NASA imagery,
I would probably go to NASA as opposed to maybe a third party
where there's a potential that something could have happened
along the way. Other than to say that, I mean,
there are other more sophisticated ways of sort of
hashing and encrypting and doing other things with imagery,
digital rights protection type of technology.
We will have to look at that in our future as this is this is
sort of a fast evolving area. So your point is very well taken
and and we'll have to keep an eye on it.
Yeah. And I'd say this is, I mean it's
early days with the tools too and they are evolving.
So if you go back just a few months, some of the generative
AI tools that were out there would produce results, but they
wouldn't tell you how they generated those results.
Where did that come from? And now some of the tools will
cite that this is an AI generated image or they will
reference at the end of the output, they give you the three
or four primary sources that were used to derive that result.
So the tools are evolving too, I think.
So some of the concerns that that we have that are
legitimate, right? I'm hopeful that the
organizations developing these tools will help with some of the
solutions as well so that we won't have to create them
ourselves necessarily, but we'll be voicing the concerns.
And just one last thing to add to that, I think this is an
ongoing discussion, you know, throughout the country and the
world about what do we do about this and how do we identify.
And it's something that we're tracking.
I've been following a lot sort of the university professor
discussions around how do you identify AI?
And there are these AI tools. There's also sorts of things to
look for that identify it. And we're also establishing
these methodologies, like what Jeff said, where you actually
acknowledge your use of AI. And so I think we're going to,
we'll keep tracking those conversations, you know, across
the US government with the university community.
And this will evolve over time, but it is something we're very
aware that people are concerned about and we're watching.
Thank you. And I want to ask you all a
question that's coming from conference dot IO.
Is there any intention of obtaining a closed off
deployment of an LLM that could be used with internal data?
Yeah. And I, I touched on that little
bit, little bit earlier. We do have one that's rated at
the FISMA low level working to get that to be able to handle
some sensitive data. I will say that's not at a
production capability level though.
It's for sort of a, you know, AI early adopter kind of experts
experimentation phase right now. So we need to take a look at
within the agency, what kinds of capabilities do we need, do we
want, do we invest in going back to that resource question?
And then figure out what that looks like in terms of
deployment. But that's not to to prevent.
We have a lot of great mathematicians, computer
scientists with the agency, these large language models,
some of them are available, right.
So we might be doing some internal experimentation in
various organizations as well as we go forward.
But I do think it will be important to have a kind of
inside the wire capability that we're experimenting with.
And then in some cases for our missions, we'll be deploying
those. I think the other thing I would
add about the experimentation piece of it, why that is
important. Our contractor community,
academic community are using these tools and if we want to be
smart buyers, we need to understand how these tools
operate even in experimental mode if not in a production
mode. So once again, I think important
to have that inside the wire for multiple reasons.
And then I might just add one other thought and that is from a
foundational model perspective, I would fully expect that
they'll be foundation model development specific to NASA use
cases. I was at a conference earlier
this week where a colleague from DoD in the intelligence
community was talking about some of the concerns that they have
with some of the at a box models and not really understanding the
underlying process within the model itself.
And they were actually talking about perhaps developing models
for themselves as well for that very reason.
If we develop it, we understand how it works.
If we buy it, we may not understand it to the same
degree. So again, an emerging area.
And this is an area where we can learn across mission
directorates because the Science Mission Directorate has done
some work with foundation models and partnerships externally to
build them. Thank you all.
Anything else that you'd like to add or mention, I'll say.
That it's exciting. There's a lot going on tomorrow,
actually, we have our inaugural Artificial Intelligence
Strategic Working Group session kick off.
That's at 2:00 PM Eastern. Organizations have been asked to
identify a working group member. This is the beginnings of our AI
governance here at NASA. What we wanted to do is we
didn't want to start at the top and work down.
We wanted to start at start at a lower level and work up.
The other thing we're going to be doing at that AI strategic
working group is we're going to be doing a spotlight series
where we highlight various different levels of various
different activities that are occurring across NASA and make
people again, situationally aware that, oh, Ames is doing
this, Langley is doing that, God is doing something else.
Oh, science is working in this particular area.
That shared awareness will go a long way to helping us
understand not only the opportunity space, but also what
the technology can do to, well, understanding the technology
better. So there's that.
I already mentioned the summer of AI as an opportunity.
So I would say get involved there on on teams.
You'll find an AI community of practice.
Feel free to participate, join that.
That community practices as robust as the people who
participate and post and share and so on.
So lots of opportunity. Get to know your AI strategic
working group member for your organization and ask them how I
can help. And then please pursue the
training. I would love the statistics on
the AI summer of learning to be off the charts as far as how
many people consume that training and how many courses
people actually complete throughout the period.
So thank you. Yeah.
And I was at a conference recently and this is going to
build on what Dave just said. And there were discussion was on
AI and there it was a physician surgeon on the stage and she was
asked, is AI going to replace you?
Is AI going to replace doctors and surgeons?
And her response was great. She was like, I'm not worried
about AI replacing me, but I do think what's going to happen is
surgeons and doctors who don't learn to work with AI are going
to be replaced by surgeons and doctors who have learned to work
with AI. And so I think that's true for
us in terms of the future is going to involve us continuing
to adopt and learn new approaches and leverage
capabilities and technologies. And so I think it is incumbent
upon us to what they've just said, take advantage to learn to
see how we can grow personally, how we can maybe be more
successful, capable at our jobs because we're learning about
these new technologies. So I would just encourage all of
you to play experiments, have those conversations, take the
take some training classes over the next several months.
And I'm excited, you know, maybe six months from now what's
happening within NASA because we're taking advantage of some
of these capabilities. I'd like to thank our panelists.
Thank you all for joining us today.
And thank you for everyone who participated in today's town
hall. It has been an enlightening
conversation. And as we continue to explore
how artificial intelligence is moving NASA forward, NASA
employees can continue to engage.
We had a lot of questions that we did not get to, but we want
to invite you to to check out your point and continue to have
the conversation. And thank you again.
And we look forward to having more conversations around AI.
I'm Melissa Howell. Thanks for joining us, everyone.
Hey, thank you so much for listening today.
I really do appreciate your support.
If you could take a second and hit the subscribe or the follow
button on whatever podcast platform that you're listening
on right now, I greatly appreciate it.
It helps out the show tremendously and you'll never
miss an episode. And each episode is about 10
minutes or less to get you caught up quickly.
And please, if you want to support the show even more, go
to patreon.com/stage Zero. And please take care of
yourselves and each other, and I'll see you tomorrow.