Taking AI Innovation Further

1
00:00:00,000 --> 00:00:03,000
Where is AI truly taking hold and generating real value?

2
00:00:03,000 --> 00:00:12,000
One of the things that happened this past year is that we moved from experimentation to production. [music] As EVP of product and engineering at

3
00:00:12,000 --> 00:00:19,000
CoreWeave, Chen brings a unique perspective across use cases. Every time you solve one hard problem, a new problem comes

4
00:00:19,000 --> 00:00:26,000
along and the pace of change is actually something we've never experienced before. In this episode of AI Cloud

5
00:00:26,000 --> 00:00:34,000
Essentials, she highlights the pioneers using AI to make major leaps forward in productivity, innovation, and

6
00:00:34,000 --> 00:00:42,000
competitiveness. Those who lean in and try and are not afraid to fail are the one that are seeing the best outcomes.

7
00:00:42,000 --> 00:00:55,000
If you are in the C-suite, this episode is for you.

8
00:00:55,000 --> 00:01:02,000
Hello everyone, welcome to season 2 of the cloud essentials a podcast series brought to you by CoreWeave. Today we are

9
00:01:02,000 --> 00:01:13,000
talking about all the enhancements in the industry in terms of agentic AI from models to agents and I'm pleased to be so excited to be joined by Chen Goldberg.

10
00:01:13,000 --> 00:01:21,000
We are going to talk about the transition in the industry and Chen welcome to the show. Today I'm going to kick things off by asking you to talk

11
00:01:21,000 --> 00:01:28,000
about your journey. What excited you with in this transitioning uh phase in the industry? What excited you to join

12
00:01:28,000 --> 00:01:34,000
CoreWeave and the foundations of AI native cloud that you guys are working on?

13
00:01:34,000 --> 00:01:43,000
Thank you so much for having me and look I've been my entire career really focused on empowering innovation. Okay.

14
00:01:43,000 --> 00:01:52,000
I used to work in an IT organization, they're in a cloud provider. H and enabling innovation is always has this

15
00:01:52,000 --> 00:01:59,000
tension you know we have ideas and not always the technology can keep up right so the technology can

16
00:01:59,000 --> 00:02:08,000
hold us back of how quickly we can move and one of the things that I've been really fortunate in my career is to be part of the cloud transformation that

17
00:02:08,000 --> 00:02:15,000
happened about 15 years ago. So I'm part of the team that have brought Kubernetes to the world when I used to work at

18
00:02:15,000 --> 00:02:24,000
Google cloud h and what I loved about that time is that you know we had this cloud infinite resources elasticity

19
00:02:24,000 --> 00:02:32,000
speed scale but if you were just doing lift and shift to the cloud you cannot really use

20
00:02:32,000 --> 00:02:42,000
those resources on your behalf and what we've done with Kubernetes and actually an entire ecosystem what we call the cloud-native ecosystem.

21
00:02:42,000 --> 00:02:49,000
We created new set of tools that for the first time made some of those promises that we've been working on forever.

22
00:02:49,000 --> 00:02:58,000
Okay, like literally forever uh possible and you can you could see teams building application without like having a a big

23
00:02:58,000 --> 00:03:08,000
data center. So everybody would were able to uh to achieve that. And really our main goal back then was to what we call make the infrastructure boring.

24
00:03:08,000 --> 00:03:12,000
Okay, let developers just focus on development.

25
00:03:12,000 --> 00:03:19,000
Um, and then something has changed uh four years ago or something like that. H

26
00:03:19,000 --> 00:03:28,000
technology was actually moving ahead beyond what we thought was possible.

27
00:03:28,000 --> 00:03:29,000
Yeah.

28
00:03:29,000 --> 00:03:33,000
And and that's like you know like you know if you remember like what happened when ChatGPT came to the world.

29
00:03:33,000 --> 00:03:44,000
Yeah. Yeah, but they were like shocked and that was that strange, right? Like suddenly everything around this new technology have to start catching up.

30
00:03:44,000 --> 00:03:54,000
And for my experience, you know, doing this kind of lips in technology and innovation is not something that you can just like retrofit.

31
00:03:54,000 --> 00:04:03,000
And I really wanted to to look for a place especially me being you know in this journey in my career in infrastructure that we can really

32
00:04:03,000 --> 00:04:11,000
reimagine how cloud computing should look like in the AI era because there were new bottlenecks and new challenges

33
00:04:11,000 --> 00:04:18,000
and I was lucky enough to meet the KOI founders uh which not only we share uh

34
00:04:18,000 --> 00:04:27,000
the same passion we believe in the same vision they're also extremely fun to work with uh and you know the rest is

35
00:04:27,000 --> 00:04:32,000
history and we're very proud with everything we've been building being the first AI native cloud.

36
00:04:32,000 --> 00:04:41,000
Awesome. Awesome. So like minds do kind of come together. Awesome. You know I've been reading a lot and uh uh in the industry about how things are evolving

37
00:04:41,000 --> 00:04:49,000
very fast and you guys have a broad you know set of customers breadth and depth and very diverse set of customer base.

38
00:04:49,000 --> 00:04:56,000
uh as we have transitioned more from you know simple chatbots and assistants to agent applications and AI applications

39
00:04:56,000 --> 00:05:04,000
data is very very integral to it right and uh I would love to hear from you uh some of your customer you know case

40
00:05:04,000 --> 00:05:13,000
studies or customer wins that customers that you're working with and use cases where data and uh is very integral to their success and how are you guys kind

41
00:05:13,000 --> 00:05:20,000
of helping your customers you know internally we we keep joking that we like running towards the hard problems

42
00:05:20,000 --> 00:05:29,000
and and I actually had a customer meeting last week and that person was mentioning like that what they are seeing is that every time we solve like

43
00:05:29,000 --> 00:05:37,000
a big problem we we find a new one and I think that's something that uh we all need to get used to with the pace of

44
00:05:37,000 --> 00:05:46,000
change so you know as we were working with customers and moving from a more training

45
00:05:46,000 --> 00:05:53,000
and and inference and just scale and more customers One of the things that uh we worked really closely with one of our

46
00:05:53,000 --> 00:06:00,000
customers, Cohere. So Cohere is a a Toronto-based company, the builders of the H platform, the agent platform

47
00:06:00,000 --> 00:06:10,000
north. And what they said like, okay, we have this really powerful cluster. GPUs are amazing. Everything is up and running all the time. This is amazing.

48
00:06:10,000 --> 00:06:18,000
However, uh how can we get then now more data into the GPU? Okay, because again the bottleneck has just shifted and we

49
00:06:18,000 --> 00:06:27,000
want to make sure that we get the best utilization, the best efficiency and we were working with them on a creative way. something that was what we've done

50
00:06:27,000 --> 00:06:35,000
is actually we were leveraging our GPU nodes in order to build a new uh distributed cache caching mechanism

51
00:06:35,000 --> 00:06:42,000
mechanism especially for a running AI workloads and we were able to achieve up

52
00:06:42,000 --> 00:06:50,000
to 7 gigabytes per second per GPU of of throughput with that solution. So you know that's has been amazing. Of

53
00:06:50,000 --> 00:06:57,000
course, it's h created a we've learned that there are some new bottlenecks along the platform. But what it really

54
00:06:57,000 --> 00:07:03,000
allowed our customers is to process and achieve really a speed of light innovation.

55
00:07:03,000 --> 00:07:08,000
Awesome. Awesome. That's so uh must have made you and your team so proud of that.

56
00:07:08,000 --> 00:07:15,000
um you know when you think about your other customers that I kind of read about you know spanning in e-commerce and you know whether in different

57
00:07:15,000 --> 00:07:23,000
industries uh is there something that uh jumps out that you know couldn't have been done on the previous version of the clouds and where your AI native cloud

58
00:07:23,000 --> 00:07:31,000
really shines to support that use case would love to hear uh perspective on that one of the things that happened

59
00:07:31,000 --> 00:07:44,000
this past year is that we moved from experimentation to production you know if you think about it. What was the the uh the sentiment around AI a year ago?

60
00:07:44,000 --> 00:07:51,000
There was a lot of uncertainty. Uh people were not sure whether they will be able to successfully demonstrate

61
00:07:51,000 --> 00:07:59,000
value, create new use cases. H and that has completely changed in just 12 months. Right? So, one of the areas that

62
00:07:59,000 --> 00:08:08,000
definitely is leading that space is uh the the coding space, right? Like how engineers are working. But this is

63
00:08:08,000 --> 00:08:16,000
actually spreading extremely quickly uh to other areas. And I think this is where it's really important to build an

64
00:08:16,000 --> 00:08:25,000
AI native cloud because those customers not not only need to be able to access new technology, but they also care about

65
00:08:25,000 --> 00:08:33,000
reliability, security, enterprise readiness. And one of the customers that I'm really proud that we are able to partner with is Marcado Libre. So,

66
00:08:33,000 --> 00:08:42,000
Mercado Libra for folks who don't know is the largest e-commerce uh company and a financial service in South America

67
00:08:42,000 --> 00:08:49,000
and they came to us because they were experimenting and they saw amazing results with what they can do in

68
00:08:49,000 --> 00:08:58,000
reimagining their search capabilities in their e-commerce platform. Okay, so they call it like search 2.0 O and

69
00:08:58,000 --> 00:09:06,000
and the way that we were able to partner with them is allowing them to get the right signals

70
00:09:06,000 --> 00:09:10,000
of how to scale these new capabilities in production.

71
00:09:10,000 --> 00:09:17,000
Okay? Because sometimes you know you work with models only and you see the results which is of course great but when you are a company like Marceto

72
00:09:17,000 --> 00:09:25,000
Libre you are serving so many customers so many users and there are so many things else to consider

73
00:09:25,000 --> 00:09:33,000
be beyond just how the application works and that's what we've been working uh with that team. So that means allowing

74
00:09:33,000 --> 00:09:40,000
them to think about their operational excellence, what kind of signals, how does observability works, performance,

75
00:09:40,000 --> 00:09:48,000
security and that's been a really amazing journey and the feedback from them is that all of that specific

76
00:09:48,000 --> 00:09:54,000
special tools that we provide allows them to to innovate in the pace they need.

77
00:09:54,000 --> 00:10:02,000
Great. Great. And I think as you were saying like you know the ability to hyperpersonalize for the customers is something you know with the transformation and all the technology

78
00:10:02,000 --> 00:10:11,000
and the advancements in search that's fantastic to hear. Uh so since you've been a pioneer in the industry and your team and you are kind of leading the air

79
00:10:11,000 --> 00:10:18,000
you know the curve here. uh what are you seeing in the next 12 to 24 months that needs to be done to meet the ongoing

80
00:10:18,000 --> 00:10:26,000
demands of innovative customers and how do you kind of you know uh support their uh you know needs right now

81
00:10:26,000 --> 00:10:34,000
what's interesting is that we are really uh just at the beginning of this journey yeah and and you know I mentioned before

82
00:10:34,000 --> 00:10:42,000
that every time you solve one hard problem a new h problem comes along and the pace of change

83
00:10:42,000 --> 00:10:45,000
is actually something we've never experienced before. Yeah.

84
00:10:45,000 --> 00:10:53,000
Uh so I I'm not I don't think that we are excellent in predicting the future as humans. But if you ask me what are I

85
00:10:53,000 --> 00:11:00,000
think are the main focus areas for us as a company, it would be really two things. One, we want to make sure that

86
00:11:00,000 --> 00:11:09,000
it's easy for everyone to innovate with AI, right? So how can we really lower that barrier to entry? H and we're doing

87
00:11:09,000 --> 00:11:16,000
that across the stack. Okay, we're doing it if you think about our weights and biases models and weave and our

88
00:11:16,000 --> 00:11:23,000
inference service making sure that uh for developers and researchers accessing

89
00:11:23,000 --> 00:11:31,000
very powerful infrastructure and models with the right performance with the right security is easy.

90
00:11:31,000 --> 00:11:40,000
uh and also helping you know platform teams enterprise customers have access for the right tools and

91
00:11:40,000 --> 00:11:49,000
services they need in order to build those new applications. So so that's like really one area uh of focus of ours. Uh the second thing that we are

92
00:11:49,000 --> 00:11:57,000
really focusing on is something that we uh h take pride in. I I believe that speed matters.

93
00:11:57,000 --> 00:12:05,000
Okay. You know we talked about how fast the the industry is moving but what it actually means that that's how fast people are moving

94
00:12:05,000 --> 00:12:13,000
right like innovation is not happening on its own and we are optimizing for speed of the customers

95
00:12:13,000 --> 00:12:21,000
like really when you know when we're asking ourselves questions internally is what can we do to achieve faster experimentation

96
00:12:21,000 --> 00:12:28,000
and we're building tools that allows you to do get uh evas we're building our own agents

97
00:12:28,000 --> 00:12:35,000
to help researchers and developers and we're also um making ourselves available. I think one of the things you

98
00:12:35,000 --> 00:12:43,000
know when we think about pace of innovation it means that uh not only you need to have the right resources you need to be able to connect with people

99
00:12:43,000 --> 00:12:51,000
faster solve problem fers make decisions faster h and uh we are showing up in

100
00:12:51,000 --> 00:13:01,000
that way and if I may maybe I will just give one example of one of our customers uh so Mistral is a is a French-based

101
00:13:01,000 --> 00:13:11,000
company they are a foundation model builder and actually one of our first customers of CoreWeave and it's been

102
00:13:11,000 --> 00:13:20,000
amazing that the journey with them partnering with them h they they'll take pride in being lean we take pride in being lean and together with them we've

103
00:13:20,000 --> 00:13:26,000
developed what we call direct to expert okay it means that how we partner with them allowing their researchers to talk

104
00:13:26,000 --> 00:13:33,000
directly with our engineering team and that's something that by doing that we

105
00:13:33,000 --> 00:13:41,000
as the team we are we are working as one we can move really fast and I know they appreciate and we appreciate because we get to learn a lot about h their work

106
00:13:41,000 --> 00:13:49,000
and how we can be more helpful yeah I think we always kind of talk about collaborative partnership to bring in the best of the industry so that's

107
00:13:49,000 --> 00:13:58,000
fantastic uh you know and as you were uh speaking about your customer scenarios I was thinking you know it's all about you know that how things are not just being

108
00:13:58,000 --> 00:14:09,000
done for experimentation anymore is being deployed in production so people need the end to to end scale, end-to-end monitoring, end-to-end evaluation and production scale. So that's fantastic.

109
00:14:09,000 --> 00:14:17,000
Uh it's really heartening to see from the outside that what you and your customers are able to achieve and the journey continues. Uh what would you you

110
00:14:17,000 --> 00:14:27,000
know your viewers who are listening to us today, what would be your parting advice? What would you tell them? And uh you know I'm sure they'll be very very excited to hear from you.

111
00:14:27,000 --> 00:14:32,000
I will uh tell everybody what I'm telling my team.

112
00:14:32,000 --> 00:14:40,000
lean in. What we have seen is that those who lean in and try and

113
00:14:40,000 --> 00:14:49,000
are not afraid to fail are the one that are seeing the best outcomes from this new technology. And I and I think you

114
00:14:49,000 --> 00:14:57,000
know it's it's it's actually deep because there's a lot of conversation that are saying talking about how can you get the best out of AI and and what

115
00:14:57,000 --> 00:15:04,000
I've seen is that people that know how to bring bring their experience and

116
00:15:04,000 --> 00:15:13,000
expertise and lean in it becomes like your superpower. Uh yesterday I was talking with a a team within my an

117
00:15:13,000 --> 00:15:23,000
engineering team within my organization here at CoreWeave and I I asked them like how do you guys feel now with all the the coding agents that we are using?

118
00:15:23,000 --> 00:15:24,000
Yeah.

119
00:15:24,000 --> 00:15:28,000
And they they said like it's a force multiplier. Yeah.

120
00:15:28,000 --> 00:15:35,000
Like we can do more we accomplish more things. We can actually work on the the hardest problems uh with this new set of

121
00:15:35,000 --> 00:15:43,000
tools. And I believe we will see this kind of effect in every industry and that's exciting. I mean a little bit

122
00:15:43,000 --> 00:15:51,000
scary for sure but I imagine that's always how it feels. Yeah. But as you said you know the pace of innovation we have never seen this kind of a pace of

123
00:15:51,000 --> 00:16:01,000
innovation and it's changing right in front of our eyes so rapidly. But having said that it's not an option to sit on the side and the fence. You have to lean in. You have to kind of play with it.

124
00:16:01,000 --> 00:16:08,000
But also I also joke that you know you have to fail fast. you know, you don't kind of prolong the uh the pro pilot

125
00:16:08,000 --> 00:16:13,000
purgatory kind of thing. But thank you so much, Chen. It was so wonderful to have you in this particular episode.

126
00:16:13,000 --> 00:16:22,000
People are going to really be uh very very happy listening to what you're doing. And with that, I'd like to thank on your behalf and my behalf, everyone

127
00:16:22,000 --> 00:16:28,000
who listened to us and stay tuned. We'll continue this journey. And thank you once again Chen.

128
00:16:28,000 --> 00:16:32,000
Thank you so much for having me.

‍