Databricks VP of AI says the AI talent wars are just getting started
December 21, 2024

Databricks VP of AI says the AI talent wars are just getting started

In this final issue of the year, I focus on the war for AI talent, a topic I’ve been covering since launching this newsletter two years ago. Read on for the latest news from inside Google and Meta this week.

But first, I need your questions about my planned mailbag issue for the first issue of 2025. Submit a question via this form Or leave them in the comments.

‘It’s like finding LeBron James’

This week, data blocks declare This is the largest funding round ever known to a private technology company. The artificial intelligence enterprise is in the final stages of raising $10 billion in funding, nearly all of which will be used to buy back shares from vested employees.

In the tech industry, how companies handle payroll is often unknown, even though these strategies play a crucial role in determining which company succeeds faster. As I introduced before, the competition for artificial intelligence talent is fiercest.

In order to better understand the development trends in 2025, this week I visited Naveen RaoVice President of Artificial Intelligence at Databricks. Rao is one of my favorite people to talk about the AI ​​industry. Not only is he technically proficient, but he also has business acumen and has successfully sold several start-up companies. His last company, MosaicML, was sold to Databricks in 2023 for $1.3 billion.

Our conversation below touches on the logic behind Databricks’ massive funding round, which specific AI talent remains scarce, why he doesn’t think AGI is imminent, and more.

The following conversation has been edited for length and clarity:

Why is this round primarily about helping employees sell stock? Because $10 billion is a lot. You can do a lot with it.

The company is a little over 11 years old. There are employees who have been here for a long time. This is a way to provide them with liquidity.

Most people don’t understand that this doesn’t go onto Databricks’ balance sheet. This will largely provide liquidity for past employees, [and] Mobility of existing and new employees. It ends up being neutral in terms of dilution because they are already existing stocks. The shares have been allocated to employees, which allows them to sell the shares to pay the taxes associated with the shares.

To what extent is the rapid rise in valuations of artificial intelligence companies related to the war for talent?

It’s true. The key here is that this isn’t just pure AI talent – the people coming up with the next big thing, the next big paper. We’re definitely working on hiring these people. A complete software and cloud infrastructure needs to be built to support these things. When you build a model and want to extend it, that’s not actually AI talent per se. This is infrastructure talent.

The sentient bubble of artificial intelligence around us creates an environment where all this talent is recruited in large numbers. We need to stay competitive.

Who is most active in setting the market price for AI talent?

OpenAI does exist. Artificial. Amazon. Google.yuan. xAI. Microsoft. We are constantly competing with all of these companies.

Will you limit the number of researchers capable of building new cutting-edge models to less than 1,000?

Yes. That’s why the war for talent is so fierce. Researchers have unprecedented influence in organizations. One researcher’s idea can revolutionize a product. This is something new. In the semiconductor world, people who come up with new transistor architectures have this kind of influence.

This is why these researchers are so sought after. The people who come up with the next big idea and the next big unlock can have a huge impact on a company’s ability to win.

Do you think the talent pool will expand or be limited in the near future?

I see some aspects of the pool expanding. These roles are expanding due to the ability to set up the appropriate infrastructure and manage it. The top researcher side is the hardest part. it’s like looking for LeBron James. Not many people have the ability to do this.

I would say turning point acquisition A lot of it is driven by this mentality. These startups are concentrated with top talent, and the fees people pay sound ridiculous. But it’s not ridiculous. I think that’s why you see Google Recruiting Norm Shazer. hard to find another Norm Shazer.

One of the people at my previous company, Nervana, was arguably the best GPU programmer in the world. He is now at OpenAI. Every inference that happens on an OpenAI model is run through its code. You start calculating the downstream costs and it’s like, “Oh my gosh, this guy saved us $4 billion.”

“You start calculating the downstream costs and it’s like, ‘Oh my gosh, this guy saved us $4 billion.'”

What advantages do you have when trying to hire a Databricks researcher?

You start to see some selection bias across different candidates. Some are AGI or busts, and that doesn’t matter. This is a huge motivator for some of the brightest minds. We believe we will achieve general artificial intelligence by building products. When people use technology, it gets better. This is part of our promotion.

Artificial intelligence is on a huge growth spurt, but it has also reached the peak of its hype and is on the decline. Gartner Hype Cycle. I think we’re on the downside right now, and Databricks has built a very strong business. That’s very attractive to some people because I don’t think we’re that susceptible to the hype.

Do the researchers you talk to really believe that general artificial intelligence is coming? Is there a consensus on when it will come?

To be honest, there’s not a lot of consensus. I’ve been working in this space for a long time, and I’ve been vocal about it not being around the corner. Large language models are a great technology. By building a great product around it, it can bring huge economic improvements and efficiencies. But this is not the spirit of what we used to call artificial general intelligence (AGI), which is human or even animal-like intelligence.

These things do not create magical intelligence. They are able to more easily segment the space we call facts and patterns. This is different from constructing causal learners. They don’t really understand how the world works.

you may have seen Ilya Suzkver speech. We are all groping in the dark. Expansions are a big unlock. It’s only natural that many people feel passionate about this. It turns out we weren’t solving the right problem.

Is the new idea of ​​AGI a test-time calculation or an “inference” method?

No, I think this is very important for performance. We can improve the quality of answers, potentially reducing the likelihood of hallucinations and increasing the likelihood of fact-based answers. This is definitely a positive for the field. But does this solve the fundamental problem of the AGI ethos? I don’t believe that’s the case. I’m also glad I made the mistake.

Do you agree with the idea that there is a lot of room to build more good products with existing models because they are so powerful but still limited by compute and access?

Yes. Meta started a few years later than OpenAI and Anthropic, but they have basically caught up, and xAI is catching up very quickly. I think this is because the rate of improvement has essentially stopped.

Nilay Patel compared the AI ​​model competition to the early days of Bluetooth. Everyone keeps saying there is more advanced bluetooth but my phone still won’t connect.

You see this in every product cycle. The first few versions of the iPhone were much better than those that came before. Now, I can’t tell the difference between a phone from three years ago and a new phone.

I think that’s what we’re seeing here. How we leverage these LL.M.s and their built-in distribution to solve business problems is the next frontier.

elsewhere

  • Google becomes flatter. CEO Sundar Pichai The company reportedly told employees this week that a series of layoffs at the company had reduced the number of managers, directors and vice presidents by 10%. business insider Multiple employees I spoke with also heard these comments. Relatedly, Pichai also took the opportunity to add the character trait of “fighting” to his internal definition of “Googleyness.” (Yes, that’s true.) He took issue with the top employee-voted question about whether layoffs would continue, although I’m told he did note that there will be “overall” headcount growth next year.
  • Meta cut a perk. File this under “Sad Violin”: I’ve learned that starting in early January, Meta will stop offering free electric vehicle charging at its Bay Area campus. Keep your head high, Metamates.

What else you should know

Job board

Some noteworthy moves this week:

  • Yuan promotion John Hagerman Reports to Chief Revenue Officer Reports to Chief Operating Officer Javier Olivan. Another report from Ollivan, Justin Osofskyhas also been promoted to head of partnerships across the company, including Llama’s go-to-market strategy.
  • Alec RedfordAn influential senior OpenAI researcher who authored the original GPT research paper is leaving, but will obviously Continue to work with the company in some capacity. and Shivakumar Venkata RamanThe man recently poached from Google to lead OpenAI search efforts has also left.
  • Co-founder and CEO of Coda Shishir Mehrotra Grammarly will now run as well The two companies are mergingwith Grammarly CEO Rahul Roy Chowdhury Continue to serve as a member of the Board of Directors.
  • Tencent removes two directors David Wallerstein and Ben Federfrom the Epic Games Board of Directors Ministry of Justice stated Their participation violated antitrust laws.
  • Former Twitter CFO Ned Siegel already percussive Serves as Director of Housing and Economic Development for the City of San Francisco.

More links

If you haven’t received the new issue yet command line, Don’t forget to subscribe edgewhich includes unlimited access to all our stories and an improved online advertising experience. You will also get Access complete files past issues.

As always, I’d love to hear from you, especially if you have suggestions or feedback. Reply here and I’ll get back to you, or Send me a ping securely with Signal.

2024-12-20 20:26:09

Leave a Reply

Your email address will not be published. Required fields are marked *