Thank you for your many comments and questions on HN and Twitter! We assembled a quick Twitter space to discuss it, > 1,000 AI engineers tuned in. Play it here!
We are watching a generation of AI applications "shift to the right" thanks to the emerging capabilities of the underlying model and the availability of open source/API.
Various AI tasks that used to take 5 years and a research team to complete in 2013 now require only API documentation and free afternoons in 2023.
As we discussed in the spatial chat, the API line is permeable - AI engineers can tweak/host models to the left, and research engineers can also build on the API to the right! But their comparative advantages and "base camp" are clear
However, the devil is in the details - the challenges of successfully evaluating, applying and productizing AI are endless:
- Models: From evaluating the largest GPT-4 and Claude models, to the smallest open source Huggingface, LLaMA and other models
- Tools: From the most popular chained, retrieval, and vector search tools such as LangChain, LlamaIndex, and Pinecone to the emerging autonomous proxy landscape (here's Lilian Weng's must-read review)
- News: On top of that, the number of papers, models, and techniques published each day grows exponentially with increased interest and funding, so much so that mastering it all is almost a full-time job.
I take this seriously and literally. I think it's a full-time job. I think software engineering will spawn a new subdiscipline that specializes in the application of AI and effectively leverages the emerging stack, just like "site reliability engineers," "DevOps engineers," "data engineers," and "analytics engineers."
Every startup I know has some sort of #discuss-AI slack channel. These channels will move from informal groups to formal teams, just as Amplitude, Replit, and Concept have done. Thousands of software engineers dedicated to producing AI APIs and OSS models, whether during company hours or evenings and weekends, in the company Slack or independent Discords, will be specialized and converged on one title – AI Engineer. This is probably the most in-demand engineering job of the decade.
AI engineers can be found everywhere, from big companies like Microsoft and Google, to independent hackers like Figma (acquired through Diagram), Vercel (e.g. Hassan El Mghari's viral RoomGPT) and Concept (e.g. Ivan Zhao and Simon Last with Notion AI) like Simon Willison, Pieter Levels (Photo/InteriorAI) and Riley Goodside (now at Scale AI). They made $300,000/year doing rapid engineering at Anthropic and $900,000 in construction software at OpenAI. They are spending free weekends researching ideas at AGI House and sharing tips at /r/LocalLLaMA.
What they have in common is that they are taking advances in AI and shaping it into real products used by millions of people, almost overnight.
Not a single doctor in sight. When shipping AI products, you need engineers, not researchers.
I call attention to this trend, not to start it. On Indeed, ML engineers work 10 times as many as AI engineers, but the higher growth rate of "AI" leads me to predict that this percentage will reverse in 5 years.
Monthly job trends for each HN recruiter
All positions have flaws, but some are useful. We are both wary and tired of the endless semantic debate about the differences between AI and ML, and are well aware that the regular "software engineer" role is more than capable of building AI software. However, the recent Ask HN question on how to get into AI engineering illustrates the fundamental perception that still exists in the market:
Most people still think of AI engineering as a form of machine learning or data engineering, so they recommend the same prerequisites. But I assure you that none of the efficient AI engineers I mentioned above have done the equivalent of Andrew Ng Coursera's course, nor do they know PyTorch or the difference between a data lake or a data warehouse.
In the near future, no one would recommend starting AI engineering by reading Attention is All You Need, any more than you would start driving by reading the schematic of a Ford Model T. Of course, it's always helpful to know the basics and history, and it really helps you find ideas and efficiency/capability gains that you haven't yet agreed upon. But sometimes you can just use products and learn about their quality through experience.
I don't want this kind of "flipping" of the course to happen overnight. It's human nature to want to fill out a resume, fill out a market map, and stand out by referencing more in-depth topics. In other words, Prompt Engineering and AI Engineering will feel inferior to people with good data science/ML backgrounds for a long time. However, I think pure demand and supply economics will prevail.
- The basic model is a "minority shot learner", showing contextual learning or even zero-shot transfer ability, which is beyond the original intention of the model trainer. In other words, the people who create the models don't fully know what they're capable of. People who are not LLM researchers can discover and exploit capabilities by spending more time on models and applying them to areas that are underestimated by research (e.g., Jasper with copywriting).
- Microsoft, Google, Meta, and large basic model labs have monopolized scarce research talent, essentially offering "AI research-as-a-service" APIs. You can't hire them, but you can rent them – if you have software engineers on the other end who know how to work with them. There are ~5000 LLM researchers in the world, but ~50m software engineers. Supply constraints dictate that the "in between" class of AI engineers will rise to meet demand.
- OpenAI/Microsoft was the first, of course, but Stability AI kicked off the GPU arms race by highlighting their GPU arms race.
- 4,000 GPU clusters.
- Remember October 2022?
- Since then, it's become commonplace for new startups like Inflection ($1.3b), Mistral ($113 million), Reka ($58 million), Poolside ($26 million), and Concontext ($20 million) to raise huge seed rounds in order to own their own hardware. Dan Gross and Nat Friedman
- It was even announced that Andromeda, their $100 million, 10 exaflop GPU cluster is dedicated to the startups they invest in. The global chip shortage is reflexively creating more shortages. AI engineers at the other end of the API line will have more capacity to use models than to train them.
- Fire, prepare, aim. Instead of requiring data scientists/ML engineers to do laborious data collection exercises before training a single domain-specific model and then putting it into production, product managers/software engineers can prompt LLM and build/validate product ideas before obtaining specific data for fine-tuning.
- Assuming the latter is 100-1000 times more than the former, the "fire, prepare, aim" workflow that prompts LLM prototypes allows you to be 10-100 times faster than traditional ML. As a result, AI engineers will be able to verify AI products, say, 1,000-10,000 times cheaper. It's waterfall with agility, again. AI is agile.
- Data/AI has traditionally been very Python-centric, and the first AI engineering tools like LangChain, LlamaIndex, and Guardrails came from the same community.→ However, there are at least as many JavaScript developers as Python developers, so tools are now increasingly catering to this widely expanded audience, from LangChain.js and Transformers.js to Vercel's new AI SDK. The expansion and opportunity of TAM is dramatic.
- Generative Artificial Intelligence and Classifier Machine Learning ." Generative artificial intelligence"
- Being a term has fallen out of favor, giving way to other analogies such as "inference engine", but still helps to succinctly illustrate the differences between existing MLOps tools and groups of ML practitioners, as well as the escalating, very different roles, preferably using LLM and text-to-image generators. While the existing generation of ML may focus on fraud risk, recommender systems, anomaly detection, and feature storage, AI engineers are building writing apps, personalized learning tools, natural language spreadsheets, and Factorio-like visual programming languages.
Whenever a subgroup emerges with a completely different background, speaks a different language, produces a completely different set of products and uses a completely different set of tools, they end up splitting into their own.
6 years ago, Andrej Karpathy wrote a very influential article describing Software 2.0 – contrasting the "classic stack" of hand-coded programming languages that precisely model logic with the new "machine learning" neural network stack of approximate logic, enabling software to solve more problems than human modeling. He went on to note this year that the hottest new programming language was English, which ended up filling in the unmarked gray area in his chart.
Update: Kapathi responds! There are some differences!
Last year, Prompt Engineering was a meme about how work will change as people start putting GPT-3 and steady diffusion to work. People derided AI startups as "OpenAI wrappers" and feared that LLM applications were proving vulnerable to prompt injection and reverse prompt engineering. Can't find a moat?
But one of the biggest themes of 2023 is re-establishing the role of humans writing code to orchestrate and replace the power of LLM, from the $200 million behemoth Langchain > to the Nvidia-backed Voyager, demonstrating the clear importance of code generation and reuse (I recently attended Harrison's Chain & Agent webinar). Prompt Engineering has been both exaggerated and will continue to exist, but the re-emergence of the software 1.0 paradigm in software 3.0 applications is both an area of opportunity/confusion and a void created for a bunch of startups:
If you don't market maps, are you even a VC?
Of course, it's not just human-written code. My recent ventures with smol-developer, the larger range of gpt-engineers, and other code-generating agents such as Codium AI, Codegen.ai, and Morph/Rift will increasingly be part of the AI engineer's toolkit. As human engineers learn to harness AI, AI will increasingly work in engineering until one day in the distant future we look up and can no longer tell the difference.
Builders need a place to talk about turpentine. That's why, after months of small gatherings, we're now announcing the first standalone AI conference for builders: the AI Engineer Summit!