Sam Altman: Next steps for OpenAI

The goal of OpenAI is to reduce the "cost of intelligence" as much as possible.

Written by: Raza Habib

Compiler: SinoDAO

Last week, I had the pleasure of sitting down with Sam Altman and 20 other developers to discuss OpenAI's API and their product plans. Sam discussed it very openly. Discussions touched on practical developer issues as well as broader issues related to OpenAI's mission and the impact of AI on society. Here is a summary of key points:

1 Currently OpenAI is severely limited by the GPU

A common theme that came up repeatedly in the discussion was that OpenAI is currently very GPU-bound, which delays many of their short-term plans. The biggest customer complaints are the reliability and speed of the API. Sam acknowledged their concern and explained that much of the problem was due to a shortage of GPUs.

There is currently no way to roll out the longer 32k context to more people. OpenAI has yet to overcome the O(n^2) scaling problem of attention mechanisms, so while it seems likely that they will be rolling out 100k to 1M context windows soon (this year), larger windows will require research breakthroughs.

The fine-tuning API is currently also bottlenecked by GPU availability. They have not yet used efficient fine-tuning methods like Adapters or LoRa, so fine-tuning operations are computationally expensive. Better fine-tuning support will be provided in the future. They might even offer a marketplace for community contributed models.

Dedicated capacity provision is limited by GPU availability. OpenAI also offers dedicated capacity, providing customers with a private copy of the model. To use this service, customers must be willing to pre-commit a spend of $100,000.

2 OpenAI's near-term roadmap

Sam shared what he considers to be an interim near-term roadmap for the OpenAI API.

2023:

Cheaper, faster GPT-4 - that's their top priority. In general, OpenAI's goal is to reduce the "cost of intelligence" as much as possible, so they will strive to continue to reduce the cost of API.

Longer context windows - context windows up to 1 million tokens are feasible in the near future.

Fine-tuning API - The fine-tuning API will be extended to the latest models, but the specific form will be determined according to the real needs expressed by developers.

Stateful API - when you call the chat API today, you have to pass the same conversation history over and over again and pay the same

Markup fees. A version of the API that remembers conversation history will be available in the future.

2024:

Multimodal - This is part of the GPT-4 release, but will not be rolled out to everyone until more GPUs come online.

3 The plugin "doesn't have PMF yet", it may not appear in the API for the time being

Many developers are interested in getting the ChatGPT plugin through the API, but Sam said he doesn't think it will be released for a while. The use of plug-ins in addition to browsing shows that they are not ready enough. He made the point that a lot of people think they want to put their application inside ChatGPT, but what they really want is to embed ChatGPT in their own application.

4 OpenAI will refrain from competing with their clients, except ChatGPT

Quite a few developers said they were nervous that OpenAI might release a product that would compete with them. Sam said that OpenAI will not release more products outside of ChatGPT. Great platform companies have a history of killer apps, and ChatGPT will allow them to improve the API by becoming customers of their own products, he said. ChatGPT's vision is to be a super-intelligent work assistant, but there are many other GPT use cases that OpenAI won't touch.

5 Governance is needed, but open source is just as important

While Sam called for future models to be regulated, he doesn't think existing models are dangerous and thinks that regulating or banning them would be a major mistake. He again emphasized the importance of open source, and said that OpenAI is considering open source GPT-3. Part of the reason they haven't been open sourced is because he's skeptical about how many people and companies are able to host and deliver large LLMs.

6 Scaling laws still apply

There have been a lot of articles lately claiming that "the age of gigantic AI models is over". This is not accurate.

OpenAI's internal data shows that the scaling laws of model performance still apply, making models larger will continue to lead to better performance. The scaling rate cannot be kept constant because OpenAI has scaled up the model millions of times over a few years, and doing so in the future will not be sustainable. That doesn't mean OpenAI won't keep working on making the models bigger, it just means they're likely to double or triple in size every year, rather than multiply.

The fact that scaling laws still apply has important implications for AGI's development timeline. The scaling hypothesis is the idea that we probably already have most of the groundwork needed to build AGI, and that most of the work left is to scale existing methods to larger models and larger datasets. If the era of scaling is behind us, then we might expect AGI to be even further away. The fact that scaling laws still apply strongly suggests a shorter timeline.