Behind digital dialogues

In an age where seeking movie recommendations from Google or asking Alexa about the weather is habitual, the technological prowess enabling these dialogues often goes unnoticed. Large Language Models (LLMs) like ChatGPT, GPT-4, and BloombergGPT are the linchpins of these interactions, yet they entail a significant financial commitment.

OpenAI’s ChatGPT, famed for its conversational capabilities, offers a subscription service, ChatGPT Plus, at USD 20 per month for an enhanced user experience. However, the day-to-day operational cost to OpenAI is a stark contrast, estimated at around USD 70,000. Moreover, the training of GPT-4 reportedly cost OpenAI tens of millions of dollars, emphasising the monumental financial undertaking involved in creating such advanced models. An expert insight from a report by arize.com accentuates the financial burden, stating, "Building a LLM can be extremely expensive. GPT-4, for example, reportedly cost USD 100 million to train. There’s also a lot of risk. With little room for error, you could end up wasting thousands or even millions of dollars — leaving you with only a suboptimal model."

ChatGPT has a significant cost of building and operation due to the vast processing power necessary to provide answers to user requests, which resulted in OpenAI investing half a billion dollars last year. Given the price of computing power, Dylan Patel, principal analyst at consulting firm SemiAnalysis, has estimated that ChatGPT would cost OpenAI over USD 700,000 per day to use! Businesses must thoroughly assess the operating costs of training and hosting their own LLMs. It's uncommon to find a company ready to invest millions of dollars to develop an LLM from scratch or host it for a sizable user base. Yet, there are companies undertaking this at scale, typically after a clear return on investment (ROI) has been demonstrated.

OpenAI’s CEO, Sam Altman, recently shed light on the financial gravity of scaling AI models during a discussion at the Massachusetts Institute of Technology. He remarked that the era of "scaling is all you need" has reached its limit due to the high costs of training and running large language models. The training process for ChatGPT alone necessitated over USD 100 million and engaged more than 10,000 GPUs. Meanwhile, Tesla has created its own supercomputer called “Dojo” for AI and machine learning, especially for working with video data from its cars. Although Tesla already has a powerful NVIDIA GPU-based supercomputer, the new Dojo computer uses chips and a setup entirely designed by Tesla. Elon Musk has also reportedly secured 100,000 GPUs for an internal artificial intelligence (AI) initiative at Twitter.

Several large enterprises have been investing in training foundational models. OpenAI’s GPT-4’s pricing structure for businesses, ranging from USD 0.03 to USD 0.12 per 1,000 tokens, hints at the underlying costs involved in maintaining such models and providing services on top of them. BloombergGPT, Bloomberg’s finance-specific foundational model with a substantial training phase, requiring nearly 1.3 million hours of GPU training time, also underscores the hefty financial commitment, although the exact cost remains undisclosed. GitHub Copilot, a collaboration between Microsoft and OpenAI, is another case in point. Priced at USD 10 per month for users, reports suggest that Microsoft incurs a loss of more than USD 20 per user per month on average, with some heavier users costing the company as much as USD 80 every 30 days. It's part of a broader strategy as Microsoft aims to create their own large language models that are more budget-friendly and compact compared to OpenAI’s offerings. Moreover, in 2019, Microsoft invested USD 1 billion in OpenAI, half in the form of Azure credits, to collaboratively work on AI models.

On the other end of the spectrum, new entrants in the AI domain and startups have rallied significant funds to fuel their ambitions. For instance, Adept AI, founded in 2022 by former Google and DeepMind engineers, garnered an impressive USD 415 million, showcasing a strong investor belief in their venture. The funding aided in the development of ACT-1, an Action Transformer designed to execute complex tasks from textual prompts, signifying a move towards creating AI assistants capable of handling intricate operations akin to human capability. Meanwhile, Inflection AI, with a mission to create an authentic personal assistant, secured USD 225 million in a Series A funding round, assembling a team of seasoned AI researchers from eminent organisations like DeepMind, Google, Meta, Microsoft, and OpenAI. Their product, Pi, aims to redefine user interaction with AI, emphasising a personalised, user-centric approach.

On a different spectrum, Runway, established in 2018, has raised USD 141 million to develop AI Magic Tools, a suite aimed at revolutionising video editing and film making processes. Their latest USD 100 million Series D round led by Google underscores the transformative potential of generative AI tools in democratising creative domains. Similarly, AI21 Labs, based in Tel Aviv, has raised USD 118.5 million to enhance text interaction through AI, offering products like Wordtune and AI21 Studio, which provide advanced text editing and AI platform services to developers, respectively. Their efforts underline the expansive scope and potential AI holds in revolutionising how humans interact with textual data.

Furthermore, Aleph Alpha and Beijing Academy of AI (BAAI) exemplify the global stride towards AI advancement. Aleph Alpha, with USD 30.4 million in funding, aims to cater to the European market with Luminous, an AI platform designed with explainability tools, aligning with the upcoming EU AI Act. On the other hand, BAAI, a state-funded initiative, is making strides in China's AI domain, with Wu Dao 2.0, a giant multimodal language model, heralding China's ambitious plans in AI.

These startups, with their unique approaches and substantial financial backing, reflect a broader trend in the AI industry where significant capital infusion is deemed necessary to drive innovation and remain competitive. The substantial funds raised underline the high costs associated with AI model training and operation, further emphasised by the estimated daily operational cost of ChatGPT. The venture capital flowing into these startups, coupled with their ambitious projects, showcase the burgeoning and capital-intensive nature of the modern AI landscape.

These hefty investments in LLMs are shaping the digital conversations of tomorrow. The costs associated with deploying and operating LLMs are substantial and will likely influence the price of digital services and products in the long run. As businesses strive to provide enriched digital interactions, understanding the financial backbone of these technologies is pivotal. The realm of casual digital chatter with AI is expanding, fueled by significant investments, making every digital interaction a testament to the monumental financial infrastructure working tirelessly behind the scenes.

The writer is Principal Data Scientist at Amazon in the field of Artificial Intelligence. Views expressed are personal