review

Empire of AI by Karen Hao

Peter Cripps review , , , , , ,

Karen Hao’s Empire of AI is without doubt a significant contribution to the ever growing collection of books digging into the workings of the AI industry. It is not just an insiders exposé of the shenanigans that went on leading up to the firing (and rapid rehiring) of Sam Altman, OpenAI’s founder and CEO, but also provides a deeply researched and incredibly well written insight into the current state of play of the AI industry as a whole, and, it’s not pretty.

The book opens with the events that took place over the long weekend of Friday 17th November to Tuesday 21st November 2023. On Friday 17th, Altman was invited to a video call set up by the OpenAI board members where he was unceremoniously told he was being fired. However by the following Tuesday, due to overwhelming pressure from OpenAI employees and, more importantly its investors, especially Microsoft, Altman was back at the helm with a new board of directors. Empire of AI examines how Altman and his fellow conspirators came to create and dominate the techno-industrial complex that is referred to generically as ‘AI’ and how, if things carry on as they currently are, we risk destroying jobs, the environment and most, if not all, forms of human endeavour and behaviour.

Empire of AI is divided into four parts. Part I covers how Sam (Altman) met Elon (Musk), the latter being a “childhood hero” of the former, and decided to build an AI company that would compete with Google head-on to beat them in the race to building AGI (artificial general intelligence). Musk was fearful that Google’s takeover of the British AI research company, DeepMind would lead them to develop AGI first and thereafter “murder all competing AI researchers“. Musk was adamant that “the future of AI should not be controlled by Larry (Page)“. To Musk, Altman was a like minded entrepreneur who wanted AGI to be for the good of all humanity, not something that would allow Google to become even richer by destroying all competitors in its wake. OpenAI was formed with good intentions therefore, to create the first AGI that could be “used for individual empowerment” and which would have safety as “a first-class-requirement“.

OpenAI was launched as a nonprofit company in December 2015. Altman’s co-founders were Greg Brockman (an engineer and fellow entrepreneur) and Ilya Sutskever (an AI researcher poached from Google). The company had a $1B commitment from, amongst others Elon Musk, Peter Thiel (Musk’s fellow cofounder of PayPal) and Reid Hoffman (another of the so called “PayPal mafia” and cofounder of LinkedIn). Having started a company with the ultimate goal of developing AGI, OpenAI needed to do three things quickly – figure out exactly what it was they were going to build to achieve AGI, hire the right talent to do this whilst at the same time securing enough funding to make these two things possible. By 2019 these problems seemed to have been solved. The company would focus on building its AI technology using a large language model (LLM) it called GPT-2. In order to secure the necessary funding to be able to pay for the enormous amounts of compute LLMs needed they switched from being a nonprofit to a for-profit company opening up the floodgates to companies like Microsoft investing in them in the hope of making huge profits if OpenAI were the first to achieve AGI. On this basis Microsoft announced in July 2019 it was to invest $1B in the company.

In Part II the book looks at some of the looming problems OpenAI and other companies began to face as they tried to scale their LLMs. From an ethics point of view many academics as well as people in the industry itself began to question the wisdom of building AI/AGI in an unregulated way. Comparisons were drawn with the development of the atom bomb during World War II and the work done by Oppenheimer and his team on the Manhattan Project. Where was the governance and regulation that was developed alongside nuclear weapons which, despite a few close shaves, have prevented nuclear Armageddon? Companies were building ethics teams to try and develop such governance models but there was often an uneasy relationship between the leadership teams whose focus was on profit and the need for an ethical approach for development. The need for ethical leadership is no more apparent when it comes to one of the activities few of us think about when using LLMs like ChatGPT and that is how these models ‘learn’. It turns out they are trained by people. But these are not the well paid engineers who live and work in San Francisco but are from third world countries like Kenya and Venezuela where labour practices are unregulated and often exploitative. As part of her research for the book Hao travels to several of these places and interviews some of the people who work long hours annotating data they are sent by the tech giants (usually via a third-party companies) describing what they see. This is not only boring and poorly paid (often just a few pennies per task) but in some cases can be hugely distressful as workers can be presented with text and images showing some of the worst forms of child sexual abuse, extreme violence, hate speech and self-harm. It’s very easy to forget, overlook or not understand that for LLMs like CharGPT to present acceptable content for our sensibilities someone, somewhere has had to filter out some of the worst forms of human degradation.

As LLMs are scaled there is a need to build more and more, ever larger data centres to house the processors that crunch the massive amounts of data, not just during training but also in their operation. Many of these large data centres are also being constructed in third-world countries where it is relatively easy to get planning permission and access to natural resources, like water for cooling, but often to the detriment of local people. In Part III, Hao discusses these aspects in detail. As new and improved versions of ChatGPT and it’s picture generation equivalent DALL-E were released and OpenAI became ever closer to Microsoft who were providing the necessary cloud infrastructure that hosted ChatGPT and DALL-E the need for ‘hyperscale’ data centres became ever greater The four largest hyperscalers – Google, Microsoft, Amazon and Meta – are now building so called ‘megacampuses’ with vast buildings containing racks of GPUs each of which will soon require 1,000 to 2,000 megawatts of power – the equivalent energy requirement of up to three and a half San Francisco’s. Such power hungry megacampuses mean that these companies can no longer meet their carbon emission targets (Google’s carbon emissions have soared by 51% since 2019 as they have invested in more and more artificial intelligence).

As Altman’s fame and influence grew his personal life inevitably began to get more attention. In September 2023 a feature writer at New York Magazine, Elizabeth Weil, published a profile of Altman which, for the first time in mainstream media, discussed his estrangement from his sister, Annie, and how financial, physical and mental health issues had caused her to turn to sex work. The New York magazine profile set side-by-side Annie’s life of financial problems with Altman’s lifestyle of expensive homes and luxury cars. Hao draws comparisons with how OpenAI (and other AI companies) ignore the anger of the data workers who try to challenge their domination by fighting for fair working conditions with how Altman seems able to do the same in ignoring his sisters cries for help. It would seem that Altman’s personal and professional lives were beginning to conspire against his so far meteoric success. In the final part of the book we see how a particular aspect of his personality lead to the events of that fateful weekend in November of 2023.

From the outside, much of what began to ensue at OpenAI after ChatGPT had propelled the company to a valuation in excess of $100B could be seen to be problems that face any company that had grown so quickly. As the spotlight on Altman himself had become ever more intense however Altman’s behaviour began to deteriorate. Often exhausted he was cracking under pressure of mounting competition as well as the punishing travel schedule he had set himself to promote OpenAI. According to Hao this pressure was causing Altman to exhibit destructive behaviour. “He was doing what he’d always done, agreeing with everyone to their face, and now, with increasing frequency, badmouthing them behind their backs. It was creating greater confusion and conflict across the company than ever before, with team leads mimicking his bad form and pitting their reports against each other“.

This, together with concerns about Altman forcing his developers to deliver new iterations of ChatGPT without sufficient testing finally drove the board, on Saturday 11th November 2023, to come to their momentous decision – “they would remove Altman and install Murati as interim CEO“. Mira Murati was OpenAIs CTO but in that role had found herself “frequently cleaning up his [Altman’s] messes“.

And so the book returns to where it started with the events of the 17th – 21st November. As we know, Altman survived what is now referred to internally as “The Blip” but pressure on him continues to mount from several directions – multiple lawsuits (including from Altman’s co-founder Elon Musk), investigations from regulators after the board investigation had observed Altman was “”not consistently candid in his communications” and increased competition, even from Microsoft who had decided to diversify its AI portfolio not wishing to put all of its AI eggs in OpenAI’s basket.

As followers of OpenAI will know, Altman and his team have gone on to deliver regular updates to ChatGPT as well as the API which can be used by developers to access its functionality. The current version (at the time of writing this review) of ChatGPT (o3-pro) is ‘multi-modal’ in that it can search the web, analyse files, reason about visual inputs, use Python, personalise responses using memory, and a whole load more. Its competitors too are releasing ever more powerful models though none (yet) claim to have achieved the holy grail of AGI. Empire of AI has captured a relatively small slice in time of the race to AGI and no doubt many more books will be written which chart the twists and turns of that race.

Empire of AI is a BIG book (nearly 500 pages with notes and index) and is the result of over 300 interviews plus a “trove of correspondence and documents” gathered by Karen Hao since she began covering OpenAI in 2019. Like many such books, you may wonder if the editing could have been a bit sharper. Perhaps reducing the number of stories and incidents would have made its points more succinctly (and in fewer pages). Ultimately however this is an important document that describes well the personalities involved in building OpenAI and the design and delivery of its products, not to mention the absolute and total belief the founders have in these products. Like the book Careless People by Sarah Wynn-Williams – which captures the power, greed and madness at Facebook during its early years – you do not come away from reading Empire of AI with much of a sense of trust or admiration for the men (for they are nearly all men) that start and run these companies. One can only hope that the steady drip of publications that are critiquing the tech industry in general and the AI companies in particular will ultimately lead to some form of change which limits and constrains the power of the people that run the companies as well as the technology itself.

I for one am not holding my breath though.

The Technological Republic According to Palantir

Peter Cripps comment, review , , , , , ,

The provocative assertion of The Technological Republic by Alex Karp and Nicholas Zamiska is that the money and time the software engineers of Silicon Valley expend on “social media platforms and food delivery apps” would be better directed at more worthwhile challenges. For the authors these would involve things like: addressing violent crime, education reform, medical research and national defence. It is no coincidence I’m sure that at least three of these is where Palantir Technologies Inc., the company co-founded by Karp, makes much of its money.

Karp and Zamiska believe that the founders and CEOs of the current crop of hugely successful, and fabulously rich, tech companies see these challenges as being “too intractable, too thorny, and too politically fraught” to address in any real way. Hence their focus is on consumer friendly apps rather than addressing some of the world’s truly wicked problems. The book is therefore a rallying cry and a wake-up call for the tech entrepreneurs of Silicon Valley to address these thorny problems if Western democracy, as we know it, is to survive and retain its technological hegemony.

It’s worth noting at this point that Palantir is a software company that specialises in advanced data analytics and artificial intelligence. It was founded in 2003 by Peter Thiel, Stephen Cohen, Joe Lonsdale as well as Alex Karp. Palantir’s customers include the United States Department of Defence the CIA, the DHS, the NSA, the FBI and the NHS here in the UK. It is obviously in Palantir’s best interest to ensure that governments continue to spend money on the kind of products they make. In April 2023, the company launched Artificial Intelligence Platform (AIP) which integrates large language models into privately operated networks. The company demonstrated its use in war, where a military operator could deploy operations and receive responses via an AI.

As Palantir moves into AI it is obvious they are going to need engineers who not only have the technical knowledge to build such systems but also don’t mind working on products used in the defence industry. As the pair state early on in the book, if such engineering talent is not forthcoming then “medical breakthroughs, education reform, and military advances would have to wait” because the required technical talent is being directed at building “video-sharing apps and social media platforms, advertising algorithms and online shopping websites“.

Whilst I do agree that an enormous amount of talent does feel as if it is being misdirected in efforts to wring every last dollar out of improving algorithms for selling “stuff” to consumers where I feel the author’s treatise becomes hopelessly sidetracked is on the reasons for this. Some of the arguments the authors put forward as to why we have arrived at this sorry state of affairs include:

  • The abandonment of belief or conviction in “broader political projects” such as the Manhattan Project to build the first atomic bomb or the Apollo space programme to put a man on the moon.
  • The failure of earlier government funded projects which created much of the technology we use and take for granted today (e.g. the internet, personal computing) to capitalise on this technology and direct its use to more worthwhile efforts. Quoting the authors again: “When emerging technologies that give rise to wealth do not advance the broader public interest, trouble often follows. Put differently, the decadence of a culture or civilization, and indeed its ruling class, will be forgiven only if that culture is capable of delivering economic growth and security for the public“.
  • Failure of universities to teach properly the ‘values’ of Western Civilisation and to properly articulate the “collective sense of identity that was capable of serving as a foundation for a broader sense of cohesion and shared purpose“.
  • Even Steve Jobs gets a reprimand for building technology (the Mac personal computer and iPhone) that was “intimate and personal” and that would “liberate the individual from reliance on a corporate or governmental superstructure“. Interestingly Elon Musk does earn some brownie points for founding Tesla and SpaceX that have “stepped forward to fill glaring innovation gaps where national governments have stepped back“. No mention is made that SpaceX has received nearly $20.7 billion in government contracts, research grants, and other forms of public assistance, with about $14.6 billion of that coming from contracts with NASA.
  • At one point in the book the authors relate a scene from George Orwell’s 1984 where Winston Smith is wandering through a wooded area and imagines that even here, Big Brother may be listening to every word through microphones concealed in trees. They use this to put forward the argument that we may be near such levels of surveillance ourselves but it is not the “contraptions built by Silicon Valley” to blame for this rather it’s “we” who are to blame for “failing to encourage and enable the radical act of belief in something beyond, and external to, the self“. An interesting observation from the co-founder of the company that develops some of the “contraptions’ that enable surveillance capitalism.

I could go on , but you get the general idea.

The first two parts of the book set about explaining why and how, in the author’s view, we have lost our way in tackling the big challenges of our age – preferring instead to do things like reimagining online shopping or building photo-sharing and food delivery apps.

Karp and Zamiska believe that the generation of engineers coming out of the prestigious universities of the West and East coasts of America in the late 1990s and early 2000s were not just able to benefit from the relatively new technology of the internet and world-wide web but were also able to take advantage of the seemingly unlimited funds being offered by the venture capitalists who had made their fortunes from “Web 1.0”. This happened to coincide with an increased lack of trust in national governments as well as frustration in delays in adopting more progressive reforms and their “grand experiments and military misadventures on a world stage“. It was easier for founders looking for something to “disrupt” to focus on solving their own problems such as how to hail a taxi or get a book delivered more quickly. They were not “building software systems for defence and intelligence agencies, and they were certainly not building bombs“.

Part III of the book concentrates on what the authors refer to as “The Engineering Mindset”. The proposition here is that the success of Silicon Valley is in how it has not just hired the best and the brightest engineers but has given them the “freedom and space to create“. Though in the authors view they are not creating the right things.

To illustrate this Karp and Zamiska reference a number of biological and psychological studies and experiments from the 1950s and 1960s. These experiments were done in what the authors refer to as the “golden age of psychology“, that is before such experiments were monitored for their sometimes cavalier approach to ethics. The results of these studies showed, alarmingly, how, often the majority of, people are prone to group think and a hive mindset. They tend to follow what others do or do what they are told for fear of standing out or looking foolish. The assertion is then made that this “instinct towards obedience” can be “lethal” when trying to create a truly disruptive or entrepreneurial organisation. Presumably Palantir go out of their way to hire disobedient disruptors when recruiting for their organisation.

One of the experiments cited is the now infamous, and much quoted one in books like this – the so called “obedience experiment” devised by the psychology professor Stanley Milgram. This is the one where a group of people were tested as to their willingness to inflict harm on innocent strangers by supposedly giving them electric shocks if they were not seen to memorise words accurately. The person who was meant to be doing the memorising was an actor who would yell and shout as the voltage was supposedly increased and implore their “teachers” to stop. The startling outcome of this experiment was that two-thirds of the people giving the electric shocks were ‘happy’ to carry on doing so even though they knew that may harm the learner. The findings from this experiment have been used to explain why, amongst other things, concentration camp guards during the Second World War were willing to carry out their atrocious acts under the instructions of their commanding officers.

Karp and Zamiska use this experiment to justify how such an instinct toward obedience can be anathema to creativity. In their view it is only those people who resist any tendency to conformity and group think who are likely to be the outliers who come up with truly novel ideas and approaches. For the authors the most effective software companies are more akin to artist colonies “filled with temperamental and talented souls“.

Having taken this detour around biology and psychology to show how conformance tends to be the trait that the majority have, the authors return to their main point that even if you can identify and hire the creative nonconformists the challenge is in how to direct that creativity “toward the nations shared goals“. These, they assert, can only be identified if “we take the risk of defining who we are or aspire to be“.

So what should “we be” and what should we “aspire to”? This is what the authors attempt to address in the final part of the book and is where they start to give away some of their own political, religious and business beliefs.

They reference Lee Kuan Yew (a second time) at this point. Lee was the first prime minister of Singapore and who was charged with convincing a sceptical public that the newly formed island nation (having split from Malaysia own 1965) could be a viable entity. Lee’s approach was to manufacture a national identity amongst its citizens by involving his government in several aspects of its citizens private lives. These included requiring that all Chinese students learn Mandarin (as well as English) at school instead of the multiple different dialects they learnt at home. The authors attribute this and other attempts at forging a national identity as being responsible for Singapore’s exponential growth from a GDP of $428 in 1960 to $84,734 in 2023. Rather confusingly, in terms of adhering to the authors other argument around the dangers of conformity, Singapore is renowned for its conformist citizens and its somewhat draconian legal system. So much so that the author William Gibson characterised Singapore as being Disneyland with the death penalty.

Karp and Zamiska then go on to state their belief that it is a failure of the “contemporary left” in the West that “deprives itself of the opportunity to talk about national identity” and that in both America and Europe the left has “neutered itself” and prevented its “advocates from having a forceful and forthright conversation about national identity“. At this point in the book the authors call on no other than J.R.R. Tolkien, he of The Lord of the Rings fame. For those not familiar with Tolkien’s three-volume tome it is the epic story of good versus evil where a group of plucky little Hobbits overcome the evil Sauron’s threats to bring death and destruction to Middle Earth. The reason this book is mentioned is because the authors see it as an example of how good storytelling around a shared narrative, even if its mythological (or religious) can show how people can come together. As Rowan Williams, the former Archbishop of Canterbury, says in an essay on Tolkien’s books, “Tolkien’s dogged concern about the terrible dangers of our desire for final solutions and unchallengeable security is even more necessary“. It’s worth noting at this point that Palantir was named after the “seeing stone” in Tolkien’s legendarium.

This then would seem to be the authors ultimate solution to the building of a technological republic. It’s no good relying on the fact that the software engineers of Silicon Valley will suddenly see the light and begin directing their talents at solving the world’s wicked problems and, more to the point today, the West’s security problems. Instead we need to build (or rebuild) a new order of “collective experience, of shared purpose and identity [and] of civic rituals that are capable of binding us together“.

The Technological Republic runs to just under 300 pages (or 218 if you take out the references and index). Unfortunately, like many books of this type, I cannot help but feel the whole argument could have been made more concisely and more persuasively if it was written as an opinion piece in The Atlantic or New Yorker magazine rather than as a long form book. The main point, that we in the West are directing our (software) technologies at building the wrong things is made over and over again with, to be frank, some fairly bizarre and academic references and too may non sequiturs. For example maybe those who did not conform to what was expected of them in the obedience experiment were not more creative but had a more pronounced ethical or moral compass. Talk of ethics is distinctly missing from this book.

I also believe there is a hidden agenda at play here to. In the UK at least Palantir is receiving some bad press, especially in its dealings with the NHS. Last year the British Medical Journal called on (the recently disbanded) NHS England to cancel its contract with Palantir citing concerns about the cost of its contract with the company and whether it offers value for money, as well as questions about public trust in Palantir, the procurement process as well as the companies public support of the Israeli Occupation Forces (IOF) with their assault in Gaza.

More widely there is a concern how smart city solutions currently rely on centralised, proprietary architectures that concentrate data and control in the hands of a few powerful tech companies like Cisco, IBM, Microsoft, and Palantir. Whereas once we may have thought it inconceivable that the data managed by an American corporation could be misused by a Western government what we are seeing happening now in America itself raises that very real fear.

Of even more concern is the apparent cosying up of Keir Starmer’s government with Palantir. According to Andrew Marr in this New Statesman article, following his meeting with Donald Trump in the Oval Office at the end of February, Starmer’s next visit was with Alex Karp at Palantir Technologies HQ in Washington. Whilst there he saw “military kit” and confirmed that Britain wouldn’t over-regulate AI so it could pursue new economic opportunities “with advanced technology at its core“. Unfortunately, it would appear much of the real economic advantage (and data) may be flowing to Trump’s America rather than staying in the UK!

To be clear, I do agree with the authors proposition, that much engineering talent is being wasted on the development of social media apps and the ever more ingenious ways that platform providers are finding to part us from our hard earned money. Where I diverge from the authors is in their rational for why this is the case. I suspect that it’s more to do with the fabulous salaries and ‘cool’ working environments that these companies offer rather than anything more sinister? As someone who has worked for both a Silicon Valley startup and on many government sites here in the UK I know where I would prefer to work given the choice (and setting aside ethical concerns).

Of course, working for Palantir you probably get to have your cake and eat it. I’m sure Palantir offers some nice working conditions for the quarter of its global workforce who operate from the UK whilst working on those profitable NHS contracts (£330m for a seven years according to Marr).

From a UK perspective the biggest issue of all is why we cannot build companies like Palantir that can be the data processing companies of choice by not just the NHS but for other government departments as well? I know at least part of the answer to this. We have a well-publicised skills gap in the UK where there are not enough good software engineers to staff such companies as well as a lack of investment capital to fund them. This has to be the real challenge for our government if we are to ever ween ourselves away from companies like Palantir and develop some home-grown talent who consider it worthwhile to work on ‘software for good’ projects rather than developing the next photo-sharing app (or developing the next great piece of surveillance software).