ethics

Empire of AI by Karen Hao

Peter Cripps review , , , , , ,

Karen Hao’s Empire of AI is without doubt a significant contribution to the ever growing collection of books digging into the workings of the AI industry. It is not just an insiders exposé of the shenanigans that went on leading up to the firing (and rapid rehiring) of Sam Altman, OpenAI’s founder and CEO, but also provides a deeply researched and incredibly well written insight into the current state of play of the AI industry as a whole, and, it’s not pretty.

The book opens with the events that took place over the long weekend of Friday 17th November to Tuesday 21st November 2023. On Friday 17th, Altman was invited to a video call set up by the OpenAI board members where he was unceremoniously told he was being fired. However by the following Tuesday, due to overwhelming pressure from OpenAI employees and, more importantly its investors, especially Microsoft, Altman was back at the helm with a new board of directors. Empire of AI examines how Altman and his fellow conspirators came to create and dominate the techno-industrial complex that is referred to generically as ‘AI’ and how, if things carry on as they currently are, we risk destroying jobs, the environment and most, if not all, forms of human endeavour and behaviour.

Empire of AI is divided into four parts. Part I covers how Sam (Altman) met Elon (Musk), the latter being a “childhood hero” of the former, and decided to build an AI company that would compete with Google head-on to beat them in the race to building AGI (artificial general intelligence). Musk was fearful that Google’s takeover of the British AI research company, DeepMind would lead them to develop AGI first and thereafter “murder all competing AI researchers“. Musk was adamant that “the future of AI should not be controlled by Larry (Page)“. To Musk, Altman was a like minded entrepreneur who wanted AGI to be for the good of all humanity, not something that would allow Google to become even richer by destroying all competitors in its wake. OpenAI was formed with good intentions therefore, to create the first AGI that could be “used for individual empowerment” and which would have safety as “a first-class-requirement“.

OpenAI was launched as a nonprofit company in December 2015. Altman’s co-founders were Greg Brockman (an engineer and fellow entrepreneur) and Ilya Sutskever (an AI researcher poached from Google). The company had a $1B commitment from, amongst others Elon Musk, Peter Thiel (Musk’s fellow cofounder of PayPal) and Reid Hoffman (another of the so called “PayPal mafia” and cofounder of LinkedIn). Having started a company with the ultimate goal of developing AGI, OpenAI needed to do three things quickly – figure out exactly what it was they were going to build to achieve AGI, hire the right talent to do this whilst at the same time securing enough funding to make these two things possible. By 2019 these problems seemed to have been solved. The company would focus on building its AI technology using a large language model (LLM) it called GPT-2. In order to secure the necessary funding to be able to pay for the enormous amounts of compute LLMs needed they switched from being a nonprofit to a for-profit company opening up the floodgates to companies like Microsoft investing in them in the hope of making huge profits if OpenAI were the first to achieve AGI. On this basis Microsoft announced in July 2019 it was to invest $1B in the company.

In Part II the book looks at some of the looming problems OpenAI and other companies began to face as they tried to scale their LLMs. From an ethics point of view many academics as well as people in the industry itself began to question the wisdom of building AI/AGI in an unregulated way. Comparisons were drawn with the development of the atom bomb during World War II and the work done by Oppenheimer and his team on the Manhattan Project. Where was the governance and regulation that was developed alongside nuclear weapons which, despite a few close shaves, have prevented nuclear Armageddon? Companies were building ethics teams to try and develop such governance models but there was often an uneasy relationship between the leadership teams whose focus was on profit and the need for an ethical approach for development. The need for ethical leadership is no more apparent when it comes to one of the activities few of us think about when using LLMs like ChatGPT and that is how these models ‘learn’. It turns out they are trained by people. But these are not the well paid engineers who live and work in San Francisco but are from third world countries like Kenya and Venezuela where labour practices are unregulated and often exploitative. As part of her research for the book Hao travels to several of these places and interviews some of the people who work long hours annotating data they are sent by the tech giants (usually via a third-party companies) describing what they see. This is not only boring and poorly paid (often just a few pennies per task) but in some cases can be hugely distressful as workers can be presented with text and images showing some of the worst forms of child sexual abuse, extreme violence, hate speech and self-harm. It’s very easy to forget, overlook or not understand that for LLMs like CharGPT to present acceptable content for our sensibilities someone, somewhere has had to filter out some of the worst forms of human degradation.

As LLMs are scaled there is a need to build more and more, ever larger data centres to house the processors that crunch the massive amounts of data, not just during training but also in their operation. Many of these large data centres are also being constructed in third-world countries where it is relatively easy to get planning permission and access to natural resources, like water for cooling, but often to the detriment of local people. In Part III, Hao discusses these aspects in detail. As new and improved versions of ChatGPT and it’s picture generation equivalent DALL-E were released and OpenAI became ever closer to Microsoft who were providing the necessary cloud infrastructure that hosted ChatGPT and DALL-E the need for ‘hyperscale’ data centres became ever greater The four largest hyperscalers – Google, Microsoft, Amazon and Meta – are now building so called ‘megacampuses’ with vast buildings containing racks of GPUs each of which will soon require 1,000 to 2,000 megawatts of power – the equivalent energy requirement of up to three and a half San Francisco’s. Such power hungry megacampuses mean that these companies can no longer meet their carbon emission targets (Google’s carbon emissions have soared by 51% since 2019 as they have invested in more and more artificial intelligence).

As Altman’s fame and influence grew his personal life inevitably began to get more attention. In September 2023 a feature writer at New York Magazine, Elizabeth Weil, published a profile of Altman which, for the first time in mainstream media, discussed his estrangement from his sister, Annie, and how financial, physical and mental health issues had caused her to turn to sex work. The New York magazine profile set side-by-side Annie’s life of financial problems with Altman’s lifestyle of expensive homes and luxury cars. Hao draws comparisons with how OpenAI (and other AI companies) ignore the anger of the data workers who try to challenge their domination by fighting for fair working conditions with how Altman seems able to do the same in ignoring his sisters cries for help. It would seem that Altman’s personal and professional lives were beginning to conspire against his so far meteoric success. In the final part of the book we see how a particular aspect of his personality lead to the events of that fateful weekend in November of 2023.

From the outside, much of what began to ensue at OpenAI after ChatGPT had propelled the company to a valuation in excess of $100B could be seen to be problems that face any company that had grown so quickly. As the spotlight on Altman himself had become ever more intense however Altman’s behaviour began to deteriorate. Often exhausted he was cracking under pressure of mounting competition as well as the punishing travel schedule he had set himself to promote OpenAI. According to Hao this pressure was causing Altman to exhibit destructive behaviour. “He was doing what he’d always done, agreeing with everyone to their face, and now, with increasing frequency, badmouthing them behind their backs. It was creating greater confusion and conflict across the company than ever before, with team leads mimicking his bad form and pitting their reports against each other“.

This, together with concerns about Altman forcing his developers to deliver new iterations of ChatGPT without sufficient testing finally drove the board, on Saturday 11th November 2023, to come to their momentous decision – “they would remove Altman and install Murati as interim CEO“. Mira Murati was OpenAIs CTO but in that role had found herself “frequently cleaning up his [Altman’s] messes“.

And so the book returns to where it started with the events of the 17th – 21st November. As we know, Altman survived what is now referred to internally as “The Blip” but pressure on him continues to mount from several directions – multiple lawsuits (including from Altman’s co-founder Elon Musk), investigations from regulators after the board investigation had observed Altman was “”not consistently candid in his communications” and increased competition, even from Microsoft who had decided to diversify its AI portfolio not wishing to put all of its AI eggs in OpenAI’s basket.

As followers of OpenAI will know, Altman and his team have gone on to deliver regular updates to ChatGPT as well as the API which can be used by developers to access its functionality. The current version (at the time of writing this review) of ChatGPT (o3-pro) is ‘multi-modal’ in that it can search the web, analyse files, reason about visual inputs, use Python, personalise responses using memory, and a whole load more. Its competitors too are releasing ever more powerful models though none (yet) claim to have achieved the holy grail of AGI. Empire of AI has captured a relatively small slice in time of the race to AGI and no doubt many more books will be written which chart the twists and turns of that race.

Empire of AI is a BIG book (nearly 500 pages with notes and index) and is the result of over 300 interviews plus a “trove of correspondence and documents” gathered by Karen Hao since she began covering OpenAI in 2019. Like many such books, you may wonder if the editing could have been a bit sharper. Perhaps reducing the number of stories and incidents would have made its points more succinctly (and in fewer pages). Ultimately however this is an important document that describes well the personalities involved in building OpenAI and the design and delivery of its products, not to mention the absolute and total belief the founders have in these products. Like the book Careless People by Sarah Wynn-Williams – which captures the power, greed and madness at Facebook during its early years – you do not come away from reading Empire of AI with much of a sense of trust or admiration for the men (for they are nearly all men) that start and run these companies. One can only hope that the steady drip of publications that are critiquing the tech industry in general and the AI companies in particular will ultimately lead to some form of change which limits and constrains the power of the people that run the companies as well as the technology itself.

I for one am not holding my breath though.

The Technological Republic According to Palantir

Peter Cripps comment, review , , , , , ,

The provocative assertion of The Technological Republic by Alex Karp and Nicholas Zamiska is that the money and time the software engineers of Silicon Valley expend on “social media platforms and food delivery apps” would be better directed at more worthwhile challenges. For the authors these would involve things like: addressing violent crime, education reform, medical research and national defence. It is no coincidence I’m sure that at least three of these is where Palantir Technologies Inc., the company co-founded by Karp, makes much of its money.

Karp and Zamiska believe that the founders and CEOs of the current crop of hugely successful, and fabulously rich, tech companies see these challenges as being “too intractable, too thorny, and too politically fraught” to address in any real way. Hence their focus is on consumer friendly apps rather than addressing some of the world’s truly wicked problems. The book is therefore a rallying cry and a wake-up call for the tech entrepreneurs of Silicon Valley to address these thorny problems if Western democracy, as we know it, is to survive and retain its technological hegemony.

It’s worth noting at this point that Palantir is a software company that specialises in advanced data analytics and artificial intelligence. It was founded in 2003 by Peter Thiel, Stephen Cohen, Joe Lonsdale as well as Alex Karp. Palantir’s customers include the United States Department of Defence the CIA, the DHS, the NSA, the FBI and the NHS here in the UK. It is obviously in Palantir’s best interest to ensure that governments continue to spend money on the kind of products they make. In April 2023, the company launched Artificial Intelligence Platform (AIP) which integrates large language models into privately operated networks. The company demonstrated its use in war, where a military operator could deploy operations and receive responses via an AI.

As Palantir moves into AI it is obvious they are going to need engineers who not only have the technical knowledge to build such systems but also don’t mind working on products used in the defence industry. As the pair state early on in the book, if such engineering talent is not forthcoming then “medical breakthroughs, education reform, and military advances would have to wait” because the required technical talent is being directed at building “video-sharing apps and social media platforms, advertising algorithms and online shopping websites“.

Whilst I do agree that an enormous amount of talent does feel as if it is being misdirected in efforts to wring every last dollar out of improving algorithms for selling “stuff” to consumers where I feel the author’s treatise becomes hopelessly sidetracked is on the reasons for this. Some of the arguments the authors put forward as to why we have arrived at this sorry state of affairs include:

  • The abandonment of belief or conviction in “broader political projects” such as the Manhattan Project to build the first atomic bomb or the Apollo space programme to put a man on the moon.
  • The failure of earlier government funded projects which created much of the technology we use and take for granted today (e.g. the internet, personal computing) to capitalise on this technology and direct its use to more worthwhile efforts. Quoting the authors again: “When emerging technologies that give rise to wealth do not advance the broader public interest, trouble often follows. Put differently, the decadence of a culture or civilization, and indeed its ruling class, will be forgiven only if that culture is capable of delivering economic growth and security for the public“.
  • Failure of universities to teach properly the ‘values’ of Western Civilisation and to properly articulate the “collective sense of identity that was capable of serving as a foundation for a broader sense of cohesion and shared purpose“.
  • Even Steve Jobs gets a reprimand for building technology (the Mac personal computer and iPhone) that was “intimate and personal” and that would “liberate the individual from reliance on a corporate or governmental superstructure“. Interestingly Elon Musk does earn some brownie points for founding Tesla and SpaceX that have “stepped forward to fill glaring innovation gaps where national governments have stepped back“. No mention is made that SpaceX has received nearly $20.7 billion in government contracts, research grants, and other forms of public assistance, with about $14.6 billion of that coming from contracts with NASA.
  • At one point in the book the authors relate a scene from George Orwell’s 1984 where Winston Smith is wandering through a wooded area and imagines that even here, Big Brother may be listening to every word through microphones concealed in trees. They use this to put forward the argument that we may be near such levels of surveillance ourselves but it is not the “contraptions built by Silicon Valley” to blame for this rather it’s “we” who are to blame for “failing to encourage and enable the radical act of belief in something beyond, and external to, the self“. An interesting observation from the co-founder of the company that develops some of the “contraptions’ that enable surveillance capitalism.

I could go on , but you get the general idea.

The first two parts of the book set about explaining why and how, in the author’s view, we have lost our way in tackling the big challenges of our age – preferring instead to do things like reimagining online shopping or building photo-sharing and food delivery apps.

Karp and Zamiska believe that the generation of engineers coming out of the prestigious universities of the West and East coasts of America in the late 1990s and early 2000s were not just able to benefit from the relatively new technology of the internet and world-wide web but were also able to take advantage of the seemingly unlimited funds being offered by the venture capitalists who had made their fortunes from “Web 1.0”. This happened to coincide with an increased lack of trust in national governments as well as frustration in delays in adopting more progressive reforms and their “grand experiments and military misadventures on a world stage“. It was easier for founders looking for something to “disrupt” to focus on solving their own problems such as how to hail a taxi or get a book delivered more quickly. They were not “building software systems for defence and intelligence agencies, and they were certainly not building bombs“.

Part III of the book concentrates on what the authors refer to as “The Engineering Mindset”. The proposition here is that the success of Silicon Valley is in how it has not just hired the best and the brightest engineers but has given them the “freedom and space to create“. Though in the authors view they are not creating the right things.

To illustrate this Karp and Zamiska reference a number of biological and psychological studies and experiments from the 1950s and 1960s. These experiments were done in what the authors refer to as the “golden age of psychology“, that is before such experiments were monitored for their sometimes cavalier approach to ethics. The results of these studies showed, alarmingly, how, often the majority of, people are prone to group think and a hive mindset. They tend to follow what others do or do what they are told for fear of standing out or looking foolish. The assertion is then made that this “instinct towards obedience” can be “lethal” when trying to create a truly disruptive or entrepreneurial organisation. Presumably Palantir go out of their way to hire disobedient disruptors when recruiting for their organisation.

One of the experiments cited is the now infamous, and much quoted one in books like this – the so called “obedience experiment” devised by the psychology professor Stanley Milgram. This is the one where a group of people were tested as to their willingness to inflict harm on innocent strangers by supposedly giving them electric shocks if they were not seen to memorise words accurately. The person who was meant to be doing the memorising was an actor who would yell and shout as the voltage was supposedly increased and implore their “teachers” to stop. The startling outcome of this experiment was that two-thirds of the people giving the electric shocks were ‘happy’ to carry on doing so even though they knew that may harm the learner. The findings from this experiment have been used to explain why, amongst other things, concentration camp guards during the Second World War were willing to carry out their atrocious acts under the instructions of their commanding officers.

Karp and Zamiska use this experiment to justify how such an instinct toward obedience can be anathema to creativity. In their view it is only those people who resist any tendency to conformity and group think who are likely to be the outliers who come up with truly novel ideas and approaches. For the authors the most effective software companies are more akin to artist colonies “filled with temperamental and talented souls“.

Having taken this detour around biology and psychology to show how conformance tends to be the trait that the majority have, the authors return to their main point that even if you can identify and hire the creative nonconformists the challenge is in how to direct that creativity “toward the nations shared goals“. These, they assert, can only be identified if “we take the risk of defining who we are or aspire to be“.

So what should “we be” and what should we “aspire to”? This is what the authors attempt to address in the final part of the book and is where they start to give away some of their own political, religious and business beliefs.

They reference Lee Kuan Yew (a second time) at this point. Lee was the first prime minister of Singapore and who was charged with convincing a sceptical public that the newly formed island nation (having split from Malaysia own 1965) could be a viable entity. Lee’s approach was to manufacture a national identity amongst its citizens by involving his government in several aspects of its citizens private lives. These included requiring that all Chinese students learn Mandarin (as well as English) at school instead of the multiple different dialects they learnt at home. The authors attribute this and other attempts at forging a national identity as being responsible for Singapore’s exponential growth from a GDP of $428 in 1960 to $84,734 in 2023. Rather confusingly, in terms of adhering to the authors other argument around the dangers of conformity, Singapore is renowned for its conformist citizens and its somewhat draconian legal system. So much so that the author William Gibson characterised Singapore as being Disneyland with the death penalty.

Karp and Zamiska then go on to state their belief that it is a failure of the “contemporary left” in the West that “deprives itself of the opportunity to talk about national identity” and that in both America and Europe the left has “neutered itself” and prevented its “advocates from having a forceful and forthright conversation about national identity“. At this point in the book the authors call on no other than J.R.R. Tolkien, he of The Lord of the Rings fame. For those not familiar with Tolkien’s three-volume tome it is the epic story of good versus evil where a group of plucky little Hobbits overcome the evil Sauron’s threats to bring death and destruction to Middle Earth. The reason this book is mentioned is because the authors see it as an example of how good storytelling around a shared narrative, even if its mythological (or religious) can show how people can come together. As Rowan Williams, the former Archbishop of Canterbury, says in an essay on Tolkien’s books, “Tolkien’s dogged concern about the terrible dangers of our desire for final solutions and unchallengeable security is even more necessary“. It’s worth noting at this point that Palantir was named after the “seeing stone” in Tolkien’s legendarium.

This then would seem to be the authors ultimate solution to the building of a technological republic. It’s no good relying on the fact that the software engineers of Silicon Valley will suddenly see the light and begin directing their talents at solving the world’s wicked problems and, more to the point today, the West’s security problems. Instead we need to build (or rebuild) a new order of “collective experience, of shared purpose and identity [and] of civic rituals that are capable of binding us together“.

The Technological Republic runs to just under 300 pages (or 218 if you take out the references and index). Unfortunately, like many books of this type, I cannot help but feel the whole argument could have been made more concisely and more persuasively if it was written as an opinion piece in The Atlantic or New Yorker magazine rather than as a long form book. The main point, that we in the West are directing our (software) technologies at building the wrong things is made over and over again with, to be frank, some fairly bizarre and academic references and too may non sequiturs. For example maybe those who did not conform to what was expected of them in the obedience experiment were not more creative but had a more pronounced ethical or moral compass. Talk of ethics is distinctly missing from this book.

I also believe there is a hidden agenda at play here to. In the UK at least Palantir is receiving some bad press, especially in its dealings with the NHS. Last year the British Medical Journal called on (the recently disbanded) NHS England to cancel its contract with Palantir citing concerns about the cost of its contract with the company and whether it offers value for money, as well as questions about public trust in Palantir, the procurement process as well as the companies public support of the Israeli Occupation Forces (IOF) with their assault in Gaza.

More widely there is a concern how smart city solutions currently rely on centralised, proprietary architectures that concentrate data and control in the hands of a few powerful tech companies like Cisco, IBM, Microsoft, and Palantir. Whereas once we may have thought it inconceivable that the data managed by an American corporation could be misused by a Western government what we are seeing happening now in America itself raises that very real fear.

Of even more concern is the apparent cosying up of Keir Starmer’s government with Palantir. According to Andrew Marr in this New Statesman article, following his meeting with Donald Trump in the Oval Office at the end of February, Starmer’s next visit was with Alex Karp at Palantir Technologies HQ in Washington. Whilst there he saw “military kit” and confirmed that Britain wouldn’t over-regulate AI so it could pursue new economic opportunities “with advanced technology at its core“. Unfortunately, it would appear much of the real economic advantage (and data) may be flowing to Trump’s America rather than staying in the UK!

To be clear, I do agree with the authors proposition, that much engineering talent is being wasted on the development of social media apps and the ever more ingenious ways that platform providers are finding to part us from our hard earned money. Where I diverge from the authors is in their rational for why this is the case. I suspect that it’s more to do with the fabulous salaries and ‘cool’ working environments that these companies offer rather than anything more sinister? As someone who has worked for both a Silicon Valley startup and on many government sites here in the UK I know where I would prefer to work given the choice (and setting aside ethical concerns).

Of course, working for Palantir you probably get to have your cake and eat it. I’m sure Palantir offers some nice working conditions for the quarter of its global workforce who operate from the UK whilst working on those profitable NHS contracts (£330m for a seven years according to Marr).

From a UK perspective the biggest issue of all is why we cannot build companies like Palantir that can be the data processing companies of choice by not just the NHS but for other government departments as well? I know at least part of the answer to this. We have a well-publicised skills gap in the UK where there are not enough good software engineers to staff such companies as well as a lack of investment capital to fund them. This has to be the real challenge for our government if we are to ever ween ourselves away from companies like Palantir and develop some home-grown talent who consider it worthwhile to work on ‘software for good’ projects rather than developing the next photo-sharing app (or developing the next great piece of surveillance software).

Is an AI ‘Parky’ the first step in big techs takeover of the entertainment industry?

Peter Cripps comment , , , , ,
Composite Image Created using OpenAI, DALL-E and Adobe Photoshop

Sir Michael Parkinson, who died in 2023 [1], was a much loved UK chat show host who worked at the BBC between 1971 and 1982, again between 1998 and 2004 and finally for a further three years at ITV until 2007. During that time “Parky” interviewed the great and the good (and sometimes the not so good [2]) from film, television, music, sport, science and industry. I remember Saturday nights during his first stint at the BBC not feeling complete unless we had tuned into Parkinson to see which celebrities he was interviewing that night. I was sad to hear of his passing last year but also grateful I had lived at the time to see many of his interviews and appreciate his gentle but probing interview style.

Last week however we learnt that just because you are dead, it does not mean you cannot carry on doing your job. Mike Parkinson, son of Sir Michael, has given permission to Deep Fusion Films to create an exact replica of his late father’s voice so he can virtually host a new eight-part, “unscripted series” [3] called Virtually Parkinson. The virtual Parky will be able to interview new guests based on analyses of data obtained from the real Parkinson’s back catalogue [4].

Deep Fusion Films was founded in 2023 and makes a big play about its ethical credentials. On its website [5] it says it aims to “establish comprehensive policies that promote the legal and ethical integration of AI in production”. Backing this up, their virtual Parky will be created with the full support and involvement of Sir Michael’s family and estate. 

So far, so ethical, right and proper, however…

Only last year, concerns over the use (and potential misuse) of AI in the film industry led to a strike by actors and writers. Succession actor Brian Cox made the statement that using AI to replicate an actor’s image and use it forever is “identity theft” and should be considered “a human rights issue” [6].

Hollywood stars like Scarlett Johansson, Tom Hanks, Tom Cruise and Keanu Reeves, have already become the subject of unauthorised deepfakes and in June of this year the Internet Watch Foundation(IWF) warned that AI-generated videos of child sexual abuse could indicate a ‘stark vision of the future’ [7].

Clearly, where Deep Fusion Films are right now, i.e. producing ethically sourced and approved imitations of celebrities voices, and where AI generated porn is threatening to take us are poles apart but…

Technology always creeps into our lives like this. A small seemingly insignificant event which we find amusing and mildly distracting entertains us for a while but then suddenly, it has become the way of all things and has fallen into the hands of ‘bad actors’. At this point, there is often no going back.

Witness how Facebook started out as an innocuous site called Facemash, created by a second-year student at Harvard University called Mark Zuckerberg, that compared two student photos side-by-side to determine who was “hot” and who was “not.” Actually this was always a questionable use case in my opinion, but I guess an indication of what went down as acceptable behaviour in Ivy League universities of the early 2000s!

Today Meta (who now owns Facebook) is the seventh largest company in the world by market capitalisation worth, at the time of writing, $1.497 T [8]. Zuckerberg’s vision for Meta, outlined in a letter to shareholders this August, is that it will become a virtual reality platform that merges the physical and digital worlds forever transforming how we interact, work, and socialise [9]. Inevitably a major part of this vision is that artificial intelligence (or even, if Zuckerberg gets his way, artificial general intelligence) will be there to “enhance user experiences”.

Facebook, and now Meta, is surely the canonical example of how a small and seemingly insignificant company from the US east coast has grown in a mere 20 years to become a largely unregulated west coast tech behemoth with over three billion active monthly users [10].

If Facebook was just used for sharing pictures of cats and dogs that would be one thing but, during its short history, it has been found guilty of spreading fake news, changing voting behaviour in key elections around the world, affecting peoples mental health as well as spreading violent and misogynistic (and deepfake) videos.

It seems like we never learn. Governments and legal systems around the world never react fast enough to the pace of technological change and are always playing catchup having to mop up the tech companies misdemeanours after they have occurred rather than regulating against tech companies in the first place. Financial penalties are one thing but these pale into insignificance alongside the gargantuan profits such companies make and anyway, no amount of fines can undo the negative effects they and their leaders have on peoples lives.

So how does the rise of the tech behemoths like Facebook, Google and X presage what might happen in the creative industries and their use of technology, especially AI?

I don’t know what proportion of a Hollywood movies costs goes to actors salaries. It is obviously not the only cost or even the largest cost however with actors like Tom Cruise, Keanu Reeves and Will Smith able to command salaries for a single film in excess of $100M [11] salaries are clearly not insignificant. It must be very tempting for movie producers to be thinking why not invest a bit more in special effects and just create a whole new actor from scratch. After all, that’s precisely what Walt Disney did with Mickey Mouse who never got paid a dime.

How long is it before we cross a red line and a movies special effects goes the whole way and uses CGI to create the characters in a completely AI scripted and generated film? Huge upfront costs (for now, but these will drop) but no ongoing costs of having to pay actors for re-runs or streaming rights etc.

I don’t know how long it might take or whether we will ever get there. Maybe the technology will never be good enough (unlikely) or maybe we will wake up to what we are doing and create some sort of legal/ethical framework that prevents such things occurring (equally unlikely I fear).

We are beginning to rub up against some pretty fundamental questions not just about how we should be using AI, especially in the creative industries, but also what it actually means to be human if we let our machines overwhelm us to the extent that our creative selves are usurped by the very things that creativity has built.

This is a hugely important question which I hope to explore in future posts. 

Notes

  1. Sir Michael Parkinson obituary,https://www.theguardian.com/media/2023/aug/17/sir-michael-parkinson-obituary
  2. Michael Parkinson speaks out on Savile scandal, https://www.itv.com/news/calendar/2012-12-01/michael-parkinson-speaks-out-on-savile-scandal
  3. AI-replicated Michael Parkinson to host ‘completely unscripted’ celebrity podcast, https://news.sky.com/story/ai-replicated-michael-parkinson-to-host-completely-unscripted-celebrity-podcast-13243556
  4. Michael Parkinson is back, with an AI voice that can fool even his own family, https://www.theguardian.com/media/2024/oct/26/michael-parkinson-virtually-ai-replica-chatshow
  5. Deep Fusion Films is a dynamic production company at the forefront of television and film,https://www.deepfusionfilms.com/about
  6. Succession star Brian Cox on the use of AI to replicate actors: ‘It’s a human rights issue’,https://news.sky.com/story/succession-star-brian-cox-on-the-use-of-ai-to-replicate-actors-its-a-human-rights-issue-12999168
  7. AI-generated videos of child sexual abuse a ‘stark vision of the future’, https://www.iwf.org.uk/news-media/news/ai-generated-videos-of-child-sexual-abuse-a-stark-vision-of-the-future/
  8. Largest Companies by Marketcap,https://companiesmarketcap.com
  9. Mark Zuckerberg’s Letter: Meta’s Vision Unveiled,https://medium.com/@ahmedofficial588/mark-zuckerbergs-letter-meta-s-vision-unveiled-2b48a57a2743
  10. Facebook User & Growth Statistics,https://backlinko.com/facebook-users
  11. 20 Highest Paid Actors For a Single Film,https://thecinemaholic.com/highest-paid-actors-for-a-single-film/

Machines like us? – Part I

Peter Cripps comment , , , , , , ,
From The Secret of the Machines, Artist unknown

Our ambitions run high and low – for a creation myth made real, for a monstrous act of self love. As soon as it was feasible, we had no choice, but to follow our desires and hang the consequences.

Ian McEwan, Machines Like Me

I know what you’re thinking – not yet another post on ChatGPT! Haven’t enough words been written (or machine-generated) on this topic in the last few months to make the addition of any more completely unnecessary? What else is there to possibly say?

Well, we’ll see.

First, just in case you have been living in a cave in North Korea for the last year, what is ChatGPT? Let’s ask it…

ChatGPT is an AI language model developed by OpenAI. It is based on the GPT (Generative Pre-trained Transformer) architecture, specifically GPT-3.5. GPT-3.5 is a deep learning model that has been trained on a diverse range of internet text to generate human-like responses to text prompts.

ChatGPT response to the question: “What is ChatGPT”.

In this post, I am not interested in what use cases ChatGPT is or is not good for. I’m not even particularly interested in what jobs ChatGPT is going to replace in the coming years. Let’s face it, if the CEO of IBM, Arvind Krishna, is saying “I could easily see 30 per cent of [non-customer-facing roles] getting replaced by AI and automation over a five-year period” then many people are already going to be worried so I’m not going to add to those fears.

I see much of what Krishna predicts as inevitable. Unless the world takes note of the recent letter from the tech/AI ‘great and the good’ (which appears to have some fake signatories anyway) then the simple fact is that if you can algorithmically capture a solution to a problem, people who solve those problems for a living are probably going to get replaced by such algorithms.

What interests me is whether deep learning models like ChatGPT really are intelligent and, more significantly, are they creative. In this post I’ll consider intelligence and in a later post creativity.

It turns out that intelligence is actually quite a hard thing to define. As Max Tegmark, president of the Future of Life Institute, says: “there’s no agreement on what intelligence is even amongst intelligent intelligence researchers!” Helpfully however, in his book Life 3.0, Tegmark puts us out of our misery and defines intelligence as the: “ability to accomplish complex goals”. This, as even Tegmark acknowledges, is a very broad definition. He also claims this definition subsumes the ability to “apply knowledge and skills” as this could be a goal one sets oneself.

Having defined intelligence it then becomes relatively easy, for Tegmark at least, to define artificial intelligence (AI) as: “non biological intelligence”; general intelligence as the: “ability to accomplish virtually any goal, including learning”; artificial general intelligence (AGI) as the: “ability to accomplish any cognitive task at least as well as humans” and superintelligence as: “general intelligence far beyond human level”.

So, given the relatively low bar that these definitions are set at, where would we say ChatGPT and its ilk are currently at on the intelligence scale?

I guess this really hinges on whether you think the multitude of tasks that these deep learning models can perform constitutes “accomplishing virtually any goal” or not? For many people (especially as we’ll see, AI researchers) the answer is probably yes.

But let’s ask ChatGPT if it thinks it’s an AGI?

It is important to note that as an AI language model, I am an instantiation of artificial general intelligence (AGI) and can provide insights based on my training up until September 2021.

Partial ChatGPT response to the question: “Given Max Tegmark’s definition of intelligence where would you say you are on the various intelligence levels he proposes?”.

Personally, and up until a few weeks ago, I would have said ChatGPT was getting a bit above itself to say it was an “instantiation” of an AGI but then I read an interview with Jaron Lanier titled How humanity can defeat AI.

Lanier works for Microsoft and is the author of a number of what you might call anti-social media books including You Are Not A Gadget and Ten Arguments For Deleting Your Social Media Accounts Right Now.

Lanier’s argument in this interview is that we have got AI wrong and we should not be treating it as a new form of intelligence at all. Indeed he has previously stated there is no AI. Instead Lanier reckons we have built a new and “innovative form of social collaboration”. Like the other social collaboration platforms that Lanier has argued we should all leave because they have gone horribly wrong this new form too could become perilous in nature if we don’t design it well. In Lanier’s view therefore the sooner we understand there is no such thing as AI, the sooner we’ll start managing our new technology intelligently and learn how to use it as a collaboration tool.

Whilst all of the above is well intentioned the real insightful moment for me came when Lanier was discussing Alan Turing’s famous test for intelligence. Let me quote directly what Lanier says.

You’ve probably heard of the Turing test, which was one of the original thought-experiments about artificial intelligence. There’s this idea that if a human judge can’t distinguish whether something came from a person or computer, then we should treat the computer as having equal rights. And the problem with that is that it’s also possible that the judge became stupid. There’s no guarantee that it wasn’t the judge who changed rather than the computer. The problem with treating the output of GPT as if it’s an alien intelligence, which many people enjoy doing, is that you can’t tell whether the humans are letting go of their own standards and becoming stupid to make the machine seem smart.

Jaron Lanier, How humanity can defeat AI, UnHerd, May 8th 2023

There is no doubt that we are in great danger of believing whatever bullshit GPT’s generate. The past decade or so of social media growth has illustrated just how difficult we humans find it to handle misinformation and these new and wondrous machines are only going to make that task even harder. This, coupled with the problem that our education system seems to reward the regurgitation of facts rather than developing critical thinking skills is, as journalist Kenan Malik says, increasingly going to become more of an issue as we try to figure out what is fake and what is true.

Interestingly, around the time Lanier was saying “there is no AI”, the so called “godfather of AI”, Geoffrey Hinton was announcing he was leaving Google because he was worried that AI could become “more intelligent than humans and could be exploited by ‘bad actors'”. Clearly, as someone who created the early neural networks that were the predecessors to the large language models GPTs are built on Hinton could not be described as being “stupid”, so what is going on here? Like others before him who think AI might be exhibiting signs of becoming sentient, maybe Hinton is being deceived by the very monster he has helped create.

So what to do?

Helpfully Max Tegmark, somewhat tongue-in-cheek, has suggested the following rules for developing AI (my comments are in italics):

  • Don’t teach it to code: this facilitates recursive self-improvement – ChatGPT can already code.
  • Don’t connect it to the internet: let it learn only the minimum needed to help us, not how to manipulate us or gain power – ChatGPT certainly connected to the internet to learn what it already knows.
  • Don’t give it a public API: prevent nefarious actors from using it within their code – OpenAI is releasing a public API.
  • Don’t start an arms race: this incentivizes everyone to prioritize development speed over safety – I think it’s safe to say there is already an AI arms race between the US and China.

Oh dear, it’s not going well is it?

So what should we really do?

I think Lanier is right. Like many technologies that have gone before, AI is seducing us into believing it is something it is not – even, it seems, to its creators. Intelligent it may well be, at least by Max Tegmark’s very broad definition of what intelligence is, but let’s not get beyond ourselves. Whilst I agree (and definitely fear) AI could be exploited by bad actors it is still, at a fundamental level, little more than a gargantuan mash up machine that is regurgitating the work of the people who have written the text and created the images it spits out. These mash ups may be fooling many of us some of the time (myself included) but we must be not be fooled into losing our critical thought processes here.

As Ian McEwan points out, we must be careful we don’t “follow our desires and hang the consequences”.

Should we worry about those dancing robots?

Peter Cripps comment , , ,
Image Copyright Boston Dynamics

The robots in question are the ones built by Boston Dynamics who shared this video over the holiday period.

For those who have not been watching the development of this companies robots, we get to see the current ‘stars’ of the BD stable, namely: ‘Atlas’ (the humanoid robot), ‘Spot’ (the ‘dog’, who else?) and ‘Handle’ (the one on wheels) all coming together for a nice little Christmassy dance.

(As an aside, if you didn’t quite get what you wanted from Santa this year, you’ll be happy to know you can have your very own ‘Spot’ for a cool $74,500.00 from the Boston Dynamics online shop).

Boston Dynamics is an American engineering and robotics design company founded in 1992 as a spin-off from the Massachusetts Institute of Technology. Boston Dynamics is currently owned by the Hyundai Motor Group (since December, 2020) having previously been owned by Google X and SoftBank Group, the Japanese multinational conglomerate holding company.

Before I get to the point of this post, and attempt to answer the question posed by it, it’s worth knowing that five years ago the US Marine Corps, working with Boston Dynamics who were under contract with DARPA, decided to abandon a project to build a “robotic mule” that would carry heavy equipment for the Marines because the Legged Squad Support System (LS3) was too noisy. I mention this for two reasons: 1) that was five years ago, a long time in robotics/AI/software development terms and 2) that was a development we were actually told about, what about all those other military projects that are classified that BD may very well be participating in? More of this later.

So back to the central question: should we worry about those dancing robots? My answer is a very emphatic ‘yes’, for three reasons.

Reason Number One: It’s a “visual lie”

The first reason is nicely summed up by James J. Ward, a privacy lawyer, in this article. Ward’s point, which I agree with, is that this is an attempt to convince people that BD’s products are harmless and pose no threat because robots are fun and entertaining. Anyone who’s been watching too much Black Mirror should just chill a little and stop worrying. As Ward says:

“The real issue is that what you’re seeing is a visual lie. The robots are not dancing, even though it looks like they are. And that’s a big problem”.

Ward goes on to explain that when we watch this video and we see these robots appearing to be experiencing the music, the rhythmic motion, the human-like gestures we naturally start to feel the joyfulness and exuberance of the dance with them. The robots become anthropomorphised and we start to feel we should love them because they can dance, just like us. This however, is dangerous. These robots are not experiencing the music or the interaction with their ‘partners’ in any meaningful way, they have simply been programmed to move in time to a rhythm. As Ward says:

“It looks like human dancing, except it’s an utterly meaningless act, stripped of any social, cultural, historical, or religious context, and carried out as a humblebrag show of technological might.”

The more content like this that we see, the more familiar and normal it seems and the more blurred the line becomes between what it is to be human and what our relationship should be with technology. In other words, we will become as accepting of robots as we are now with our mobile phones and our cars and they will suddenly be integral parts of our life just like those relatively more benign objects are.

But robots are different.

Although we’re probably still some way off from the dystopian amusement park for rich vacationers depicted in the film Westworld, where customers can live out their fantasies through the use of robots that provide anything humans want we should not ignore the threat from robots and advanced artificial intelligence (AI) too quickly. Maybe then, videos like the BD one should serve as a reminder that now is the time to start thinking about what sort of relationship we want with this new breed of machine and start developing ethical frameworks on how we create and treat things that will look increasingly like us?

Reason Number Two: The robots divert us from the real issue

If the BD video runs the risk of making us more accepting of technology because it fools us into believing those robots are just like us, it also distracts us in a more pernicious way. Read any article or story on the threats of AI and you’ll aways see it appearing alongside a picture of a robot, and usually one that Terminator like is rampaging around shooting everything and everyone in sight. The BD video however shows that robots are fun and that they’re here to do work for us and entertain us, so let’s not worry about them or, by implication, their ‘intelligence’.

As Max Tegmark points out in his book Life 3.0 however, one of the great myths of the dangers of artificial intelligence is not that robots will rise against us and wage out of control warfare Terminator style, it’s more to do with the nature of artificial intelligence itself. Namely, that an AI whose goals are misaligned with our own, needs no body, just an internet connection, to wreak its particular form of havoc on our economy or our very existence. How so?

It’s all to do with the nature of, and how we define, intelligence. It turns out intelligence is actually quite a hard thing to define (and more so to get everyone to agree on a definition). Tegmark uses a relatively broad definition:

intelligence = ability to accomplish complex goals

and it then follows that:

artificial intelligence = non-biological intelligence

Given these definitions then, the real worry is not about machines becoming malevolent but about machines becoming very competent. In other words what about if you give a machine a goal to accomplish and it decides to achieve that goal no matter what the consequences?

This was the issue so beautifully highlighted by Stanley Kubrick and Arthur C. Clarke in the film 2001: A Space Odyssey. In that film the onboard computer (HAL) on a spaceship bound for Jupiter ends up killing all of the crew but one when it fears its goal (to reach Jupiter) maybe jeopardised. HAL had no human-like manifestation (no arms or legs), it was ‘just’ a computer responsible for every aspect of controlling the spaceship and eminently able to use that power to kill several of the crew members. As far as HAL was concerned it was just achieving its goal – even if it did mean dispensing with the crew!

It seems that hardly a day goes by without there being news of not just our existing machines becoming ever more computerised but with those computers becoming ever more intelligent. For goodness sake, even our toothbrushes are now imbued with AI! The ethical question here then is how much AI is enough and just because you can build intelligence into a machine or device, does that mean you actually should?

Reason Number Three: We maybe becoming “techno-chauvinists”

One of the things I always think when I see videos like the BD one is, if that’s what these companies are showing is commercially available, how far advanced are the machines they are building, in secret, with militaries around the world?

Is there a corollary here with spy satellites? Since the end of the Cold War, satellite technology has advanced to such a degree that we are being watched — for good or for bad — almost constantly by military, and commercial organisations. Many of the companies doing the work are commercial with the boundary between military and commercial now very blurred. As Pat Norris, a former NASA engineer who worked on the Apollo 11 mission to the moon and author of Spies in the Sky says “the best of the civilian satellites are taking pictures that would only have been available to military people less than 20 years ago”. If that is so then what are the military satellites doing now?

In his book Megatech: Technology in 2050 Daniel Franklin points out that Western liberal democracies often have a cultural advantage, militarily over those who grew up under a theocracy or authoritarian regime. With a background of greater empowerment in decision making and encouragement to learn from, and not be penalised by, mistakes, Westerners tend to display greater creativity and innovation. Education systems in democracies encourage the type of creative problem-solving that is facilitated by timely intelligence as well as terabytes of data that is neither controlled nor distorted by an illiberal regime.

Imagine then how advanced some of these robots could become, in military use, if they are trained using all of the data available to them from past military conflicts, both successful and not so successful campaigns?

Which brings me to my real concern about all this. If we are training our young scientists and engineers to build ‘platforms’ (which is how Boston Dynamics refers to its robots) that can learn from all of this data, and maybe to begin making decisions which are no longer understood by their creators, then whose responsibility is it when things go wrong?

Not only that, but what happens when the technology that was designed by an engineering team for a relatively benign use, is subverted by people who have more insidious ideas for deploying those ‘platforms’? As Meredith Broussard says in her book Artificial Unintelligence: “Blind optimism about technology and an abundant lack of caution about how new technologies will be used are a hallmark of techno-chauvinism”.

As engineers and scientists who hopefully care about the future of humanity and the planet on which we live surely it is beholden on us all to morally and ethically think about the technology we are unleashing? If we don’t then what Einstein said at the advent of the atomic age rings equally true today:

“It has become appallingly obvious that our technology has exceeded our humanity.”

Albert Einstein

Tech skills are not the only type of skill you’ll need in 2021

Peter Cripps comment, education , , ,
Image by Gerd Altmann from Pixabay

Whilst good technical skills continue to be important these alone will not be enough to enable you to succeed in the modern, post-pandemic workplace. At Digital Innovators, where I am Design and Technology Director, we believe that skills with a human element are equally, if not more, important if you are to survive in the changed working environment of the 2020’s. That’s why, if you attend one of our programmes during 2021, you’ll also learn these, as well as other, people focused, as well as transferable, skills.

1. Adaptability

The COVID-19 pandemic has changed the world of work not just in the tech industry but across other sectors as well. Those organisations most able to thrive during the crisis were ones that were able to adapt quickly to new ways of working whether that is full-time office work in a new, socially distanced way, a combination of both office and remote working, or a completely remote environment. People have had to adapt to these ways of working whilst continuing to be productive in their roles. This has meant adopting different work patterns, learning to communicate in new ways and dealing with a changed environment where work, home (and for many school) have all merged into one. Having the ability to adapt to these new challenges is a skill which will be more important than ever as we embrace a post-pandemic world.

Adaptability also applies to learning new skills. Technology has undergone exponential growth in even the last 20 years (there were no smartphones in 2000) and has been adopted in new and transformative ways by nearly all industries. In order to keep up with such a rapidly changing world you need to be continuously learning new skills to stay up-to-date and current with industry trends. 

2. Collaboration and Teamwork

Whilst there are still opportunities for the lone maverick, working away in his or her bedroom or garage, to come up with new and transformative ideas, for most of us, working together in teams and collaborating on ideas and new approaches is the way we work best.

In his book Homo Deus – A Brief History of Tomorrow, Yuval Noah Harari makes the observation: “To the best of our knowledge, only Sapiens can collaborate in very flexible ways with countless numbers of strangers. This concrete capability – rather than an eternal soul or some unique kind of consciousness – explains our mastery over planet Earth.”

On our programme we encourage and demand our students to collaborate from the outset. We give them tasks to do (like drawing how to make toast!) early on, then build on these, leading up to a major 8-week projects where students work in teams of four or five to define a solution to a challenge set by one of our industry partners. Students tell us this is one of their favourite aspects of the programme as it allows them to work with new people from a diverse range of backgrounds to come up with new and innovative solutions to problems.

3. Communication

Effective communication skills, whether they be written spoken or aural, as well as the ability to present ideas well, have always been important. In a world where we are increasingly communicating through a vast array of different channels, we need to adapt our core communications skills to thrive in a virtual as well as an offline environment.

Digital Innovators teach their students how to communicate effectively using a range of techniques including a full-day, deep dive into how to create presentations that tell stories and really enable you to get across your ideas.

4. Creativity

Pablo Picasso famously said “Every child is an artist; the problem is staying an artist when you grow up”.

As Hugh MacLeod, author of Ignore Everybody, And 39 Other Keys to Creativity says: “Everyone is born creative; everyone is given a box of crayons in kindergarten. Then when you hit puberty they take the crayons away and replace them with dry, uninspiring books on algebra, history, etc. Being suddenly hit years later with the ‘creative bug’ is just a wee voice telling you, ‘I’d like my crayons back please.’”

At Digital Innovators we don’t believe that it’s only artists who are creative. We believe that everyone can be creative in their own way, they just need to learn how to let go, be a child again and unlock their inner creativity. That’s why on our skills programme we give you the chance to have your crayons back.

5. Design Thinking

Design thinking is an approach to problem solving that puts users at the centre of the solution. It includes proven practices such as building empathy, ideation, storyboarding and extreme prototyping to create new products, processes and systems that really work for the people that have to live with and use them.

For Digital Innovators, Design Thinking is at the core of what we do. As well as spending a day-and-a-half teaching the various techniques (which our students learn by doing) we use Design Thinking at the beginning of, and throughout, our 8-week projects to ensure the students deliver solutions are really what our employers want.

6. Ethics

The ethical aspects on the use of digital technology in today’s world is something that seems to be sadly missing from most courses in digital technology. We may well churn out tens of thousands of developers a year, from UK universities alone, but how many of these people ever give anything more than a passing thought to the ethics of the work they end up doing? Is it right, for example, to build systems of mass surveillance and collect data about citizens that most have no clue about? Having some kind of ethical framework within which we operate is more important today than ever before.

That’s why we include a module on Digital Ethics as part of our programme. In it we introduce a number of real-world, as well as hypothetical case studies that challenge students to think about the various ethical aspects of the technology they already use or are likely to encounter in the not too distant future.

7. Negotiation

Negotiation is a combination of persuasion, influencing and confidence as well as being able to empathise with the person you are negotiating with and understanding their perspective. Being able to negotiate, whether it be to get a pay rise, buy a car or sell the product or service your company makes is one of the key skills you will need in your life and career, but one that is rarely taught in school or even at university.

As Katherine Knapke, the Communications & Operations Manager at the American Negotiation Institute says: “Lacking in confidence can have a huge impact on your negotiation outcomes. It can impact your likelihood of getting what you want and getting the best possible outcomes for both parties involved. Those who show a lack of confidence are more likely to give in or cave too quickly during a negotiation, pursue a less-aggressive ask, and miss out on opportunities by not asking in the first place”. 

On the Digital Innovators skills programme you will work with a skilled negotiator from The Negotiation Club to practice and hone your negotiation skills in a fun way but in a safe environment which allows you to learn from your mistakes and improve your negotiation skills.

The ethics of contact tracing

Peter Cripps architecture, comment , , ,

After a much publicised “U-turn” the UK government has decided to change the architecture of its coronavirus contact tracing system and to embrace the one based on the interfaces being provided by Apple and Google. The inevitable cries of a government that does not know what it is doing, we told you it wouldn’t work and this means we have wasted valuable time in building a system that would help protect UK citizens have ensued. At times like these it’s often difficult to get to the facts and understand where the problems actually lie. Let’s try and unearth some facts and understand the options for the design of a contact tracing app.

Any good approach to designing a system such as contact tracing should, you would hope, start with the requirements. I have no government inside knowledge and it’s not immediately apparent from online searches what the UK governments exact and actual requirements were. However as this article highlights you would expect that a contact tracing system would need to “involve apps, reporting channels, proximity-based communication technology and monitoring through personal items such as ID badges, phones and computers.” You might also expect it to involve cooperation with local health service departments. Whether or not there is also a requirement to collate data in some centralised repository so that epidemiologists, without knowing the nature of the contact, can build a model of contacts to see if they are serious spreaders or those who have tested positive yet are asymptomatic, at least for the UK, is not clear. Whilst it would seem perfectly reasonable to want the system to do that, this is a different use case to the one of contact tracing. One might assume that because the UK government was proposing a centralised database for tracking data this latter use case was also to be handled by the system.

Whilst different countries are going to have different requirements for contact tracing one would hope that for any democratically run country a minimum set of requirements (i.e. privacy, anonymity, transparency and verifiability, no central repository and minimal data collection) would be implemented.

The approach to contact tracing developed by Google and Apple (the two largest providers of mobile phone operating systems) was published in April of this year with the detail of the design being made available in four technical papers. Included as part of this document set were some frequently asked questions where the details of how the system would work were explained using the eponymous Alice and Bob notation. Here is a summary.

  1. Alice and Bob don’t know each other but happen to have a lengthy conversation sitting a few feet apart on a park bench. They both have a contact tracing app installed on their phones which exchange random Bluetooth identifiers with each other. These identifiers change frequently.
  2. Alice continues her day unaware that Bob had recently contracted Covid-19.
  3. Bob feels ill and gets tested for Covid-19. His test results are positive and he enters his result into his phone. With Bob’s consent his phone uploads the last 14 days of keys stored on his phone to a server.
  4. Alice’s phone periodically downloads the Bluetooth beacon keys of everyone who has tested positive for Covid-19 in her immediate vicinity. A match is found with Bob’s randomly generated Bluetooth identifier.
  5. Alice sees a notification on her phone warning her she has recently come into contact with someone who has tested positive with Covid-19. What Alice needs to do next is decided by her public health authority and will be provided in their version of the contact tracing app.

There are a couple of things worth noting about this use case:

  1. Alice and Bob both have to make an explicit choice to turn on the contact tracing app.
  2. Neither Alice or Bob’s names are ever revealed, either between themselves or to the app provider or health authority.
  3. No location data is collected. The system only knows that two identifiers have previously been within range of each other.
  4. Google and Apple say that the Bluetooth identifiers change every 10-20 minutes, to help prevent tracking and that they will disable the exposure notification system on a regional basis when it is no longer needed.
  5. Health authorities of any other third parties do not receive any data from the app.

Another point to note is that initially this solution has been released via application programming interfaces (APIs) that allow customised contact tracing apps from public health authorities to work across Android and iOS devices. Maintaining user privacy seems to have been a key non-functional requirement of the design. The apps are made available from the public health authorities via the respective Apple and Google app stores. A second phase has also been announced whereby the capability will be embedded at the operating system level meaning no app has to be installed but users still have to opt into using the capability. If a user is notified she has been in contact with someone with Covid-19 and has not already downloaded an official public health authority app they will be prompted to do so and advised on next steps. Only public health authorities will have access to this technology and their apps must meet specific criteria around privacy, security, and data control as mandated by Apple and Google.

So why would Google and Apple choose to implement its contact tracing app in this way which would seem to be putting privacy ahead of efficacy? More importantly why should Google and Apple get to dictate how countries should do contact tracing?

Clearly one major driver from both companies is that of security and privacy. Post-Snowden we know just how easy it has been for government security agencies (i.e. the US National Security Agency and UK’s Government Communications Headquarters) to get access to supposedly private data. Trust in central government is at an all time low and it is hardly surprising that the corporate world is stepping in to announce that they were the good guys all along and you can trust us with your data.

Another legitimate reason is also that during the coronavirus pandemic we have all had our ability to travel even locally, never mind nationally or globally, severely restricted. Implementing an approach that is supported at the operating system level means that it should be easier to make the app compatible with other countries’ counterparts, which are based on the same system therefore making it safer for people to begin travelling internationally again.

The real problem, at least as far as the UK has been concerned, is that the government has been woefully slow in implementing a rigorous and scaleable contact tracing system. It seems as though they may have been looking at an app-based approach to be the silver bullet that would solve all of their problems – no matter how poorly identified these are. Realistically that was never going to happen, even if the system had worked perfectly. The UK is not China and could never impose an app based contact tracing system on its populace, could it? Lessons from Singapore, where contact tracing has been in place for some time, are that the apps do not perform as required and other more intrusive measures are needed to make them effective.

There will now be the usual blame game between government, the press, and industry, no doubt resulting in the inevitable government enquiry into what went wrong. This will report back after several months, if not years, of deliberation. Blame will be officially apportioned, maybe a few junior minister heads will roll, if they have not already moved on, but meanwhile the trust that people have in their leaders will be chipped away a little more.

More seriously however, will we have ended up, by default, putting more trust into the powerful corporations of Silicon Valley some of whom not only have greater valuations than many countries GDP but are also allegedly practising anti-competitive behaviour?

Update: 21st June 2020

Updated to include link to Apple’s anti-trust case.

On Ethics and Algorithms

Peter Cripps comment , , , ,

franck-v-g29arbbvPjo-unsplash
Photo by Franck V. on Unsplash

An article on the front page of the Observer, Revealed: how drugs giants can access your health records, caught my eye this week. In summary the article highlights that the Department of Health and Social Care (DHSC) has been selling the medical data of NHS patients to international drugs companies and have “misled” the public that the information contained in the records would be “anonymous”.

The data in question is collated from GP surgeries and hospitals and, according to “senior NHS figures”, can “routinely be linked back to individual patients’ medical records via their GP surgeries.” Apparently there is “clear evidence” that companies have identified individuals whose medical histories are of “particular interest.” The DHSC have replied by saying it only sells information after “thorough measures” have been taken to ensure patient anonymity.

As with many articles like this it is frustrating when some of the more technical aspects are not fully explained. Whilst I understand the importance of keeping their general readership on board and not frightening them too much with the intricacies of statistics or cryptography it would be nice to know a bit more about how these records are being made anonymous.

There is a hint of this in the Observer report when it states that the CPRD (the Clinical Practice Research Datalink ) says the data made available for research was “anonymous” but, following the Observer’s story, it changed the wording to say that the data from GPs and hospitals had been “anonymised”. This is a crucial difference. One of the more common methods of ‘anonymisation’  is to obscure or redact some bits of information. So, for example, a record could have patient names removed and ages and postcodes “coarsened”, that is only the first part of a postcode (e.g. SW1A rather than SW1A 2AA)  are included and ages are placed in a range rather than using someones actual age (e.g. 60-70 rather than 63).

The problem with anonymising data records is that they are prone to what is referred to as data re-identification or de-anonymisation. This is the practice of matching anonymous data with publicly available information in order to discover the individual to which the data belongs. One of the more famous examples of this is the competition that Netflix organised encouraging people to improve its recommendation system by offering a $50,000 prize for a 1% improvement. The Netflix Prize was started in 2006 but abandoned in 2010 in response to a lawsuit and Federal Trade Commission privacy concerns. Although the dataset released by Netflix to allow competition entrants to test their algorithms had supposedly been anonymised (i.e. by replacing user names with a meaningless ID and not including any gender or zip code information) a PhD student from the University of Texas was able to find out the real names of people in the supplied dataset by cross-referencing the Netflix dataset with Internet Movie Database (IMDB) ratings which people post publicly using their real names.

Herein lies the problem with the anonymisation of datasets. As Michael Kearns and Aaron Roth highlight in their recent book The Ethical Algorithm, when an organisation releases anonymised data they can try and make an intelligent guess as to which bits of the dataset to anonymise but it can be difficult (probably impossible) to anticipate what other data sources either already exist or could be made available in the future which could be used to correlate records. This is the reason that the computer scientist Cynthia Dwork has said “anonymised data isn’t” – meaning either it isn’t really anonymous or so much of the dataset has had to be removed that it is no longer data (at least in any useful way).

So what to do? Is it actually possible to release anonymised datasets out into the wild with any degree of confidence that they can never be de-anonymised? Thankfully something called differential privacy, invented by the aforementioned Cynthia Dwork and colleagues, allows us to do just that. Differential privacy is a system for publicly sharing information about a dataset by describing the patterns of groups within the dataset while withholding information about individuals in that dataset.

To understand how differential privacy works consider this example*. Suppose we want to conduct a poll of all people in London to find out who have driven after taking non-prescription drugs. One way of doing this is to randomly sample a suitable number of Londoners, asking them if they have ever driven whilst under the influence of drugs. The data collected could be entered into a spreadsheet and various statistics, e.g. number of men, number of women, maybe ages etc derived. The problem is that whilst collecting this information lots of compromising personal details may be collected which, if the data were stolen, could be used against them.

In order to avoid this problem consider the following alternative. Instead of asking people the question directly, first ask them to flip a coin but not to tell us how it landed. If the coin comes up heads they tell us (honestly) if they have driven under the influence. If it comes up tails however they tell us a random answer then flip the coin again and tell us “yes” if it comes up heads or “no” if it is tails. This polling protocol is a simple randomised algorithm which is a form of differential privacy. So how does this work?

differential privacy
If your answer is no, the randomised response answers no two out of three times. It answers no only one out of three times if your answer is yes. Diagram courtesy Michael Kearns and Aaron Roth, The Ethical Algorithm 2020

When we ask people if they have driven under the influence using this protocol half the time (i.e. when the coin lands heads up) the protocol tells them to tell the truth. If the protocol tells them to respond with a random answer (i.e. when the coin lands tails up), then half of that time they just happen to randomly tell us the right answer. So they tell us the right answer 1/2 + ((1/2) x (1/2)) or three-quarters of the time. The remaining one quarter of the time they tell us a lie. There is no way of telling true answers from lies. Surely though, this injection of randomisation completely masks the true results and the data is now highly error prone? Actually, it turns out, this is not the case.

Because we know how this randomisation is introduced we can reverse engineer the answers we get to remove the errors and get an approximation of the right answer. Here’s how. Suppose one-third of people in London have actually driven under the influence of drugs. So of the one-third who have truthfully answered “yes” to the question, three-quarters of those will answer “yes” using the protocol, that is 1/3 x 3/4 = 1/4. Of the two-thirds who have a truthful answer of “no”, one-quarter of those will report “yes”, that is 2/3 x 1/4 = 1/6. So we expect 1/4 + 1/6 = 5/12 ~ 1/3 of the population to answer “yes”.

So what is the point of doing the survey like this? Simply put it allows the true answer to be hidden behind the protocol. If the data were leaked and an individual from it was identified as being suspected of driving under the influence then they could always argue they were told to say “yes” because of the way the coins fell.

In the real world a number of companies including the US census, Apple, Google and Privitar Lens use differential privacy to limit the disclosure of private information about individuals whose information is in public databases.

It would be nice to think that the NHS data that is supposedly being used by US drug companies was protected by some form of differential privacy. If it were, and if this could be explained to the public in a reasonable and rational way, then surely we would all benefit both in the knowledge that our data is safe and is maybe even being put to good use in protecting and improving our health. After all, wasn’t this meant to be the true benefit of living in a connected society where information is shared for the betterment of all our lives?

*Based on an example from Kearns and Roth in The Ethical Algorithm.

Why I Became a Facebook Refusenik

Peter Cripps comment , , , , , , , , , , , ,

I know it’s a new year and that generally is a time to make resolutions, give things up, do something different with your life etc but that is not the reason I have decided to become a Facebook refusenik.

Image Copyright http://www.keepcalmandposters.com
Image Copyright http://www.keepcalmandposters.com

Let’s be clear, I’ve never been a huge Facebook user amassing hundreds of ‘friends’ and spending half my life on there. I’ve tended to use it to keep in touch with a few family and ‘real’ friend members and also as a means of contacting people with a shared interest in photography. I’ve never found the user experience of Facebook particularly satisfying and indeed have found it completely frustrating at times; especially when posts seem to come and go, seemingly at random. I also hated the ‘feature’ that meant videos started playing as soon as you scrolled them into view. I’m sure there was a way of preventing this but was never interested enough to figure out how to disable it. I could probably live with these foibles however as by and large the benefits outweighed the unsatisfactory aspects of Facebook’s usability.

What’s finally decided me to deactivate my account (and yes I know it’s still there just waiting for me to break and log back in again) is the insidious way in which Facebook is creeping into our lives and breaking down all aspects of privacy and even our self-determination. How so?

First off was the news in June 2014 that Facebook had conducted a secret study involving 689,000 users in which friends’ postings were moved to influence moods. Various tests were apparently performed. One test manipulated a users’ exposure to their friends’ “positive emotional content” to see how it affected what they posted. The study found that emotions expressed by friends influence our own moods and was the first experimental evidence for “massive-scale emotional contagion via social networks”. What’s so terrifying about this is whether, as Clay Johnson the co-founder of Blue State Digital asked via Twitter is “could the CIA incite revolution in Sudan by pressuring Facebook to promote discontent? Should that be legal? Could Mark Zuckerberg swing an election by promoting Upworthy (see later) posts two weeks beforehand? Should that be legal?”

As far as we know this has been a one off which Facebook apologised for but the mere fact they thought they could get away with such a tactic is, to say the least, breathtaking in its audacity and not an organisation I am comfortable with entrusting my data to.

Next was the article by Tom Chatfield called The Attention Economy in which he discusses the idea that “attention is an inert and finite resource, like oil or gold: a tradable asset that the wise manipulator (i.e. Facebook and the like) auctions off to the highest bidder, or speculates upon to lucrative effect. There has even been talk of the world reaching ‘peak attention’, by analogy to peak oil production, meaning the moment at which there is no more spare attention left to spend.” Even though I didn’t believe Facebook was grabbing too much of my attention I was starting to become a little concerned that Facebook was often the first site I visited in the morning and was even becoming diverted by some of those posts in my newsfeed with titles like “This guy went to collect his mail as usual but you won’t believe what he found in his mailbox”. Research is beginning to show that doing more than one task at a time, especially more than one complex task, takes a toll on productivity and that the mind and brain were not designed for heavy-duty multitasking. As Danny Crichton argues here “we need to recognize the context that is distracting us, changing what we can change and advocating for what we can hopefully convince others to do.”

The final straw that has made me throw in the Facebook towel however was reading The Virologist by Andrew Marantz in The New Yorker magazine about Emerson Spartz the so called ‘king of clickbait”. Spartz is twenty-seven and has been successfully launching Web sites for more than half his life. In 1999, when Spartz was twelve, he built MuggleNet, which became the most popular Harry Potter fan site in the world. Spartz’s latest venture is Dose a photo- and video-aggregation site whose posts are collections of images designed to tell a story. The posts have names like “You May Feel Bad For Laughing At These 24 Accidents…But It’s Too Funny To Look Away“. Dose gets most of its feeds through Facebook. A bored teenager absent mindedly clicking links will eventually end up on a site like Dose. Spartz’s goal is to make the site so “sticky”—attention-grabbing and easy to navigate—that the teenager will stay for a while. Money is generated through ads – sometimes there are as many as ten on a page and Spartz hopes to develop traffic-boosting software that he can sell to publishers and advertisers. Here’s the slightly disturbing thing though. Algorithms for analysing users behaviour are “baked in” to the sites Spartz builds. When a Dose post is created, it initially appears under as many as two dozen different headlines, distributed at random to different Facebook users. An algorithm measures which headline is attracting clicks most quickly, and after a few hours, when a statistically significant threshold is reached, the “winning” headline automatically supplants all others. Hence users are “click-bait”, unknowingly taking part in a “test” to see how quickly they respond to a headline.

The final, and most sinister aspect to what Spartz is trying to do with Dose and similar sites is left to the end of Marantz’s article when Spartz gives his vision of the future of media:

The lines between advertising and content are blurring,” he said. “Right now, if you go to any Web site, it will know where you live, your shopping history, and it will use that to give you the best ad. I can’t wait to start doing that with content. It could take a few months, a few years—but I am motivated to get started on it right now, because I know I’ll kill it.

The ‘content’ that Spartz talks about is news. In other words he sees his goal is to feed us the news articles his algorithms calculate we will like. We will no longer be reading the news we want to read but rather that which some computer program thinks we should be reading, coupled of course with the ads the same program thinks we are most likely to respond to.

If all of this is not enough to concern you about what Facebook is doing (and the sort of companies it collaborates with) then the recent announcement of ‘keyword’ or ‘graph’ search might. Keyword search allows you to search content previously shared with you by entering a word or phrase. Privacy settings aren’t changing, and keyword search will only bring up content shared with you, like posts by friends or that friends commented on, not public posts or ones by Pages. But if a friend wanted to easily find posts where you said you were “drunk”, now they could. That accessibility changes how “privacy by obscurity” effectively works on Facebook. Rather than your posts being effectively lost in the mists of time (unless your friends want to methodically step through all your previous posts that is) your previous confessions and misdemeanors are now just a keyword search away. Maybe now is the time to take a look at your Timeline or search for a few dubious words with your name to check for anything scandalous before someone else does? As this article points out there are enormous implications of Facebook indexing trillions of our posts some we can see now but others we can only begin to guess at as ‘Zuck’ and his band of researchers do more and more to mine our collective consciousness’.

So that’s why I have decided to deactivate my Facebook account. For now my main social media interactions will be through Twitter (though that too is obviously working out how it can make money out of better and more targeted advertising of course). I am also investigating Ello which bills itself as “a global community that believes that a social network should be a place to empower, inspire, and connect — not to deceive, coerce, and manipulate.” Ello takes no money from advertising and reckons it will make money from value added services. It is early days for Ello yet and it still receives venture capital money for its development. Who knows where it will go but if you’d like to join with me on there I’m @petercripps (contact me if you want an invite).

I realise this is a somewhat different post from my usual ones on here. I have written posts before on privacy in the internet age but I believe this is an important topic for software architects and one I hope to concentrate on more this year.

Let’s Build a Smarter Planet – Part IV

Peter Cripps architecture , , , , , ,

This is the fourth and final part of the transcript of a lecture I recently gave at the University of Birmingham in the UK.In Part I of this set of four posts I tried to give you a flavour of what IBM is and what it is trying to do to make our planet smarter. In Part II I looked at my role in IBM and in Part III I looked at what kind of attributes IBM looks for in its graduate entrants. In this final part I take a look at what I see as some of the challenges we face in a world of open and ubiquitous data where potentially anyone can know anything about us and what implications that has on people who design systems that allow that to happen.

So let’s begin with another apocryphal tale…ec12d-whosewatchingyou

Target is the second largest (behind Walmart) discount retail store in America. Using advanced analytics software one of Target’s data analysts identified 25 products that when purchased together indicate a women is likely to be pregnant. The value of this information was that Target could send coupons to the pregnant woman at an expensive and habit-forming period of her life.

In early 2012 a man walked into a Target store outside Minneapolis and demanded to see the manager. He was clutching coupons that had been sent to his daughter, and he was angry, according to an employee who participated in the conversation. “My daughter got this in the mail!” he said. “She’s still in high school, and you’re sending her coupons for baby clothes and cribs? Are you trying to encourage her to get pregnant?”

The manager didn’t have any idea what the man was talking about. He looked at the mailer. Sure enough, it was addressed to the man’s daughter and contained advertisements for maternity clothing, nursery furniture and pictures of smiling infants. The manager apologized and then called a few days later to apologize again.

On the phone, though, the father was somewhat abashed. “I had a talk with my daughter,” he said. “It turns out there’s been some activities in my house I haven’t been completely aware of. She’s due in August. I owe you an apology.”fd140-thisisforeveryone

Two of the greatest inventions of our time are the internet and the mobile phone. When Tim Berners-Lee appeared from beneath the semi-detached house that lifted up from the ground of the Olympic stadium during the London 2012 opening ceremony and the words “this is for everyone” flashed up around the edge of the stadium there can surely be little doubt that he had earned his place there. However as with any technology there is a downside as well as an upside. A technology that gives anyone, anywhere access to anything they choose has to be treated with great care and responsibility (as Spiderman’s uncle said, “with great power comes great responsibility”). The data analyst at Target was only trying to improve his companies profits by identifying potential new consumers of its baby products. Inadvertently however he was uncovering information that previously would have been kept very private and only known to a few people. What should companies do in balancing a persons right to privacy with a companies right to identify new customers?

There is an interesting book out at the moment called Age of Context in which the authors examine the combined effects of five technological ‘forces’ that they see as coming together to form a ‘perfect storm’ that they believe are going to change forever our world. These five forces are mobile, social media, (big) data, sensors and location aware services. As the authors state:

The more the technology knows about you, the more benefits you will receive. That can leave you with the chilling sensation that big data is watching you…

In the Internet of Things paradigm, data is gold. However, making that data available relies on a ‘contract’ between suppliers (usually large corporations) and consumers (usually members of the public). Corporations provide a free or nominally-priced service in exchange for a consumer’s personal data. This data is either sold to advertisers or used to develop further products or services useful to consumers. Third-party applications, which build off the core service, poach customers (and related customer data) from such applications. For established networks and large corporations, this can be detrimental practice because such applications eventually poach their customers. In such a scenario, large corporations need to balance their approach to open source with commercial considerations.

Companies know that there is a difficult balancing act between doing what is commercially advantageous and doing what is ethically the right. As the saying goes – a reputation takes years to be built but can be destroyed in a matter of minutes.

IBM has an organisation within it called the Academy of Technology (AoT) which has as its membership around 1000 IBM’ers from its technical community. The job of the AoT is to focus on “uncharted business and technical opportunities” that help to “facilitate IBM’s technical development” as well as “more tightly integrate the company’s business and technical strategy”. As an example of the way IBM concerns itself with issues highlighted by the story about Target one of the studies the academy looked at recently was into the ethics of big data and how it should approach problems we have mentioned here. Out of that study came a recommendation for a framework the company should follow in pursuing such activities.

This ethical framework is articulated as a series of questions that should be asked when embarking on a new or challenging business venture.

  1. What do we want to do?
  2. What does the technology allow us to do?
  3. What is legally allowable?
  4. What is ethically allowable?
  5. What does the competition do?
  6. What should we do?

As an example of this consider the insurance industry.

  • The Insurance Industry provides a service to society by enabling groups of people to pool risk and protect themselves against catastrophic loss.
  • There is a duty to ensure that claims are legitimate.
  • More information could enable groups with lower risk factors to reduce their cost basis but those in higher risk areas would need to increase theirs.
  • Taken to the extreme, individuals may no longer be able to buy insurance – e.g. using genetic information to determine medical insurance premium.

How far should we take using technology to support this extreme case? Whilst it may not be breaking any laws to raise someones insurance premium to a level where they cannot afford it, is it ethically the right thing to do?Make no mistake the challenges we face in making our planet smarter through the proper and considered use of information technology are considerable. We need to address questions such as how do we build the systems we need, where does the skilled and creative workforce come from that can do this and how do we approach problems in new and innovative ways whilst at the same time doing what is legally and ethically right.

The next part is up to you…

Thank you for your time this afternoon. I hope I have given you a little more insight into the type of company IBM is, how and why it is trying to make the planet smarter and what you might do to help if you choose to join us. You can find more information about IBM and its graduate scheme here and you can find me on Twitter and Linkedin if you’d like to continue the conversation (and I’d love it if you did).

Thank you!