Sunday Sweep: Week in Review (March 3, 2024)

Ankesh Chandaria

March 3, 2024

(

March 11, 2024

sunday sweep

regulation

global

large language models

artificial general intelligence

tl;dr

This week, Brazil adopted new A.I. regulation tackling election disinformation. It requires disclosing A.I. generated content as such, restricts chatbot use, and prohibits deepfakes. In an important election year across the globe, this'll be a space to watch closely. Another lawyer (this time in B.C., Canada) fell victim to LLM hallucinations and received a costs penalty for her blunder. Meanwhile, a Caribbean court is turning to A.I. for automatic transcript generation. Finally, Elon's suing OpenAI, ostensibly because they're taking money from Microsoft and that isn't what OpenAI was meant to be. While it's hard not to be skeptical about his intentions, the case certainly opens up interesting questions around AGI.

Here are some of the stand-out A.I. stories from last week.

Brazil Adopts Election A.I. Regulations

On Tuesday, 27th February, Brazil’s top electoral court (the TSE) adopted a regulation controlling the sorts of disinformation that might unduly influence the country’s upcoming (2024) elections [1]. The regulation:

requires disclosure of A.I. generated content (i.e., if it's made with gen-A.I. then it should be labeled as such); ‍
restricts on the use of chatbots (where they might be confused for electoral candidates or other real humans); and
features an outright prohibition on deepfakes. This isn't surprising. It comes in the context of Brazil's work on disinformation issues generally since 2018 [2].

But is the approach effective? Depends on who's held accountable and what the end goal is. In Brazil's case, it appears the sights are levelled at candidates in breach, who, according to Alexandre de Moraes (president of the TSE), "will face the penalty of having their registration revoked and, if they have already been elected, their mandate revoked." [3]

So it's got teeth.

Won't this mean that deepfakes will just percolate under the surface in WhatsApp Groups, Telegram, etc.? Probably. But forcing candidates and, ideally, their affiliates, to think hard before stooping to these tactics or reposting material they're unsure about should still be helpful. Particularly where the price to pay is their spot on the ballot.

That's not to say that tech companies that serve up this sort of content shouldn't also be taken to task. In Brazil, TSE Resolution 23.714 of 2022, Article 2, gives the TSE power to order social media companies to remove offending disinformation under significant time-based financial penalties that start racking up an hour after notification.

Ultimately, A.I. is going to continue to facilitate exponentially easier ways to conjure up the sorts of believable fictions that may influence elections. As such, information integrity is a fundamental challenge democratic countries will have to wrestle with. Surely, as a starting point -- and no matter our political affiliations -- we all want a robust system of elections that is inured to influences of bad actors who can increasingly use A.I. as a tool to ‘legitimize’ their practices. I, for one, want to vote for whomever I think is the right person for the job, on the basis of the right (read real and relevant) information available about them.

I'm working on a deeper dive into how other jurisdictions are tackling this poignant issue. In a year where many eyes will be turned to the U.S. lead up to elections in November, it's a conversation we need to be having. Watch this space.

‍

DNNs In Court

LLM Troubles in Canada

A lawyer in British Columbia has been ordered to pay special costs in a novel case of ChatGPT (mis)use in Canadian courts [4]. The lawyer submitted an application which cited two non-existent cases, only discovered to be A.I. hallucinations when opposing counsel was unable to find them by citation alone. The B.C. Supreme Court judge found no "intent to deceive"; however, ordered the lawyer to pay costs for the time it took opposing counsel to uncover that the two cases were non-existent.

The moral of the story is simple: approach LLMs with eyes wide open.

For most people, LLMs appear miraculous black-boxes that can conjure up the answer to any question. However, ignorance as to their limitations is obviously no excuse when putting them to use in a professional advisory capacity. It's important to bear in mind what they really are. Shanahan puts it all rather concisely in a paper I highly recommend: LLMs are models that we can query and their purpose is to "generate statistically likely sequences of words" [5]. Nothing more, nothing less. Hallucinations are an almost inevitable byproduct of these models due to issues such as overfitting or the quality of training data. They should be expected.

All that said, it’s not the first time and it won’t be the last time. LLMs save time and are a useful drafting tool. But, as it stands, they can’t yet reliably replace good-ol'-fashioned research.

AI Transcription in the Caribbean

Meanwhile, the Eastern Caribbean Supreme Court (ECSC) is soft-launching A.I. powered transcript tool in St Kitts and Nevis [6]. Ideally, this technology should save a tremendous amount of manual labour. The same lessons from the B.C. blunder mentioned above apply. Hopefully, a pair of human eyes reviews the transcripts for any nonsense before they become part of any sort of official record.

‍

Musk Sues OpenAI, For Reasons

Another week, another Elon story. Cynicism aside, this one raises some intriguing A.I.-related questions.

In a lawsuit filed against OpenAI on Thursday, 29 February, Musk has alleged breach of contract, suggesting that the company has deviated from its original purpose of pursuing AGI openly for the benefit of the public [7]. According to reports, there's no obvious founding agreement per se -- rather, it's a patchwork creation of conversations and statements from documents of incorporation [8]. There's also apparently a claim of breach of fiduciary duty in there.

The wildest element, however, might be that Elon wants the court to determine that OpenAI has, in fact, created AGI with its latest model. The embedded demand is that this technology must be released to the public and should not benefit any of OpenAI's leadership or Microsoft (which has invested billions into a partnership with OpenAI for its tech).

But it's worth backing up for a moment. If this goes the way Elon intends, it will lead to the court -- indeed, a jury -- having to determine the bar for when something is AGI. This is arguably a crucial philosophical question still doing the rounds. It might be hard to find consensus about how to define AGI even amongst A.I. experts, let alone lawyers and laypeople. With the determination of AGI comes a suite of other ethical issues about legal standing, and even discussions around the boundaries of rights that may or may not be afforded to AGI.

I'm not so sure the court is equipped to step into this particular minefield. That said, stranger things have happened.

References

[1] Mari, Angelica. 2024. “Brazil Outlines Rules For AI Use During Elections.” Forbes, February 28, 2024. https://www.forbes.com/sites/angelicamarideoliveira/2024/02/28/brazil-outlines-rules-for-ai-use-during-elections/?sh=8a78ac11f6ab.

[2] For a deeper regulatory overview, see: Rafael Rubio & Vitor de Andrade Monteiro (2023) Preserving trust in democracy: The Brazilian Superior Electoral Court's quest to tackle disinformation in elections, South African Journal of International Affairs, 30:3, 497-520, DOI: 10.1080/10220461.2023.2274860

[3] Grattan, Steven. 2024. “Brazil Justice Moraes warns political candidates not to use AI against opponents.” Reuters. February 29, 2024. https://www.reuters.com/world/americas/brazil-justice-moraes-warns-political-candidates-not-use-ai-against-opponents-2024-02-29/.

[4] Proctor, Jason. 2024. “B.C. Lawyer Reprimanded For Citing Fake Cases Invented by ChatGPT.” CBC, February 27, 2024. https://www.cbc.ca/news/canada/british-columbia/lawyer-chatgpt-fake-precedent-1.7126393.

[5] Shanahan, M. (2022). Talking About Large Language Models. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2212.03551

[6] EIN News. 2024. “ECSC Initiates Pilot Project in St Kitts and Nevis With AI Technology to Revolutionise Court Proceedings.” EIN News, March 2, 2024. https://tech.einnews.com/pr_news/692890894/ecsc-initiates-pilot-project-in-st-kitts-and-nevis-with-ai-technology-to-revolutionise-court-proceedings.

[7] Reuters. 2024. “Elon Musk Launches Lawsuit Against OpenAI, Sam Altman. Why?” Global News, March 1, 2024. https://globalnews.ca/news/10328745/elon-musk-openai-sam-altman-lawsuit/.

[8] Ephrat Livni, Lauren Hirsch, Sarah Kessler and Michael J. De La Merced. 2024. “The Big Questions Raised by Elon Musk’s Lawsuit Against OpenAI.” The New York Times, March 2, 2024. https://www.nytimes.com/2024/03/02/business/dealbook/the-big-questions-raised-by-elon-musks-lawsuit-against-openai.html.

You may also be interested in...

A Deep and Digital Breath: Designing Responsible Anthropomorphic AI

November 1, 2024

When ChatGPT started "breathing" during voice interactions this year, this design choice seemed innocuous. However, it highlights a concern around increasingly human-like AI features being implemented without proper evaluation of their necessity or impact. The tragic case of Character.AI -- where anthropomorphic design choices allegedly contributed to a teenager's death -- demonstrates the urgent need for systematic analysis of these features. This article proposes a two-step framework for evaluating anthropomorphic AI design: first assessing both social and technical necessity, then implementing appropriate guidelines and safeguards (some of which are suggested herein). While some applications (like accessibility tools) may well justify human-like features, others (like companionship AI) tread on ethically difficult territory. As AI becomes more sophisticated, we need clear criteria for determining when anthropomorphic design serves legitimate social purposes. An LLM doesn't really need to breathe, and perhaps we ought to think much harder about how we would like human-to-human relationships to look like in the future before delving too far into enabling human-to-AI facsimiles.

Mindreading Models: Do LLMs Have Theory of Mind?

June 2, 2024

Did you know we're all mind readers? Humans develop what’s called “theory of mind” (ToM) around the age of four, enabling us to hold views and beliefs about our own mental states and the mental states of others. It’s one of those things that, well, makes us human. However, a recent paper by Strachan et al. explores this capability in the context of AI -- a finding which has some interesting implications. The authors compared the performance of a series of LLMs (GPT-3, GPT-4, and LLaMA2) against around 1,900 humans across a battery of tests designed to explore ToM. They determined that, in every task bar one (designed to test faux pas), LLMs can perform at or above human level. My own cursory testing suggests that GPT-4o may even be able to demonstrate ToM across the board. While we can’t conclude that these models actually have ToM (in a cognitive sense) they certainly seem to behave as if they do. So what does this mean in practice? These LLMs open the door to exciting potential applications across areas such as mental health and decision support making. Ultimately, this paper’s conclusions give us a moment to stop and think about the state of our technology: to be impressed by what we’ve accomplished, and acknowledge the risks that should be considered as we move forward.

Whistleblower Woes: The Right to Warn

June 5, 2024

On 4 June 2024, employees at OpenAI and Google DeepMind released an open letter imploring their companies to facilitate what they call a right to warn about advanced AI. In the absence of proper government oversight, they call upon their companies to: (1) not enter or enforce any non-disparagement clauses when it comes to risk-related concerns; (2) develop internal, verifiably anonymous, whistleblower processes; (3) support a culture of open criticism which enables disclosure of risk-related concerns; and (4) not retaliate against employees (past or present) who choose to publicly voice risk-related concerns where other processes have failed. In the uncomfortable light of commercial reality, most of these requests appear unrealistic. A demand for robust internal whistleblowing processes, however, is an actionable and arguably best next step -- particularly given the poor state of protection under the current whistleblowing regime in England and Wales. There should be a right to warn about advanced AI. This demand may well be the starting point.

Who’s Responsible for AI Safety?

May 22, 2024

The past few weeks have brought the issue of AI Safety to the fore in two contrasting ways: first, OpenAI launched GPT-4o, a promising new model that will be free for public use. Around the same time, Ilya Sutskever (OpenAI's renowned co-founder and chief scientist) left the company amidst reports that OpenAI had dissolved its long-term AI Safety team. Although the company reiterated its commitment to safe development, questions linger. Second, the UK's Department of Science, Innovation and Technology released its interim International Scientific Report on the Safety of Advanced AI. The report focuses on general-purpose AI and explores various risks as well as some technical risk mitigation techniques. It also leaves plenty unanswered. With so much uncertainty, it's imperative we seek to resolve the most practically-relevant question as soon as possible: who's responsible for AI Safety?

Sunday Sweep: Week in Review (March 31, 2024)

March 31, 2024

With this week's condensed version of the Sunday Sweep, we focus on a couple of key developments. First, Neuralink's released an incredible video showing its first human patient, Noland Arbaugh, manipulating a cursor on a monitor entirely with his mind. This sort of technology demands serious consideration both for the tremendous benefits it can bring as well as the potential ethical issues that come hand-in-hand with it. On the regulatory side, representatives of the US Congress proposed the Protecting Consumers from Deceptive AI Act -- federal legislation designed to protect the public from deepfakes. The White House Office of Management and Budget issued its first major AI policy, and a pair of additional government departments released their own AI reports (including a Treasury report on managing AI cybersecurity risk in the financial services sector).