In what may be a first-of-its-kind study, artificial intelligence (AI) firm Anthropic has developed a large language model (LLM) thatโs been fine-tuned for value judgments by its user community.
What does it mean for AI development to be more democratic? To find out, we partnered with @collect_intel to use @usepolis to curate an AI constitution based on the opinions of ~1000 Americans. Then we trained a model against it using Constitutional AI. pic.twitter.com/ZKaXw5K9sU
โ Anthropic (@AnthropicAI) October 17, 2023
Many public-facing LLMs have been developed with guardrails โ encoded instructions dictating specific behavior โ in place in an attempt to limit unwanted outputs. Anthropicโs Claude and OpenAIโs ChatGPT, for example, typically give users a canned safety response to output requests related to violent or controversial topics.
However, many pundits argue that guardrails and other interventional techniques can serve to remove usersโ agency, as whatโs considered acceptable isnโt always useful, and whatโs considered useful isnโt always acceptable. At the same time, definitions for morality or value-based judgments can vary between cultures, populaces and periods of time.
Related: UK to target potential AI threats at planned November summit
One possible remedy to this is to allow users to dictate value alignment for AI models. Anthropicโs โCollective Constitutional AIโ experiment is an attempt at this โmessy challenge.โ
Anthropic, in collaboration with Polis and Collective Intelligence Project, tapped 1,000 users across diverse demographics and asked them to answer a series of questions via polling.
The challenge centers around allowing users the agency to determine whatโs appropriate without exposing them to inappropriate outputs. This involved soliciting user values and then implementing those ideas into a model thatโs already been trained.
Anthropic uses a method called โConstitutional AIโ to direct its efforts at tuning LLMs for safety and usefulness. Essentially, this involves giving the model a list of rules it must abide by and then training it to implement those rules throughout its process, much like a constitution serves as the core document for governance in many nations.
In the Collective Constitutional AI experiment, Anthropic attempted to integrate group-based feedback into the modelโs constitution. The results, according to a blog post from Anthropic, appear to have been a scientific success in that it illuminated further challenges toward achieving the goal of allowing the users of an LLM product to determine their collective values.
One of the difficulties the team had to overcome was coming up with a novel method for the benchmarking process. As this experiment appears to be the first of its kind, and it relies on Anthropicโs Constitutional AI methodology, there isnโt an established test for comparing base models to those tuned with crowd-sourced values.
Ultimately, it appears as though the model that implemented data resulting from user polling feedback โslightlyโ outperformed the base model in the area of biased outputs.
Per the blog post:
โMore than the resulting model, weโre excited about the process. We believe that this may be one of the first instances in which members of the public have, as a group, intentionally directed the behavior of a large language model. We hope that communities around the world will build on techniques like this to train culturally- and context-specific models that serve their needs.โ
