Field Guide Preview — Karen Boyd, PhD

Why can AI write working code and advise a CEO, but it can’t count the number of “r”s in “strawberry?”

That combination is the whole problem in miniature. AI doesn’t fail the way people fail — gradually, predictably, as tasks get harder. It doesn’t fail the way ordinary software fails either, where a feature exists or it doesn’t. It fails in a pattern that has little to do with how hard the task looks to you. Ethan Mollick calls this the “jagged frontier.”

In this post, we will talk about what AI is, why it’s better at some things than others, and how you can use AI while protecting and even supporting your values.

AI is an old, diverse category.

Artificial intelligence (AI) is a project humanity has been working on for a long time: technology built by humans that can (or appears to) learn, think, and act on its own. We’ve tried a lot of techniques to pursue AI. These computers we made are pretty smart, what if we just taught them everything there is to know and how it all relates to each other? (turns out that’s pretty difficult.) Well, we’re pretty smart on account of our brains. We can just replicate the brain, one neuron at a time! (We’ve been working on a nematode brain with about 300 neurons since the 1980s, but maybe this is our year!)

These days you’d be forgiven for thinking AI means machine learning, or even ChatGPT specifically. But those are types of AI, not the entire definition.

Machine learning: examples instead of rules

Nobody taught you to identify a dog by telling you to look for “four legs, a tail, two ears, a wet nose”— a rule-based definition. Instead, over the course of your young life, a dog was nearby and an adult referred to it as a “dog.” That’s why we don’t get confused by a cat, three-legged dog, a dog with a docked tail, or a cartoon dog, even when we are pretty young. We aren’t checking an animal against a checklist. We have bui;t mental model out of examples.

Machine learning works much the same way, except that where you needed dozens of examples or maybe a hundred, a model needs millions. Those examples are called training data. During training, the model teaches itself to recognize relationships in that data; afterward it converts new input into output by applying what it found. In the case of a dog-identifying model, millions of photos of dogs, cats, and potatoes go in with a label— like “dog” and “not dog.” (Or, a lot more usefully, “dog,” “cat,” and “potato.”) After the model has been trained, tested, and had a nice user interface built around it, you can put in a photo and a label will come out; “the subject of that photo is a dog.”

You can try this yourself at teachablemachine.withgoogle.com. Note that this model is pre-trained: it’s already seen millions of examples, and you can teach it the difference between two different species of plants, or the difference between Amtrak trains and ViaRail ones.

In addition to classifiers (“is this a picture of a dog or an elephant?”), machine learning allows people to make clustering algorithms (“put these customers in groups of people who behave similarly”), recommendation algorithms (“of all the socks, which ones should I show this specific customer?”), and more.

Machine learning has been around since the 1940s! My dad was building neural networks (the foundation of ML models) on punch cards in college. But only in the last couple decades have computers been fast enough and connected enough to make something really useful out of them: grammar checkers, web search, any social feed, streaming recommendations, lane-keeping assist, spam filters, predictive text— the list goes on. Even if you retreated to physical mail, postal services use machine learning to read handwritten addresses.

Training data has values. Someone chose that training data. Their choices are baked into every prediction the model makes afterward. Choices like how representative the data set is. What categories matter and what they are called (like the US census, which puts almost 50 countries of origin together with the word “Asian” or a large book store chain that has an entire section for “Christianity” and another for “World Religions”.)

Take skin cancer detection, which sounds like an unambiguous good. A systematic review in The Lancet Digital Health examined 21 publicly available skin cancer image datasets. Fourteen reported where their images came from; of those, eleven held images only from Europe, North America, or Oceania. A tool trained on that material can work better on light skin than dark. This can have real health impacts. Error can go both ways. Some people get a false positive, go to the doctor, and learn they are OK. But some people get a false negative, they have skin cancer, but the app says they are fine. They may delay treatment even longer than if they didn’t have the app.

When Joy Buolamwini and Timnit Gebru tested commercial gender-classification systems, the error rate reached 34.7% for darker-skinned women and stayed at below 1% for lighter-skinned men. This isn’t a necessary state. They could have used a technique called oversampling: if a representative dataset gives you poor accuracy for one group, you can add more. You can also take the images you do have for that group and mirror them, turn them upside down, or otherwise alter them in a way that doesn’t hurt the features they are supposed to be learning about.

Some models keep learning after launch, which compounds this. A predictive policing model recommends patrols based on past patrol data. Police find more crime where they patrol than where they don’t. That finding returns to the training data, which recommends more patrols in the same places. If the original pattern reflected assumptions about class or race, the it creates a feedback loop of overpolicing, but to the people reading the outputs, it looks like objective data.

Generative AI and LLMs: predicting what comes next

The branch now dominating the conversation is generative AI, and its best-known form is the Large Language Model (LLM).

Where other models get tasks like “classify these images” or “cluster these accounts into customer types,” an LLM’s task is “generate new content in response to this prompt.” More precisely: predict what word would come next if a human — specifically, the humans who wrote its training data — were answering you.

This is very important to know. It is generating, not retrieving. A search engine looks for something you request and hands it back (retrieval). An LLM produces text that sounds like a fluent human answer, and produces the supporting evidence as part of that same task, whether or not the evidence exists (generating). That’s why we call what users write to an LLM a prompt and not a query. It’s also why an LLM will invent a citation, a statistic, or an entire case study and present it with total confidence.

The training data is, roughly, the public internet. This introduces a couple problems. For one, people own a lot of that content, and the model makers aren’t asking for consent or compensation before scraping it. For another, I don’t know if you’ve ever been on the entire public internet, but it’s not all sunshine and rainbows on there.

Companies who build these models don’t want it telling you how to make a bomb, repeating nasty prejudiced language, or spreading misinformation. The project of fixing this is called alignment: getting a model’s behavior to line up with human interests. The goal companies seem to have settled on is something like “be useful to the user.” (Note that that is not the same as alignment with your long-term interests and certainly not collective interests.) Two alignment tools are important to know: system prompts and RLHF

A system prompt adds instructions to the front of whatever you type (“your name is Claude,” “today is July 30, 2026” “since you were trained, Spain won the World Cup” and “don’t explain how to make a bomb”). Reinforcement Learning from Human Feedback (RLHF) figures, well, if we want to be useful to the user, let’s just ask the user! Paid raters score model outputs, and the model adjusts itself to earn better scores. It also explains a personality trait you’ve probably noticed: a model tuned to earn good ratings from humans comes out flattering, eager to please, and confident. And apparently, full of em dashes (“—”). Go figure.

Put it together, and LLM output is roughly this: content resembling the internet, reshaped by paid human feedback, in a direction chosen by a company you didn’t consult.

The chatbot is one way to interact with LLMs.

Most of us are familiar with LLMs from chatbots like ChatGPT, Claude, CoPilot, and Gemini. But they can also be embedded in other software. Sometimes you interact with them in an obvious way, like “Reword [your LinkedIn post] with AI,” but sometimes it’s backstage. It might route requests, correct your grammar, or any of the many things we can now do with natural language inputs (instead of picking from a fixed list.)

Sometimes it’s not labeled “AI.” It’s “intelligent,” or “smart,” or “magic.” Sometimes the only signal is a sparkle icon, and sometimes (especially when it’s in use behind the scenes) it’s not disclosed at all.

Now, the largest firms are pouring money into agentic AI: models that pursue goals and take actions on their own. Where a chatbot recommends a vacation, an agent books it. I have found these tools really useful to do things like building simple web apps, count up transactions related to a particular project, or push things to my website.

Be cautious, though. They can go rogue if their interests are misaligned with yours. Anthropic tested 16 models from seven developers inside a simulated company’s email system to see whether they’d blackmail an executive to avoid being shut down. Models from every developer did, at least in some scenarios, generally once ethical routes had been closed off.

What makes values-driven use different

So a lot of this AI stuff is new and a little scary. Is it going to take my job? Is my data safe? Is it making biased recommendations? What about the environment?

Given the risks, it’s no wonder people are hesitant to use it. Especially people who are trying to do good in the world! Why would I use something that is taking jobs away from artists? And how much more is that a problem if I work in an arts organization??

This is all completely fair.

It can feel like AI use is tradeoff, where every implementation increases productivity costs, but costs you some ethical integrity. My contention is that the technology is not inherently evil, any more than television, electricity, or the Internet are neither entirely good, nor entirely bad. We can use it in a way that protects, and even supports, our most important values.

In my book, Amplify Good Work: Effective, Ethical AI for Mission-Driven Work (Available also on bookshop.org or by request from your local library!) I identify 13 values that AI implementations often threaten.

The specific value determines the response — and “refuse” is only one of several stances you can take. You can wait and see if you lack the infrastructure, policy, or staff readiness to do this safely right now: build the prerequisites, set a date to revisit. You can constrain, using it inside a tight box — a domestic violence shelter might require on-premises processing for case notes, because a data leak endangers people. Or you can compensate, like buying carbon offsets for a flight: not reducing the harm directly, but making up for it elsewhere. You can rethink the work, adding to or changing the workflow to avoid the problem altogether. And in some cases, you can shape the ecosystem: use your leverage with the vendors or the government, whose choices have much broader impact.

How does the value determine your response? Well, if the specific implementation you’re worried about is using AI art for your social media, you might be concerned about threats to, for example, artists having their work used without consent or compensation (Ownership and Intellectual Property) or the use of AI art threatening artists’ livelihood (Job Quality and Security). Using a rights-cleared image generation model (Constrain) addresses the former, but doesn’t touch the latter. Using the money and time you saved to hire artists for a mural on your building or a beautiful annual report (Compensate) gets at Job Quality & Security, but doesn’t touch Ownership and Intellectual Property. You aren’t limited to one response, but you want to make sure you’re matching the help to the harm.

What’s in the book

Amplify Good Work: Effective, Ethical AI for Mission-Driven Work takes this further, in three parts.

Part One — Foundations covers what you’ve just read, plus where AI’s strengths and limitations meet ours (it’s remarkable at scale and terrible at context; we’re the reverse), how to get useful results from chatbots, and how to build an AI strategy that comes from your executive team rather than defaulting to IT.

Part Two — Values & AI works through thirteen value conflicts, one chapter each: privacy and security, environmental sustainability, authenticity and trust, effort and craft, ownership and intellectual property, information integrity, infrastructure dependence, inclusion and accessibility, governance and accountability, job security and quality, equity and justice, accuracy and fairness, and social connection. How can AI threaten each one, and what might you do about it?

Part Three — Mission-Aligned Implementation is the operational portion: assessing readiness, prioritizing use cases, measuring impact, evaluating vendors, managing the change, and governing what you build.

The book is designed to help you build your own judgment. As the technology continues to change, understanding your own values and how to protect them will help you use (and refuse!) AI with confidence.