Robots, fever dreams, and artworks with escape fantasies: for more than ten years, the Harvard Art Museums have been asking what really happens when we trust AI to look at our art. What began as a practical effort to improve search and discovery by augmenting curatorial descriptions with machine‑generated synthetic metadata has grown into a long‑term experiment in co‑creating knowledge with machines.
Over that time, we’ve processed more than 500,000 images (depicting artworks, archival material, and exhibitions and galleries) and amassed over 120 million synthetic annotations to enrich our collections catalogue. Today, new images entering our systems are automatically queued for processing by a “panel” of commercial services: Amazon Nova and Rekognition, Clarifai, Imagga, Google Gemini and Vision, Microsoft Cognitive Services, OpenAI GPT, Anthropic Claude, Meta Llama, Mistral, Qwen, and others. Together, these models tag features, write descriptions, identify colors, read text, detect faces, and more. Their outputs are compared, combined, and surfaced through our public API (hvrd.art/api) and AI website (ai.harvardartmuseums.org) alongside human‑written cataloguing. This processing is not a temporary experiment, but part of how we work. It’s baked into our data pipeline and designed so that new models can be added over time, recognizing that describing collections is a forever project. Opinions, interpretations, and experiences of art are always in flux, for humans and machines alike.
Along the way, we’ve learned that the real questions are not just about accuracy. What happens when humans and machines look at art together? How much does “accuracy” matter for material that is inherently subjective? How can we embrace the inconsistencies, hallucinations, and disagreements between models (and between models and people) to broaden how collections are described and experienced?
This solo session will share concrete examples from over a decade of practice, including short collective looking exercises where attendees attempt to reconcile their own responses to artworks with those of AIs. We’ll explore how this work has changed our thinking about trust, authorship, and longevity in collections data, and how we are designing systems where AI is a long‑lived, transparent tool in the hands of people, rather than a faceless, monolithic black-box of implied authority.
Speakers
Director of Digital Infrastructure and Emerging Technology, Harvard Art Museums
Jeff Steward is Director of Digital Infrastructure and Emerging Technology at the Harvard Art Museums, where they’ve spent more than 18 years building long lived digital systems and short lived experiments. They oversee APIs, data pipelines, and AI projects amongst other things...
Read More →
Friday October 23, 2026 12:10pm - 12:30pm
PDT
512 Willapa