Computational Formalism: Art History and Machine Learning, by Amanda Wasielewski

Le résumé fourni par la source

Little attention has yet been paid to how computers “view” art despite the growing cultural importance of machine vision technologies. Putting in conversation the work of computer scientists and art historians, Amanda Wasielewski, a digital humanist and art historian at Uppsala University, offers a potent critique of what she calls “computational formalism.” Her eponymously titled book argues that computer vision replicates formalist processes of classification and taxonomy from the discipline of art history in unexpected ways —through the predictive pattern-finding of various techniques used to design and implement machine vision models. Algorithmic logic, she argues, searches for common patterns that are often treated as if they were timeless and universally accepted features of all artworks. Art historians have argued that such a formalism elides the important work of historicization and contextualization. The development of computational formalism, argues Wasielewski, is therefore a step backward, an argument she unpacks in the following chapters.Chapter 1 outlines how computer vision systems, designed specifically for the analysis of art and built using data sets of disproportionately Western artworks, reify Western ways of seeing. Wasielewski's audit of Art Historian—an art classification system for identifying content in artworks to support information retrieval—focuses on the underlying foundations of digitization and metadata. Digitization determines which artworks become data for computer vision and, at present, follows the selections of national museums and archives with long histories of prioritizing Western art. Because these powerful institutions have had the resources to make digital surrogates available, they have become key sources of data and metadata for resources such as WikiArt that shape the computational analysis of art. Digitization can introduce additional distortions such as shifts in color from the original. Computer analytics generate metadata by, for example, turning features such as shape and lines into genre descriptions such as “Abstract Art.” These automated labels sometimes conflate formal style with historical categories. The reduction of art history to decontextualized forms that are legible to machines is at the core of Wasielewski's concerns regarding computation. She argues for greater care when deciding which categories are built into computer vision systems and caution toward models that claim to provide context. While Wasielewski does not elaborate what her reforms would enable computer systems to teach us, the chapter compellingly argues for the importance of questioning and contextualizing the (meta)data at the core of computer vision and the quantification of art.Chapter 2 turns from efforts to generalize style across artists and time to the role of deep learning (a kind of machine learning that involves multilayered software architectures) in the identification of artworks and the attribution of artworks to specific artists. Art authentication is a lucrative practice. Deep learning is now a part of the authentication repertoire. Authenticators leverage the social power of computation, which entails assuming the stability of an artist's oeuvre, and does not account for how an artist's work might change over time. Wasielewski worries that the rise of deep learning in authentication is undoing the work of the visual turn in art history to move beyond a narrow canon toward a wider range of art, including vernacular and “low” cultures. She offers the term “Deep Connoisseurship” to name the process of using deep learning for artwork attribution. The last third of the chapter turns to the creation of art through generative artificial intelligence. Wasielewski argues that generative AI produces images that draw from the modeling of data across different time periods, which she suggests we could think of as “art forgeries,” a provocative reframing warranting further debate. Overall, the chapter reveals the fascinating ways that deep learning is being used in art markets and as a creative practice, while flagging dangers not to be taken lightly. In contrast to chapter 1, this chapter briefly explores possible intersections between art historians and computer scientists, leaving the reader with future directions for computational analysis in art history.In the third and concluding chapter, Wasielewski calls for interdisciplinary collaboration, for computer scientists and art historians to partner when working with art as data. If we start from the position that all data brings a perspective, then understanding that perspective means making sure the proper experts and stakeholders work together. Given the absence of such alliances, Wasielewski's book repeatedly criticizes computer scientists who cavalierly wade into the study of art without engaging art historical research. We must bridge the divide between the humanities, social sciences, and STEM, she suggests. By using the lab as a metaphor, we might find an invitation to experiment and build across institutional divisions. Such collaborative experimentation and labs are already prominent in the digital humanities. Although Wasielewski does not offer specific examples of successful partnerships or other potential directions to guide next steps, she does convince her reader that both art history and computer science would benefit from the embrace of interdisciplinarity. Art historians need to learn more about computational methods so as to attune their humanistic inquiries to the challenges of computational formalism.Should we be engaging in computational formalism at all? This question hovered in the background as I read. The close, critical readings throughout the chapters often suggest that there is little of use that computation can provide. Add the return of formalism in art history and computational formalism seems like a regression. Considered as a whole, the book invites a cautious approach. As computer vision and the study of cultural data accelerate at a breathtaking speed, those involved in computational analysis of artworks need to be careful not to rely on computational formalism as their sole form of evidence. We must pair formalism with historical, social, and political context.The challenge ahead is, if possible, to design machine vision approaches that can account for the nuance and precision that animates the study of art history today—a topic that will resonate with those following debates over how machine vision systems label people through, for example, facial recognition or models that supposedly interpret human emotions. Or is it more likely that “computational formalism” will remain a technique that finds dominant patterns at the expense of aesthetic nuance? If collaboration with humanists helps data scientists and machine vision researchers to develop systems sensitive to historical particularity, both are likely to benefit from the challenges and rewarding opportunities.Scholars such as Beatrice Joyeux-Prunel and Leo Impett are already practicing a kind of computational formalism that is deeply steeped in contextual knowledge in being careful and specific with the results of machine vision. They add historical context through archival research, use types of machine vision such as pose detection (which supports research on composition), and draw on spatial analysis. The key here is their refusal to rely exclusively on the results of machine vision. This raises a question of terminology. Perhaps what these scholars are doing should not be reduced to a term so pejorative as Wasielewski's computational formalism?If so, her book is opening space for another term, a term that captures the methodological interventions of computer vision for the nonreductive study of images. Zooming out further to methodological debates in areas such as media studies and computational literary studies could offer an interdisciplinary framework for grappling with

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.