Belay: 23 January 2011

Belay: 23 January 2011 (For other belay points and the idea behind them, see here).

I am looking at a way of accounting for the relationship between the jpeg compression protocol and the new photographic and imaging practices of sharing, publishing, streaming, archiving and remixing online, by understanding jpeg as a ‘scopic apparatus’ – a particular technology of a new distributed ‘scopic regime’.

My aim is to understand protocol as “doing things” in the world, establishing ways of seeing, imaging and imagining (or iamg(in)ing) as I call it, just as other scopic apparatuses from the camera obscura to the most modern medical imaging tools have done. And just as those technologies have been implicated and enfolded in powerful governmental, legal and material relations, so jpeg is also raises issues of intellectual property and copyright, ethics, materiality and affect.

These research questions arise from practice-research, or more correctly the failures at the heart of that practice-research. Through a series of imaging experiments, I have been trying to ‘chase protocol’, to locate it rather than see its effects; to separate it from the image, the signification and the visualisation; to track its relations to corporate and state interests that use it in surveillance, marketing, archiving and social networking businesses. I have failed. I have found the trails of the issues it raises, the traces of its operation and relations but not it. I find dot jpeg not jpeg. Jpeg withdraws from view

This need to account for the jpeg apparatus as something that withdraws from view, that is simultaneously materiality and immaterial, real and virtual led me to object-oriented philosophy as a framework that can provide an account of this protocol-object that not only explains its position but opens up the possibility for its political reconfiguration or counter-protocological struggle.

Object-oriented philosophy takes all objects seriously. From this perspective, protocol is an object doing things in the world, an object-actant. As with all other objects, whether material or immaterial, real or virtual, the protocol-object is folded into relations with other actants in the network (NB not just computer networks). Its power arises from the relations or alliances in which it is folded. Jpeg’s presence in Google and Facebook’s businesses, Microsoft and Apple’s operating systems and Adobe’s software as well as my iPad and my daughter’s phone and social networking relations means that jpeg has become so enfolded and so everyday and transparent that it can be considered as a ‘black box’, a power-full object so firmly established we take its ‘interior’ (those actant-relations) for granted.

This approach allows me conduct an object-oriented media archaeology of my chosen scopic apparatus – to trace its alliances (through experiments that highlight its black-box transparency), its topological position in the current scopic regime. My project is not a discussion of philosophy, whether OOP is a valid ontology, let alone whether it is a good reading of Latour. My discussion is not around the nature of objects, or what counts as an object, but rather what happens when you treat protocol as an object.

An object-oriented approach allows us to approach and map jpeg without recourse to a foundationalist or essentialist position that would see it as the source of scopic effects or power relations. Rather it can be addressed as a scopic apparatus – a device for imaging – that has a specific position, a history and a future.

Dragging philosophy down

Mine is not a PhD about philosophy but it is a philosophical PhD. It is not a discussion of ontology or even a defence or exploration of object-oriented philosophy (OOP). I would also hope it not simply a bland application of a philosophical framework to a media problematic. I will argue that offers a number of concepts and tools that can help to explain and explore the elusiveness and power of protocol but it is important to note that the presence of OOP within this project is not simply as supplier of vocabulary. Rather an ontological philosophy is necessary to this project in order to prevent a slip into a discourse of representation.

Dealing with images, imaging and even imagining, it is tempting to reach for semiotics. Even widening the area of analysis to a media archaeology of the apparatuses enfolded in a particular scopic regime, it is possible to see an account of signification and representation (whether tinged with linguistics, psychoanalysis, feminism or political economy) as a way forward. As Daniel Miller argues in relation to clothing, semiotics can be “as much a limitation as an asset” (Miller 2010: 12). Miller argues that a semiotic approach, one that looks at clothing as representation locates clothing as ‘superficial’, a sign of something more basic and more important. Miller seeks to develop an account of “the minutiae of the intimate” (p 41) through a materialist philosophy he builds from Hegel. For Miller, philosophy is necessary so that he can account for that minutiae and materiality. It is not a parlour game or an overarching ‘theory’ but a building block allowing him to focus on his “stuff” without recourse to a meta-description.

In terms of my own project looking at protocol as a scopic apparatus, it is important I too steer clear of the discourse of representation. As my experiments are showing, protocol is entangled with the the image, jpeg is enfolded with jpeg/jfif – or perhaps to signify it in a different way jpeg is enfolded with dot jpeg. Jpeg withdraws from view. To approach it in terms of how it signifies or look for what is representational work it does is to miss the material relations within which it is enfolded, the particular alliances and power relations that make it such a transparent black box, so overlooked yet so topologically significant. Miller uses Hegel as a way of squaring the universalism-particularity circle through an account of the process of objectification. Miller’s account of Hegelian dialectics has parallels with OOP. In almost a Latourian litany, he says: “We start with the need for a theory of stuff as material culture… that can account for every kind of stuff: bodies, streaming videos, a dream, a city, a sensation, a derivative, an ideology, a landscape, a decay, a philosophy” (p 54). “Ultimately,’ he concludes, “there is no separation of subjects and objects” (p 12). There are of course differences. Harman’s more Heidegegarian approach proposes a fourfold approach to objects rather than a dialectical one but the common phenomenological heritage is clear.

The point however is not to engage in philosophical exegesis of Hegel and Heidegger or even Miller and Harman but rather find tools that can approach the questions that need addressing. What sort of an object is protocol? What does it do in the world? Where does its power as a scopic apparatus arise from? What are the possibilities for reconstituting that power? To address these questions demands historical and political-economic tools and concepts but also philosophical ones if we are to take the (im)material object seriously, to account for its foldings and map its particular and diverse connections.

As Miller says: “The aim is not at all to become a philosopher. The aim of anthropology [or in my case media archaeology] is to take any such pure, clean philosophy and drag it back down to the valley, to the muddy terrain of particularity and diversity” (p 41).

  • Miller, D., 2010, Stuff, Polity Press, Cambridge.

My practice is a failure

Software studies has a long tradition of seeing the value of practice. Lev Manovich says: “it helps to practice what one writes about” (2008: 8). Matthew Fuller as editor of the Software Studies Lexicon makes clear that “one rule of thumb for the production of this book is that the contributors had to be involved in some way in the production of software as well as being engaged in thinking about it in wider terms” (2008: 10). Noah Wardrip-Fruin and  Nick Montfort say: “for deep understanding, actually creating new media projects is essential to grasping their workings and poetics” (2003: xii). Geert Lovink lauds “the artists and critics featured in this book [as] working with the technology itself” (2002: 4). The work celebrated in these accounts are practice artworks such as I/O/D 4:The Web Stalker  (Fuller 2003) or Google-Will-Eat-Itself (Parikka 2010), interventions by code-artist-analysts. Digital objects that deconstruct or at least critique media ecologies and assemblages. They need not be finished or complete – benign viruses such as the Biennale.py (Parikka 2009) are never complete, finished or even stable – the object is its working, but its critical power arises from its work(ing).

My practice on the other hand is a failure. Mine is a practice-research PhD. For me, practice-research is an emergent phenomenon where the “mess” of social realities, assemblages and media ecologies, the complex interrelationships of alliances and the topographical foldings and unfoldings of multidimensional actants appear as the black boxes, the transparent objects under investigation are forced open. This cannot happen just through research. To simply study protocol  – as my experiments show – is simply to study its traces, to chase its presence. To look at protocol through a particular theoretical lens is to stabilise an object that derives its power, position and work from motion, movement and relations. To apply a research paradigm is to choose to analyse light as a wave or a particle when its nature is to be both. The transparent, everyday, seemingly innocuous but very powerful objects are best forced open by pushing them until they break.

It is my failure to separate the jpeg protocol from the JPEG/JFIF image; my failure to build a mashup that mashes jpeg not images; my failure to prise jpeg from its home in software packages; my failure to hold jpeg outside its alliances that highlights its object-actant status, its enfolding in alliances, its topographical infolding, the importance of its instantiations. It is only by failing to see it, to practice it, that its black box begins to crack and the alliances and foldings come to view.

My practice is not photographic nor is it coding. Although I have images and code experiments these are just the traces of the experiments – the laboratory notebooks as it were. The actual practice is the process of chasing protocol that fails and leaves just these photographic, code and mashup traces behind.

  • Fuller, M., 2003, A Means of Mutation: Notes on I/O/D 4: The Web Stalker, in Behind The Blip : Essays On The Culture Of Software, Autonomedia, Brooklyn, NY, USA, pp. 51-68.
  • Fuller, M., 2008, Software Studies : a Lexicon, MIT Press, Cambridge, Mass..
  • Lovink, G., 2002, Uncanny Networks : Dialogues With The Virtual Intelligentsia, MIT Press, Cambridge, Mass..
  • Manovich, L., 2008, Software Takes Command. unpublished ms., 2008, unpublished ed. .
  • Parikka 2009, Archives of Software: Malicious Code and the Aesthesis of Media Accidents , in Parikka & Sampson (eds), The Spam Book: On Viruses, Porn, And Other Anomalies From The Dark Side Of Digital Culture, Hampton Press, Cresskill, N.J., pp. 105-23.
  • Parikka 2010, Ethologies of Software Art: What Can a Digital Body of Code Do? in Zepke & O’Sullivan (eds), Deleuze and Contemporary Art, Edinburgh University Press, Edinburgh, pp. 116-32.
  • Wardrip-Fruin, N. & Montfort, N., 2003, The NewMediaReader, MIT Press, Cambridge, Mass..

Clear to see… or imag(in)e

Just because the jpeg protocol withdraws from sight, that its role is to build alliances within software and between software and imag(in)ing practices as well as between media and web business and strategies, doesn’t mean that it can’t be understood, imag(in)ed perhaps, as an object. Object-oriented philosophy allows us to account for the ontology of that object but it is also possible to approach it from a media-archaeological positions as a scopic (imag(in)ing) apparatus.

Such an approach would locate the protocol’s work of data compression and creation in terms of a long material and discursive history of scopic apparatuses for creating, rendering and viewing/consuming images and imaginings. Here jpeg is looked at alongside the camera obscura, the window, medical and astronomical devices and those pencils of nature apparatuses from the Leica to the iPhone. Each has operated and related to each other in terms of constructing and instantiating scopic regimes and practices. Each, as an apparatus-object-actant, has been enfolded in powerful alliances and translations. Some have become black boxes others have not.

Jpeg can be seen as every bit as much a scopic apparatus as a perspective frame or an iPhone both held at arms length, framing an image, constructing an imaginary and working as an object with other objects, be that Renaissance perspective, philosophy and religion or the marketing of an ‘App Store’ – both of course sites of struggle, the former in relations of church and state, the latter in terms of Microsoft’s challenge to Apple’s copyright claims. No object nor scopic apparatus is ever outside alliances framed in terms of struggle.

Somewhere in the code of Windows 7 there is an instantiation of the jpeg protocol. Windows 7’s “snipping tool” allows a user to grab a section of the screen and save it as a jpeg-encoded image file. This is imaging, a form of photography even. Regardless of whether the screen that is ‘grabbed’ (an interesting metaphor itself) includes images (copyrighted or not) or text or indeed just a plain colour renedered by the operation system, jpeg images. It creates an image. And by doing so, it imagines. Just as Alberti’s window or an optical coherent tomographic (OCT) imaging apparatus create an image  (whether that is preserved or not), so the jpeg protocol takes information and constructs  an image.

You cannot hold jpeg (as you would a camera). You cannot even separate it from the other software actants with which it works (as you can separate the camera obscura from the light sensitive materials that could fix the image). It will always withdraw and remain parasitic on its alliances but without it, there would be fewer images and imag(in)ings.

Even if one were not to see jpeg as an object (in the OOP sense) but rather as a language or even a practice, its status, position and power as a scopic, imag(in)ing apparatus expanding and also contracting what is, can be and should be imag(in)ed is clear to see… or maybe that’s imag(in)e.

An absent presence

Jpeg operates at different scales. As an object it has an ontological status which object-oriented approaches see as equivalent to any other object in a network but which other approaches could address as material, powerful, ideological, structural, textual or discursive. Obviously an object oriented approach would not rule out these frameworks, indeed it argues that it’s flat ontology offers a way into the power of objects that other approaches do not.

My experiments have highlighted that the protocol withdraws from view at some level, leaving only its traces behind. To take that and simply say that the jpeg protocol is a slippery thing, a virtual object or a metaphysical puzzle is not only to engage in philosophical sophistry but to explore only one scale of operation.

Jpeg as a scopic apparatus, a technology of imag(in)ing, is enfolded at different scales within the scopic regime. Its traces are visible across the distributed web but it is instantiated in the real world if only at the scale of specifications, RFCs and code instructions. It ‘exists’ as documents online and in print within the Joint Photographic Experts Group, doubtless in Google, Yahoo and Adobe engineers’ libraries. It exists within the software those engineers produce. Somewhere ‘inside’ the software in my Olympus DSLR that allows me to shoot a RAW and jpeg simultaneously, there is the jpeg protocol. Some lines of code instantiate that protocol. It is not instantiated in the same way in my iPhone but somewhere in the code of the Camera app, is the jpeg protocol.

From an object-oriented perspective these ‘presences’ can be seen as ‘alliances’, the protocol-object coming up against the institution object, the software object, the corporate object, the camera object. Other perspectives may draw that presence in terms of scales within an assemblage. What is important is that the absent presence is not simply a philosophical issue, it is deeply real and material as well as virtual and immaterial. It’s instantiation at these scales, its realisation as a scopic apparatus, is real. It is implicated in software that structures representation and surveillance. It is enfolded in economic shifts that structure social media culture and practices. Whether it is protocol or the protocol traces that have those effects is a moot point. The issue is that those effects and affects are real and protocol is a way into tracing, tracking and problematising them.

Jpeg is an absent presence but It is not a simulacrum. It is present at different scales. Although it withdraws from view, becomes a black box, it has a presence as well as effects. It is an object in an ontological but also a practical sense. Chasing protocol through the experiments in the Olympic Arcades Laboratory is not a metaphysical party game, it is exploring that presence as well as the absence – the black box that allows, enables, empowers and structures Google searches, iPhone citizen media, Facebook archiving and governmental biopower.

Tracing an image: chasing protocol

It began, as so many media journeys do, with a search. I Googled “2012 olympics factory” and clicked on the image tab to be presented with a mosaic of imag(in)ings, some from Flickr, some from Blogs and some from news organisations. I screengrabbed the page (which was compressed and encoded as an image file by the jpeg protocol), because by the time you click on that link to perform the same search, the stream of imag(in)ings will have flowed past.

I clicked on one image, the most ‘obvious’ sign, a worker in a Far Eastern factory, bent in the dark over a sewing machine. The connotations of ‘sweatshop’ were obvious, notably when colliding with other significations around legacy, regeneration, health and fitness within the mosaic/montage. Before I had even clicked on the image, by simply ‘hovering’ over the space, the image expanded  The click on the image expanded slightly to include anchoring text: “ShoeFACTORY_102809‑450×300.jpg, 450 × 300 – As the Olympic Games will be held on London soil for the year 2012, … cyclodelic.wordpress.com”. Of course that file name and dimensions referred to the ‘original’ (sic) file not to this thumbnail that was now cached on my computer. Clicking, the image expanded further and, still on a Google page the image was overlaid on the website on which it appeared alongside text under the Google logo: “Website for this image. cyclodelic.wordpress.com. Full-size image – Same size. Size: 450 × 300. Type: 29KB JPG. This image may be subject to copyright.” Closing the floating image, one leaves Google’s servers and is on the cyclodelic blog on WordPress.com (a blog linked to a a cycling clothes manufacturer in East London (http://cyclodelic.myshopify.com/).)

It turns out that the image is not on WordPress servers, let alone cyclodelic’s (although it may well be ‘in’ caches on those servers). It is actually being pulled in from an external site. In These Times describes itself as: “a nonprofit and independent newsmagazine committed to political and economic democracy and opposed to the dominance of transnational corporations and the tyranny of marketplace values over human values” (http://www.inthesetimes.com/about/). It is unclear whether cyclodelic found this image on their site through an image search, through a search for “transnational corporations and the tyranny of marketplace values” or any other search. The image appears on an article headlined: “Corporate Sweatshop Apologists Star at Anti-Slavery Conference” with the caption: “A worker labors at a shoe factory in December 2006 in Chongqing Municipality, China.   (Photo by China Photos/Getty Images)”. (http://www.inthesetimes.com/working/entry/5106/corporate_apologists_star_at_anti-slavery_conference/). The magazine has clearly paid Getty Images for a digital copy of the image, encoded as a jpeg so it will work with its website and my browser will display it. Tracing the image to Getty’s servers, one finds the image, taken in 2006, with the title: “Chinese Shoemakers To File Suit Against EU” and the caption: “CHONGQING, CHINA – DECEMBER 12: (CHINA OUT) A worker labors at a shoe factory on December 12, 2006 in Chongqing Municipality, China. Chinese shoemakers plan to file a suit against the European Union (EU) contesting the legal and factual basis of the anti-dumping measures the EU has just taken against them. According to EU figures, China exported 1.25 billion pairs of shoes to Europe in 2005. (Photo by China Photos/Getty Images)” (http://www.gettyimages.co.uk/detail/72820938/Getty-Images-News#).

A copy of that image now ‘exists’ in the cache on my computer (and if you have clicked on the link, on yours).  Interestingly Getty images charges for 3 months Web use. How that fits with PC and serach engines caches let alone embedding is another jpeg-stimulated issue. The question arises whether I, you, cyclodelic or even Google have crossed or maybe reconfigured the copyright of that image or even its ethical position as it appears in the scopic montage/dialectical images that started this journey.

Screengrabbing the image creates a new image, a copy encoded as jpeg but this is different than saving the image. The image that is saved may for instance be at a higher resolution than a screengrab creates. It also includes all the jpeg-encoded metadata that the photographer, his/her camera, Getty images or the site owner wrote into the data. Downloading that image (as opposed to caching it – a human/software rather than just a software act) gives me a copy of the jpeg-encoded image. Decoding the data in the image (the traces of the jpeg protocol’s work (using JPEGsnoop) it is possible to find information about the data downloaded:

The file is a  JPEG JFIF, a “file format created by the Independent JPEG Group (IJG) for the transport of single JPEG-compressed images” (http://www.w3.org/Graphics/JPEG/) compressed at a quality setting of 90 a rating of 14.35:1. JPEGsnoop also tries to match the compression signature encoded into the data. It conclude that the “Image is processed/edited” but cannot determine which software was used. It also says there is a “susbsample match” between the compression signature (the traces of the jpeg-protocol’s work as the CCD data is encoded as an image) of this image and that of Treo and a Blackberry. It is of course impossible to say if this image was taken on a mobile phone but bearing in mind it is available as a 17.2 MB – 3000 x 2000 px (25.40 x 16.93 cm) – 300 dpi file, it is unlikely. What is interesting is that the operations of the jpeg protocol in encoding information into a JPEG/JFIF file can leave such questions open.

JPEGsnoop offers a wealth of other data JPEGsnoop reports a wealth of information, including: quantization table matrix (chrominance and luminance), chroma subsampling, Huffman tables, EXIF metadata, Makernotes, RGB histograms, etc. (often referred to as Digital Image Ballistics/Forensics). This information is the trace of protocol’s work in getting the digital data from the CCD into a format that allows an image to be rendered and made usable, sharable, mashable, downloadable, embeddable. What is important is that all this data are the traces of the operations of the jpeg-protocol. This journey, through issues of engineering, legality, sociality, campaigning, politics, representation we have been dealing with the traces of protocol, its effects and affects in the world, it relations with other structures, institutions, discourses and practices – actants in the network. The connotations that drew me to the image, its role in cyclodelic’s marketing, branding and business, In These Times’ campaigns and business and indeed my PhD are all set in motion by the operations of protocol. Jpeg’s work as an actant is enfolded in those alliances. The fact that those relations have been set in motion and we have been able to trace them is down to protocol, but what we have seen, traced and tracked is not protocol. It has withdrawn from view.

Footnote:

I emailed Getty Images:

“As part of my work I am exploring mashups and what happens when ‘official’ 2012 images sit alongside other images of 2012, the sportswear industry etc. I would like to use this photo  as part of a new image/montage that I would include in the project Flickr album (http://www.flickr.com/photos/content2bdifferent/sets/72157622279154599/)and geolocate within the Olympic site. This would mean that anyone searching for 2012 images or images taken in the area of the stadium would find the picture. I would change the title and description of the image to reflect the photographer, his copyright and the site.

I am aware that this is an unusual use – part research, part artwork. I am more than happy to answer any questions or discuss further with the agency or the photographer.

More details about my project can be found on my website: olympicaracdes.theinternationale.com”