Methodology: being protocol

Being protocol

The third set of experiments I used to explore my research questions were focused around “being protocol”, making protocol the work itself rather than the tool or even the inevitable centre. My aim was to approach jpeg as a scopic apparatus, not merely a component in a camera or screen-grab apparatus. Because the nature of jpeg was a withdrawal from view and a process of becoming and perishing, I designed the apparatus to focus on that process. Although I knew I could no more capture jpeg’s working than a particle accelerator could capture fundamental particles, I could trace its moment of becoming and perishing and, by focusing on the edges of that process, imag(in)e that process. My aim was to build an apparatus where one could see where jpeg was not and so be left with the hole, the moment where it was.

I built my apparatus around the idea of the “digital imaging pipeline”, a conception of how digital imaging works that engineers use to design and produce hardware and software. This pipeline draws an admittedly linear and hermetically sealed picture of light photons hitting a CCD sensor, generating (or being turned into) electronic signals which via software and software protocols are encoded as image files that are written to a memory card and then transferred to other storage and access devices (for a fuller technical account, see the Digital Imaging Pipeline appendix).

The apparatus (so named to locate it within an assemblage of past, present and future scopic and imag(in)ing devices that my media archaeology had looked to trace) is best thought of as itself enfolded. It is tempting to use the metaphor of an onion but to do so would perhaps be to invite conceptions of depth, essence and foundation. It is an enfolded topology of software and hardware[ref]This must be conceived of as material. My digital imag(in)ing apparatus is deeply material not only in terms of the physical existence of the  digital camera, a WiFi memory card, a router and a ‘computer’, but also in terms of the software and protocols which are material presences in the world, enfolded in alliances, subjects and objects of legal and corporate struggles as well as actant players in networks.[/ref].

Like all digital cameras, my Canon S95 has an “imaging processing engine”, software that gathers the luminance and chrominance information from the individual pixels and uses it to compute/interpolate the correct colour and brightness values for each pixel on the basis of neighbouring pixels using a demosaicing algorithm. Further in-camera software engages in noise reduction, image scaling, gamma correction, image enhancement, colorspace conversion and chroma subsampling. This process is engineered to deliver information that matches, as closely as possible, what the engineers have decided is ‘quality’ information – the best flesh tones, the right degree of sharpness and contrast range. Here data is enfolded into human judgements and aesthetics. There is no one objective image processing decision. This is a difference engine. And then, as part of what is known as the imaging (and I would argue imagining) pipeline, there is compression. The crunching of that data ready to be saved to the camera’s quaintly named memory card. Here the jpeg protocol, enfolded within the “imaging processing engine” compresses the data, writes information into the data stream and create a jpeg/JFIF file ready to be written to the card. At the same time the processing engine parses data from the CCD directly to the card as a RAW file[ref]Technically the data is not passed unprocessed. This is not binary signal that is written to the card but image information from the CCD. The difference with a RAW file is that it includes as much data from the CCD as possible. It is uncompressed and unprocessed. Also, usually the RAW file includes a jpeg preview file included in the data.[/ref].

Because I use an EyeFi card that connects to a WiFi network and directly uploads the image files/imag(in)ings to a server, my apparatus is enfolded in the Wi-Fi network relations that Adrian Mackenzie discusses{Mackenzie 2010}. The jpeg/JFIF file is automatically uploaded via WiFi to a folder on a website. The image/imagining is enfolded in the social stream and potentially immediately replicated and distributed as it is copied, shared, downloaded and cached.

The final component of the digital imag(in)ing apparatus is the viewfinder, the interface or window through which a user sees and interacts with my imag(in)ings. This again is an enfolded software/hardware device a browser window on a phone, a tablet or a PC. The viewfinder that the user looks through in my apparatus (paralleling the stereo viewer or the mashup frame) is a scopic device. What unites all these views is that they show the limits, the edges of imag(in)ing. They show the hole where jpeg’s becoming and perishing is. The show the traces of its working and nature by showing the gap where it should be. The show its presence by its absence.

When a user looks through the viewfinder, they see a list of digital files some RAW-encoded, some jpeg-encoded. This is a directory listing orf the uploaded files (images of the rag ’n refuse around 2012, of a screen with the slideflow running, of a 2012 poster on a bus shelter… whatever). This is a listing of imag(in)ings, some enabled by jpeg… some not. A viewer can click on any file to view the imag(in)ing… or not. Because if she tries to open/view an image that has been enabled by jpeg’s becoming and perishing as it encodes light as data, the image will open in the browser. The imag(in)ing will be render visible. If she clicks on that image’s sister RAW file (a software encoded record of exactly the same light), the image will not render. The browser will not “recognise” the format. Usually it will opt out of the visualising pipeline and offer to save the file. The imag(in)ing is unvisible.

My digital imag(in)ing device not show protocol. It cannot capture its becoming and perishings. What it can do is show the gap where it works, its absence (in RAW imag(in)ing) as a way of highlighting its process nature. It is only when presented with its absence in a RAW file that is an unvisible image, that its nature becomes apparent. What is does show is the alliances within which protocol works and through which it derives its power. As a jpeg/JFIF comes into view in a browser or on the camera screen or in the invitation from Flickr, Facebook or Twitter to upload, so jpeg’s fundamental enfolding with web 2.0 businesses and imaging industries becomes clear.

I chose to add one more element to the apparatus to disrupt those enfoldings and to attempt to take the apparatus off-Web while remaining networked. As a website directory, the files uploaded from the apparatus could be indexed and searched by software spiders from Google etc. Each imag(in)ing (RAW and jpeg/JFIF) could be catalogued, cached, archived and enfolded into data-mining businesses[ref]This issue is discussed in the Digital Detritus chapter.[/ref]. I added a script to my server that rewrote the name of the jpeg/JFIF files at regular intervals. This meant firstly that there was a chance that someone clicking on a jpeg/JFIF filename would not see the imag(in)ing rendered because the link between name and visible data file had been broken in between the rendering of the directory listing and the request for the file. It also meant that any attempt to pull my visible imag(in)ings into archives, search databases or indexes or datamines would fail as the search result/database entry would point toward a non-existint imag(in)ing… unless of course those search indexes and datamines worked with RAW, which on the whole they did not.

The digital imag(in)ing apparatus was not ‘beyond protocol’ but it was also more than just using it. In that the apparatus was the process, it was protocol. It was an an instantiation of protocol’s becoming, perishing and enfolding, just as it was an instantiation of processual and object-oriented theory. As with all my experiments this practice was run through with, and formed a space for the instantiation of theory. The practices of thinking through my “digital imag(in)ing apparatus” as well as constructing and using it, was theoretically informed. This does not mean that theory acted as some sort of background to this experiment or even as some sort of guiding principle as I worked on the design or use. Rather it was enfolded in the process/practice and the apparatus served as an instantiation of object-oriented and processual philosophy insofar as to use it was to set in motion processes of becoming and perishing and actant-alliances. The emergence of issues around search indexes, archives, scopic datamining and their attendant issues of biopower and governmentality came from the clash of practice and theory. It was as the apparatus was designed around process, and the the practices of becoming and perishing with which it worked provided a moment of theory becoming practice, that these alliances and issues emerged.

To work with this apparatus, both in terms of designing it and using it, was to bring together the experience of imag(in)ing using protocol (with its attendant network effects, affects and aesthetics) and imag(in)ing outside protocol. What it crucially brought in was a focus on the continuous tense, imag(in)ing. The practice-research experience of imaging using this ‘being protocol’ apparatus was one of focusing on process. This was not a matter of images or even the flow of images but a matter of imag(in)ing. The “digital imag(in)ing apparatus” is best thought of as an event. This is not a ‘performance’ but a moment of becoming in a Whiteheadian sense. Running my “digital imag(in)ing apparatus”, or anyone else’s for that matter, is a moment of becoming, an instantiation of process (including jpeg as a process-actant) that constitutes the scopic regime in a particular configuration. If one were to see the “digital imag(in)ing apparatus” as a thing, a product, then this process would be collapsed, the alliances flattened and the network stabilised. Protocol could withdraw from view and the alliances and processes (in relation to Google, Facebook, Apple et al) collapsed into a foundationalist or essentialist map. Where events, in their becoming and perishing, their process and failure, place the emphasis on heterogeneity, complexity and connection, products (even practice-products) stress homogeneity and stability.

I could have looked to explore the protocol alliances, topological relations of governmentality within search and archiving through theory alone. A political-economic critique of information space and media ecologies could certainly have mapped Google algorithms onto a structural critique of cyber-capitalism and data globalisation. As I discuss in the Literature chapter, software studies has provided just such mappings in its accounts of software packages and even protocols. Similarly I could have used my philosophical framework as a way of accounting for those process of subjectification, alliances and power relations. What both such theoretical approaches would have missed or at least marginalised would have been the experience of protocol. As Adrian Mackenzie argues using William James, the technosocial assemblage needs to be addressed as experience. The process of wirelessness or digital imag(in)ing is lived. It is in movement. It is process at many scales from jpeg encoding through the digital imaging pipeline’s hardware/software assemblages to the social space of image sharing and consumption, search and archiving. What a practice-research approach to that analytical argument does that a purely theoretical account cannot, is to firstly to be open to the presence of theory within any practice and practice within any theory. I could not build or use the apparatus outside of theory. The apparatus was – as an object and in its operation – an instantiation of that theory.

Secondly, the colliding of the practice of protocol with a theory of protocol “in camera”, through the experience of pressing the RAW/jpeg button and clicking on the RAW and jpeg/JFIF file links, make that scopic and governmental enfolding part of the assemblage. When I or anyone else use the apparatus by taking a photograph or trying to view one, theory and governmentality become part of the experience. I become part of Facebook’s walled gardens or Google’s databases. I become part of a social imag(in)ing practice that subjectifies and objectifies. I become enfolded in alliances. They are not only more apparent, they are coterminous, co-present to me as imager. Where a purely theoretical imager can avoid that experience and enfolding a practice-theory one cannot.