ABSTRACT. Big Data has recently become the buzzword across many disciplines. As an alternative, we propose a Big Video manifesto that moves away from quantitative big data analytics in order to develop an enhanced infrastructure and workflow for qualitative video analysis with innovation in four key areas: 1) capture, storage, archiving and access of digital video; 2) visualisation, transformation and presentation; 3) collaboration and sharing; 4) and tools to support qualitative analysis. In this keynote, we will place Big Video in the context of a critical history of scientific audiovisual technologies, discuss the assumptions and aporias of qualitative video-based research since the 1950s, and challenge the ‘black box’ mentality and algorithmic normativity of default functions that undergirds data collection in much contemporary research. Then we will propose a set of tenets for Big Video to rethink epistemological and methodological assumptions and provoke new directions for qualitative video analysis. Finally, we will illustrate current and future trends in Big Video with examples from our own data collection in complex everyday settings using a variety of new technologies and enhanced methods. We focus, in particular, on methods close to our heart such as ethnomethodological conversation analysis and video ethnography.

ABSTRACT. René Tuma TU Berlin rene.tuma@tu-berlin.de

On the State and Future of Videography: New Tools <–> New Methods? Videography means the combination of ethnography and the use of video analysis. While video allows for the conservation of the sequential order of visual and audible forms of communication, ethnography in our (Knoblauch, Tuma, Schnettler 2014) understanding is essential to be able to understand and analyze the communicative actions in front of the camera. Whereas for some more language focused research the “background” knowledge might suffice in a sound understanding of the language for a more sociological research the situational context (space, objects, technologies, knowledge) are essential. Starting from this assumption, all data collection is part of an ethnographic enterprise. The researcher has to choose technologies that are available and suit the problem at hand. That on the one hand fits into the field situation and on the other hand produce videos that “conserve” as much as possible of the relevant ongoing action. The use of the camera has helped us a great deal in understanding the fine grained coordination of action in turns down to the 1/60 of a second (and even further with new cameras). Multi camera setups allow for the capture from different angles and also are well established. Many projects have used them productively. Now our new “toys”, such as 360° Cameras, drones and other devices might offer new perspectives. However, on the other hand they also enter the situations, change the perspective and will be accounted for by the participants, sometimes changing the situation, sometimes being ignored and sometimes being forbidden or excluded (our research on audiences has massive problems just bringing a usual camera to a first league football match, so how to work with a drone there?). Furthermore, new technologies open up the question of transcription again. Whereas there has been established a standard for CA transcription, there are many more variants for video analysis. However, newer technologies are going to produce new challenges. Based on a number of research projects I have participated in, I´ll give examples of the problems risen and show some ideas for further developments. A special focus will be the comparison with the video-work in other “vernacular” fields using video, which I have studied in my (Dr.) dissertation (Market Research, Police Work, Sports Training).

ABSTRACT. In this presentation, we show how we have worked with combining video ethnographic data collection and analysis with big data video analytics software in a largescale research project for a large Danish optician chain. The overall objective was to identify clients’ needs and behavioural patterns when visiting a store as well as how employees should best address and interact with their clients. For this purpose, we used a classic video ethnographic approach with handheld cameras and mounted Go-Pros to record the in-store client-employee interaction in 11 of the optician chain’s stores (cf. Due, 2017; Heath, Hindmarsh, & Luff, 2010). We also mounted a 360-degree fisheye camera in the ceiling that could cover the entire activity of a store in one of the stores, and we analysed this data real-time by a local server running the software savVi identifying movement paths.

Based on the findings from this case, we discuss how the EMCA analyst can benefit from using big data and video analytics software. We argue, that video analytics software can help the EMCA analyst overcoming two fundamental difficulties: First, video analytics software can analyse amounts of data that would cause a severe data overload for the qualitative EMCA analysis, at the same time adding valuable information about the generalizability of interactional phenomena. Second, video analytics software can guide the EMCA analyst’s focus towards interesting findings that could otherwise have been hidden, as moments during the day or week as well as spatial areas with the most, the least and the average amount of interaction can be pointed out to the researcher, thereby giving an idea of both the typical interaction and deviant cases. Finally, we propose a model for how big data analysis and thick EMCA analysis in some cases can be fruitfully combined in a blending process that we call “big thick blending” (Bornakke & Due, forth.), and we address some of the theoretical as well as the emerging ethical and legal issues that the use of big data and automatic video analytic software within the study of social interaction engender.

The research is based on a large corpus of 52 handheld recordings of client’s store visits; more than 700 hours of video recordings with mounted cameras from a total of eleven Danish optician stores as well as approximately one month of data collections with a 360-degree fisheye camera adding up to many million data points.

Bornakke, T., & Due, B. L. (forth.). Big-Thick Blending. A method for mixing together big and thick analytical insights. Big Data & Society. Due, B. L. (2017). Multimodal interaktionsanalyse og videoetnografisk dataindsamling. Samfundslitteratur. Heath, C., Hindmarsh, J., & Luff, P. (2010). Video in Qualitative Research. SAGE Publications Ltd.

ABSTRACT. The paper is based on a three-year video-based field study in the outpatient clinic of a hospital in the French-speaking part of Switzerland (González-Martínez et al. 2016, 2017). We studied interactions taking place between members of the staff as they moved around through the various areas of the clinic. To do this, we collected a corpus of 336 hours of video using a set-up composed of four cameras and eight wireless microphones placed in the corridors of the clinic, recording simultaneously over the course of seven consecutive days, twelve hours per day. Methodologically as well as analytically, our study is part of the "spatial and mobility turn" in studies of social practices and interaction (McIlvenny et al., 2009). We relied on multimodal analysis (Streeck et al., 2011) to study the interplay of talk, gesture, gaze, movement and other bodily conduct in the interactive production of social action.

In the first part of our presentation, we will cover the technical aspects of our recording set-up and the challenges it allowed us to meet. As we were studying interactions in a hospital, a particularly sensitive type of setting, we wanted to use a very discreet recording system that would not require the subjects to wear or carry any devices and that could function with only a minimal on-site presence by the researchers. Our video system covered a large area of the clinic simultaneously, making it possible to closely observe people's movements without losing sight of the larger area. Moreover we were also able to record high-quality sound in a large part of the spaces that the staff navigated. In this part of the presentation, we will also present our approach in terms of processing and coding a particularly voluminous and complex corpus using professional software.

In the second part of our talk, we will cover a phenomenon that reveals the potential and the limitations of our data. It has to do with identifying the sound to be included in the analysis and presentation of selected extracts. For example, we have an extract in which two nurses are speaking to each other as they approach each other, pass each other and move farther and farther away from each other in a long hallway, walking in opposite directions. We will address the issue of accounting for what the two nurses can hear and produce in terms of sound during their trajectories, given that these sound elements are relevant for their interaction but identically available to them only during a brief moment as they pass each other. Apart from methodological questions, these extracts further our understanding of in-passing interactions, which are organized in a different way than the eye-to-eye, face-to-face or side-by-side ecological huddles (Goffman, 1963; Ciolek, Kendon, 1990) that have received the most attention so far.

ABSTRACT. Having been involved in collecting data in “plural” (Davidsen & McIlvenny 2016) and with several 360 cameras to capture groups and individuals in motion (in nature and in museum sites), the time has come to ask the ethnomethodological question: (How) does this help to understand the participants’ perspective?

The footage, especially a combination of different camera views, provides a holistic way of ‘seeing again’, in a mediated form, the situation, and not just from the cameraperson’s perspective, but covering almost the whole situation. Laurier & Philo (2006) remind us of Raffel’s (1979) notion that video recordings make it possible to “examine past activities not as past but rather as ‘formerly present’” (Laurier & Philo 2006: 188). When watching or, rather, interacting with the 360/3D videos to focus on what is interesting in the data, the feeling of presence is definitely stronger than examining traditional 2D video footage. Seeing the situation again is a much more intensified viewing experience, especially in nature tour settings where participants examine plants through vision, sense of smell and touch. This might result in an enhanced awareness of the sensuous experiences in relation to revisiting the data in the visually constrained 2D video footage.

Marks (2000) launched ‘haptic visuality’ as a concept to understand the sensuous character of film. She underlines the importance of immediate experience in contrast to interpreting the seen, an approach that does not easily translate to meticulous multimodal interaction analysis of encounters in and with nature. However, being able to revisit the participants and their whole environment from various (close) perspectives might help analyse better their bodily experience of nature as embodied participants in a group event. In fact, big video of this kind might help us combine the two differing theoretical interests in affect: Blackman and Venn’s focus on affect as non-conscious bodily phenomenon (cp. senses) and Wetherell’s emphasis on affect as social practice (cp. sense-making). Thus 360/3D video brings us closer – as viewers and analysts – to the sensuous character of encounters with nature. But is also gives us the possibility to examine more exactly how these encounters are organised as social practices and in changing formations while adjusting to the varying material environment. This necessitates more than ethnomethodological membership knowledge (as in understanding the language or routines); the immersion of the analyst in the data where the participants are engaging with the material and natural world makes it possible to better analyse the ‘immaterial’ (Blackman & Venn 2010) nature of this specific embodied conduct.

References: Blackman, Lisa & Venn, Couze (2010). Affect. Body & Society 16(1): 7-28. Davidsen, Jacob & Mcilvenny, Paul (2016). Experiments with Big Video. ROLSI guest blog. Available from https://rolsi.net/guest-blogs/guest-blog-jacob-davidsen-and-paul-mcilvenny-on-experiments-with-big-video/ Laurier, Eric & Philo, Chris (2006). Cold shoulders and napkins handed: gestures of responsibility. Transactions of the Institute of British Geographers, 31(2), 193–207. Marks, Laura, U. (2002). Touch: Sensuous Theory and Multisensory Media. Minnesota: Minnesota University Press. Wetherell, Margaret (2015). Trends in the turn to affect: A social psychological critique. Body & Society 21(2), 139-166.

ABSTRACT. This presentation considers the significant challenges that recording multimodal and mediated collaboration poses, and discusses ways in which these challenges can be addressed in recording complex settings. Following the “availability principle” (Mondada 2006), all potentially relevant aspects of interactional encounters should be documented. Our reflection is based on recent experience in recording news production in a media house in Sweden, with the aim of describing the embodied and technological practices in and through which news stories get formatted for different media outlets (print, web and video). The communication processes are characterized by sequential, simultaneous and locational complexity. By sequential complexity, we mean that a contribution to the process, by e.g. the news editor, may be taken up by several recipients, each orienting to that move according to one’s own task. By simultaneous complexity, we mean that participants mobilize a range of communicative resources. News professionals talk, gaze, gesticulate, send e-mails and text messages, make phone calls, and pass papers around. Lastly, by locational complexity we mean that the news production takes place at different locations, both within an editorial office and between editorial offices, located in different cities.

The “morning meeting”, involving the news editor and reporters, was recorded using one GoPro camera and another handheld camera. The GoPro’s wide angle captures the room and all participants and the second camera the screen of the news editor’s computer (also projected on a screen at the back of the room), where the “morning list” is edited partly live in interaction with the present reporters. As the morning meeting is finished, the list is finalized by the news editor and then sent off to the page planning (paper newspaper production), to the web center, and to the video production unit. The work of finalizing the list was captured using one handheld camera alternating its focus between a close up of the editor’s screen and a slightly more contextual view. The work of the page planners and the web center was documented using similar set-ups: one GoPro for an overview, and 2-4 standard video cameras, zoomed in to capture screen events. Sound was recorded using an omni-directional microphone.

The way the different settings were recorded was based on a step-wise procedure, where we first studied the setting to discover the relevancies of the activity to be recorded and the environment for setting up the cameras, and then made first recordings to try out our recording configuration. Some events (e.g. fixed screens and overall stationary configurations of participants) proved relatively easy to record using our rather classic, but multiple and small devices, whereas others proved very difficult to capture (especially relating to the “micro-mobility” and “micro-granularity” of objects, e.g. a note passed around or a text message on a mobile phone). So far, approximately 8 hours of news production have been recorded using five cameras. By the time of the conference, we will have completed the projected last step: a simultaneous and longitudinal recording tracking one morning of distributed news production work.

ABSTRACT. In my doctoral studies on resemiotization as social process in editorial work on talk show interviews, I seek to trace the dynamics of journalistic work in the televised mass media. I focus on how current stories are chosen, discussed and angled – and especially how semiotics are negotiated and translated into exo-somatic artefacts (e.g. onto black boards, and into manuscripts, notes and cue cards) in this process. The aim is to uncover the social constructions of representations of the interviewees’ personal experiences leading up to the live actualization of the interviews. To conduct such qualitative work flow analyses, I have closely followed two journalists from the Danish live televised talk show Aftenshowet. My data collection has been conducted with state-of-the-art recording equipment including multiple 4K 360 cameras and spatial sound recording devices. Due to the development in size, price and resolution of 360 cameras combined with ambisonic audio recordings that can be linked to the 360 videos, it has become increasingly relevant for us to reimagine sound and vision when collecting data. Therefore, my overarching aims are 1) to show how these newly available video and audio recording technologies can help tracking social phenomena ‘in the wild’ – and why this matters, or should matter, to video ethnographic researchers; and 2) to discuss some of the challenges we encounter in working with 360 video material. Once we start recording and, thus, seeing the world in new ways, we must also find new ways to represent our findings in data sessions, journals, at conferences, etc. For example, how can a full-sphere video still shot be unfolded onto the 2D pages of a journal, so it makes sense visually to readers who are used to experiencing classic visual photo material from cameras with a restricted field of view within a rectangular frame? And what are the best ways to transcribe this omnidirectional audio-visual data? In my presentation, I wish to elaborate on the advantages of using advanced technologies to collect complex, high quality, multifaceted data when investigating journalistic practices. I will illustrate how these newly available audio-visual tools offer a greater width and depth in data collection that can help to trace specific parts of the final broadcasted interview back to early editorial meetings, prepping interviews and sometimes even semi-personal small talk between the journalists.