Computational Processing of Ethnographic Data

After a few intensive days in the field, you and your team have returned to the familiar settings of the university. In front of you, there is a big pile of observation schemes and seating charts from the field awaiting you to turn them all into one common spreadsheet. Luckily, most ethnographers appear to have carefully recorded audience attention in full accordance with the instructions. After having typed in the last crinkled seating chart, you finally have a full overview of all the recorded quantified attention behavior from the field. You log on to the Ethno-platform to fetch a file with all the fieldnotes from the festival and load it to a programming application. You swiftly extract all the notes that accompany your newly created spreadsheet. Overwhelmed by this huge corpus of fieldnotes and observations, you wonder: Which computational techniques would be most helpful to find patterns in these data?

Picture 6. Structuring ethnographic data

Computational Potential

Computational programming is, unfortunately, often presented as more complicated or math-demanding than it needs to be. In many ways, it is like learning the grammar structure of a new language. As soon as you know the basic rules for how to construct a sentence and bend your verbs, you can slowly begin to communicate. Same thing with programming languages; when you understand the syntax and learn the basic logic behind building up a “script”, you can execute simple code. And even with a few basic skills, you can benefit from programming tools when working with ethnographic data. In the field of social data science, there have been different suggestions to how computers can help process and analyze ethnographic text: some find the machines helpful when coding their material; some have entrusted them with the responsibility to automatically code large parts of their fieldnotes; while others have used text mining techniques to explore notes and interviews to find new themes or patterns that they hadn’t noticed before. These are just a few examples of how computational potential paves the way for new ways to analyze ethnographic data. So, how did we put computational power to good use? We wanted to use computational techniques for two things: to explore our ethnographic material and to combine it with other data types that we collected at the People’s Meeting.

Uniting Ethnographic Data Sources

During the few days the festival lasted, we compiled a ton of beautifully aligned fieldnotes. When accessing the Ethno-platform, the infrastructure allowed us simply to press a button to fetch a file that contained all of them. We loaded the file to a programming application and converted it to a spreadsheet. Now they were ready for computational processing. Imagine a spreadsheet where each row holds the data of a fieldnote, and the different columns help to divide the different information and metadata related to that fieldnote (see Picture 7). Now, returning to the common format of our fieldnotes: each note was written, following three formalities (see Post 2) and holds meta-data about the described situation. Therefore, we could extract information by using these features with different search commands in the programming application. This meant that we could sort the data by date, time of day, ethnographer, event tent, and we could fetch quotes and analytical comments. These can surely be helpful features for the initial data exploration, and our aspirations to computationally process our fieldnotes were slowly being realized. However, we also wanted to combine our systematized quantitative observations with the spreadsheet of fieldnotes.

As alluded to in the beginning of this post, we turned the piles of attention schemes and seating charts into a common spreadsheet. The next step was to merge it with our fieldnotes from the Ethno-platform. The result was one grand spreadsheet of all our ethnographic data. The columns contained the text from fieldnotes and metadata as well as different levels of attention and seating information at each event. And though the data from the seating charts and attention schemes were of a different kind, namely reduced quantitative measures of attention and presence, they were now merged with the accompanying descriptive (though also structured) fieldnotes in which our group of ethnographers had strived to capture attention dynamics in interactions during events. We were now finally piecing together the somewhat fragmented ethnographic puzzle.

Picture 7. A spreadsheet of fieldnotes from the Ethno-platform

From Potential to Beneficial

With the united ethnographic data, we could finally begin to experiment with computational techniques for analysis. After having discussed different ways we could approach this sort of dataset, we decided to start simply by visualizing the quantitative observations of attention. In Figure 8, we have plotted the audience’s attention levels for each observed event. From our master spreadsheet, we extracted all events observed on Friday at the People’s Meeting (vertical axis). We used our observations of how many looked at their phone and at the stage throughout events to create a combined attention score for the two types behavior ranging from 0-10 for each 15 minutes of the event (horizontal axis). As each event lasted an hour this meant that the maximum attention score for an entire event is 40.

Figure 8 might not look very interesting at first glance but visualizing ethnographic observations does bring potential: it can guide parts of our analysis and bring some transparency to analytical choices. Questions and surprises emerging from what we see in the visualization of attention during events could be explored more by diving into the related fieldnotes. We can for instance examine how the audience preserves attention over time in a political event, and we can hold this up against the theme discussed during the event and observations from fieldnotes.[1]

From the visualization, we could also see that the approximate fraction of people paying attention to the stage was relatively stable overall across time intervals and across events, but we saw some small variations. And if we dove into the fieldnotes, we learned for the event with the lowest score, that it was extremely hot around the stage where the event was held. This meant that many in the audience were struggling with the heat, and instead of looking at the stage some were fanning themselves with magazines while others were focusing on ice cream they had bought before the event started.

Picture 8. Sorted bar chart of attention over time among audiences at different events

This was just one example of how we could explore our ethnographic data computationally. A possible next step could be to examine the differences between attention in the back and the front of the audience section, or to try to track temporal and spatial variation in attention at the festival site. When we had metadata recorded such as time and place for observations then we can also move on to merge other spreadsheets with other data types to our grand spreadsheet of ethnographic data. This could be data containing ticket sales for each event, tweets posted by event organizers, or maybe even weather data for each day during the festival. When we have the ethnographic data and metadata united in one spreadsheet loaded into a programming application then we can combine it with other data types.

So now we’ve unfolded our methodological and to some extent experimental approach to ethnographic data collection in an interdisciplinary setting. In the coming post, we will move on to discuss thick versus broad data and the implications of the kind of data we ended up collecting.

Notes

[1] We could for instance see that some ethnographers didn’t record attention scores all four times during events, as bars were missing for some events. In the fieldnotes from these events, we learned that this is due to events starting or ending early or the ethnographer arriving late.

DISTRACT

The authors if this blogpost series are Sofie Læbo Astrupgaard — PhD fellow in Social Data Science at the University of Copenhagen, Clara Rosa Sandbye — PhD fellow at the Department of Anthropology at Aarhus University, and Emilie Gregersen — MSc student in Social Data Science at the University of Copenhagen. The trio has been working as a part of the interdisciplinary research project DISTRACT, studying the dynamics of issue attention at a political festival.

Source

Our ethnographic data is in the cloud, but our heads are not

More and more anthropologists are conducting, storing and circulating their research in the cloud. Cloud storage – typically in the form of Apple iCloud, Google Drive and Microsoft OneDrive – is now the default storage option on the smartphones, netbooks, tablets and other digital devices that have become commonplace tools of fieldwork. Messages are sent to interlocutors through cloud platforms like WhatsApp. Interviews are carried out through Skype and Facetime. Apps for ethnographic research are proliferating. Evernote is replacing the field notebook. Articles are written collaboratively in browser-based cloud environments like Google Docs or Microsoft Office Online. Field reports and article drafts are circulated via Dropbox, WeTransfer, Box and Mozy.

Cloud infrastructure increasingly underpins growing areas of academic research practice. Yet to date there has been little – if any – critical reflection on the ethical, political and legal implications of cloud computing for social science researchers. The aim of this post is to begin moving discussions of digital security beyond the bare essentials of locked filing cabinets, password-protected laptops and hard drive encryption. Having spent a year and half conducting fieldwork in the cloud, becoming progressively more paranoid about data security in the process, I’d like to draw some much-needed attention to cloudy digital research practices that anthropologists increasingly engage in but may not recognise as security issues. In doing so, I hope to prompt discussion on the implications of cloud computing as it becomes increasingly infrastructured into research, teaching and administrative activities across universities. With higher education institutions turning to cloud services to deliver their e-learning and information management systems, and with research funders requiring grant awardees to deposit their field data in cloud databases, anthropologists urgently need to begin getting their heads around the cloud.

The bearable lightness of laptops 

While most anthropologists have long been aware of the ethical and security concerns surrounding the sending of sensitive information through email, the problem with the cloud is that many people don’t know what it is or even realise they are using it. Like most infrastructure, it is designed to disappear. This problematic invisibility means that cloud computing seems to fly under the ethics and security radar.

Despite the image of fluffy ethereality that the cloud metaphor conjures, the cloud is concrete, political and aggressively expanding across the surface of the planet. At its most basic, cloud computing refers to an infrastructural shift from desktop computing – where files and applications were stored on the local hard drives of our computers – to a form of online computing, where these are stored in data centres accessed remotely ‘as a service’ through the Internet. In the context of my fieldwork, ‘the cloud’ was mostly a windowless, subterranean data centre repurposed from the ruins of a Cold War bunker. It was about as far away from the sky you could possibly get, and distinctly un-cloudlike – except for its whiteness:

It’s thanks to data centres that our digital devices are so light, portable, and fieldwork-friendly. Laptops no longer have CD or DVD drives because we download our apps, programs and software online, directly from data centres. As more of our files and applications are stored in and streamed from data centres, the bulky storage drives and connectivity ports that once weighed down our devices, are being stripped away to reduce weight and replaced with minimal capacity internal memory. With most of our computing needs now implemented as web services, the main task left for our devices, as powerful as they are, is more and more just to act as portals to data centres.

But this lightness comes at a significant cost. Removing ports removes possibilities for increasing memory using external storage like USB drives or micro SD cards. And shrinking internal storage capacity means that users increasingly have little choice but to store their data in the cloud. Cloud storage is now infrastructured into smartphones, tablets and other digital devices as the default storage option. Taking these devices off-cloud is often made deliberately unclear by tech manufacturers. It is also becoming increasingly difficult, with cloud-connected devices designed to silently upload files without any fanfare, potentially leading to the inadvertent sharing of ethnographic data.

Data murk

With smartphones being used to record interviews, capture video footage, take photos, send files and write and store fieldnotes, anthropologists can now quickly generate large quantities of born-digital ethnographic data that soon exceed our mobile device’s storage capacity. In this context, the cloud, with its promise of ‘free’ and ‘unlimited’ data storage space is a tempting solution.

However, data stored in the cloud remains legally, ethically and epistemically murky. A severe lack of legislative regulation means online data is typically subject to the service level agreements and terms and conditions of each cloud provider. In cases where data stored in the cloud is unprotected by intellectual rights, you may effectively be transferring ownership of your ethnographic data. You should therefore exercise caution before storing data with any third-party cloud service providers.

Even when an online service is not specifically marketed as a ‘cloud service’, the basic rule of thumb is that any files exchanged or interactions that occur over the Internet will be stored in data centres. That means conversations through Skype, Facetime and WhatsApp. It means the mundane e-learning platforms and management systems (like Moodle), that we regularly encounter but rarely reflect upon. It also means any emails or attachments that you send (even to yourself as a back-up copy). Emails sent outside of your university network are sent in plain text and are therefore never ‘private and confidential’. As I heard many times during my fieldwork, ‘email is about as secure as a postcard’.

Passing private and perhaps sensitive ethnographic data on to unknown others in the form of cloud providers could be considered a serious breach of the fiduciary duty anthropologists have to their research participants. In the post-Snowden securityscape, we must assume that data stored in the cloud will be subject to surveillance. Commonly used cloud file-sharing services, such as Google Drive, Apple’s iCloud, Dropbox, WeTransfer, Mozy and Box will not be appropriate for sensitive or personal data.  If you find yourself having to use the cloud then you need to encrypt your files before uploading them. VeraCrypt  is an easy-to-use free tool for encrypting files in secure way before sending them online. pCloud offers fully encrypted cloud storage. Mega  is also worth mentioning – it runs some basic encryption inside the browser before the file is uploaded to protect data that is being transmitted over an open/public Wi-Fi connection against low-level snooping. Though it is certainly not ‘government-proof’.

Most university networks offer secure files storage on servers located on campus that will meet data security and privacy requirements. This provides a layer of assurance that cloud providers, who could store your data anywhere in the world, cannot.  With increasingly stringent data sovereignty regulations – where data is subject to the laws of the country in which it is stored – it may also be necessary to know the physical location(s) of the data centre(s) you are using. Storing data in local data centres may become a standard condition of future fellowships and confidentially agreements.

Ideally, anthropology departments would provide PhD students and supervisors with a secure online storage space for the transferring of field reports, research materials and other file exchanges (anything sent over the Internet should, of course, be anonymised, unless your informants have specifically requested otherwise or the conditions of consent explicitly state otherwise). Undoubtedly the safest way to share files is to physically exchange a storage device. Data centre professionals call this the ‘sneakernet’. Despite all the cloud hype, in the data centre industry, the most secure and the fastest way of transporting large volumes of data to the so-called cloud is simply to load it in the back of a ‘hardened’ truck and drive it there, giving a whole new meaning to ‘hard drive’.

The Right to Erasure 

The new EU General Data Protection Regulation (GDPR) framework provides ‘data subjects’ (interlocutors) with the right to have any personal data the anthropologist may hold on them permanently erased. My fieldwork experiences highlighted considerable ethical and legal dilemmas surrounding the safe and secure disposal of data stored online.

When you delete an email, file, photo, social media post or even close an online account, you are not necessarily deleting them from the data centre in which they are stored. From the cloud provider’s perspective, deletion often simply means removal from the end-user’s interface, while the information typically remains locatable at the data centre-end. Most of your online activity is simply left on data centre servers in a state of involuntary permanence. This could be considered a serious infringement of research participants’ privacy if they want or expect their data to be deleted – raising problems if researchers have promised to destroy certain data upon completion of their project.

Cloudy Futures 

Cloud technologies offer valuable new tools and virtual spaces for the storage, sharing and writing of ethnographic data. But they also pose challenges to the ethical structures of anthropology that we are only just beginning to articulate and that require us to accordingly reflect on data security in the cloud as a standard part of ethical practice. Anthropology departments, institution review boards and ethics committees need to begin to respond to the changing security requirements of the digital research environment by offering more effective training in this domain.

Confidentiality agreements, ethical obligations or digital import/export restrictions tied to research grants will no doubt soon preclude the use of third-party cloud services as standard practice. At the same time, research councils increasingly require grantees to submit their ethnographic data for indefinite storage and re-use by third parties through online public cloud platforms. These often contradictory codes and requirements at different bureaucratic, legal and ethical levels mean that the cloud is at once being infrastructured into research practice and at the same time regulated out, which will make meaningfully navigating and negotiating this cloudy terrain difficult. The powerful commercial imperatives of connectivity and the energy-intensive environmental destruction that underpin the creeping ubiquity of this computing infrastructure, make interrogating the cloud all the more urgent.

Alexander Taylor is a PhD candidate with the Department of Social Anthropology at the University of Cambridge. His research explores how technologies and infrastructures of data storage intersect with planetary scales of security and dystopian digital futures in the data centre industry. In this post, he explores some of the security implications of cloud computing for social science research practice.

Guest Contributor

This account is used to upload posts by guest contributors to the blog. For more information about contributing to anthro{dendum} please see our contact page.

Source