29 January 2018

Ethnography & Case Study Research: Saying yes to the project

Over the past few Mondays, we have been posting on the recent case-study research project I worked on with OCWI. Working with OCWI on this short-term project was an opportunity

In this post, I want to reflect on one more unexpected learning opportunity that came out of this experience: what it means to say "yes" to a new project.

What did it look like to say "yes"?

Interestingly, I hadn't applied to OCWI for this research contract. In fact, I had applied to the organization a few months earlier for a different research role, and had OCWI keep my resume on file. So, when I got an email from my future supervisor at OCWI about this position, it was unexpected, and at the time I wasn't sure if I would be able to juggle more work with my already busy teaching schedule.

Even so, I felt like it was important to meet for an interview to network, learn more about the organization, and learn about the proposed project. I left for the meeting telling myself that I would probably have to turn down the opportunity, but it would be good to "show my face" and let this organization know I was interested and had important skills and experience to offer.

In the meeting, as soon as I heard what the project would be about... I was sold.

Not only was the program I was asked to write a case study on reflective of my previous research interests and areas of expertise, but it was also a chance to gain valuable work experience with this research network, and to actually do field research again -- after years of writing my dissertation, and then the time-consuming teaching duties entailed in preparing new courses year after year.

I immediately began calculating how many hours I could squeeze out of my week, and weighing whether or not I could realistically complete this work to the high standard I would want. I was up front with my soon-to-be supervisor about my availability, my enthusiasm, and interest in this project. When I left the meeting, I had all but signed the contract.

Thinking about what it means to say "yes"

Two months later, I met with my former PhD supervisor for lunch. As I told her about the project I was then deep in the middle of, she remarked how one of my strengths was my willingness to try new things, to take on new challenges and say "yes!" to opportunities outside of my comfort zone.

This comment really struck me because I hadn't considered this willingness to try new things to be an out-of-the-ordinary strength for a) an anthropologist, and b) a recent PhD grad today.

Anthropological adventurousness?

As a relatively shy person, I have always found conducting social research to be a step out of my comfort zone. Connecting with new people and asking them questions about their lives was never something that came naturally to me.

I chose to become an anthropologist because I loved what this approach to social questions could tell us, not because I love talking to strangers. Yet, now that I am an anthropologist, I know that in order to be an effective social researcher, I need to step out of my comfort zone sometimes.

Enacting (neoliberal) agency in the face of neoliberal structures?

Secondly, as a recent PhD grad in a highly precarious academic labour market, many of my conversations with colleagues over the past several years have focused on how not to get stuck in adjuncting purgatory. My approach has been to apply for interesting opportunities that seem like a good (if not perfect) fit... We are all expected to be "entrepreneurs of ourselves" in a neoliberal society, right Aihwa Ong?

I was therefore surprised to hear that it seemed that many of these same colleagues aren't doing the same. One of the great strengths of someone who, for a very long time, identified as a professional student is that I am able and willing to learn new things!

On the other hand... the more I thought about "adventurousness" as a strength, I also found myself thinking about the structures that limit the ability to say yes.

I was only able to take on this new research project because of the modicum of stability I have this academic year in teaching the same courses. Instead of dedicating 8-20 hours a week in preparing new lectures and assignments on top of my regular teaching duties, these are mostly already in place -- meaning that time can be given to exciting new research opportunities.

If this opportunity had fallen into my lap during the two previous years, I would have had to turn it down during the academic term. In working as a professor, I love the opportunity to teach anthropology to my students, but as an adjunct I work in a structure that has often limited my ability to do other kinds of "professorial" tasks (such as supported time for writing and research).

Lessons from saying "yes" to the project

Taking on this research contract has been a valuable experience, both personally and professionally. Yes, there was a bit of a learning curve to this project for me as I engaged with new research approaches and methodological vocabulary. Importantly, I didn't need to identify these problems and their solutions all on my own. A key lesson has been that being up to a new challenge doesn't mean tackling it alone.

I was hired because my supervisor/ co-researcher recognized how ethnography (and an ethnographer) could add a valuable dimension to OCWI's series of case-studies. So, I decided to approach this opportunity as an ethnographer; in addition to making sense of the community-based program at the heart of our case study, I would also make sense of the unfamiliar culture of this research network.

In approaching my co-researcher/ supervisor at OCWI as a key informant, I positioned myself to ask naive questions, check-in about my understanding of the project and directions, and draw on her local cultural expertise. For instance, she suggested methodological resources on program evaluation and case study research, facilitated introductions to key research participants in our community program, and helped orient me to key landmarks in the grey literature. Rather than getting caught up in "imposter syndrome" in an unfamiliar field, I was instead doing what I have been trained to do as an anthropologist.

So, to close out this post -- the last in this series until the research communications from this case study are published -- I would like to say "thank you!" to my supervisor/ co-researcher, Dr. Michelle Coyne, and OCWI for these enriching opportunities to think about the connections between Ethnography and Case Study Research.

Further reading:

22 January 2018

Ethnography & Case-Study Research: Grey Literature

What is "grey literature"? Is this a term that you use in your work? Why does knowing this language matter?

The first time I heard the term "grey literature" I was speaking with an anthropologist who had become a public servant with the Canadian federal government. He was describing some of the tasks of his job in policy development and mentioned reviewing grey literature.

I could not recall ever using myself this term myself, or coming across it in ethnographic studies or methods handbooks. When I asked him to elaborate, he referenced all of the internal reports and white papers that provided important background for current policy issues. The term struck me as a peculiar, almost demeaning way to talk about existing documents that seemed like they were rather important in giving a researcher a holistic picture of the context under study.

Although a strange term to my ears, clearly the concept of "grey literature" was a normative term in the field of policy development and in the public service.

The next time I heard the term grey literature, it was when I was hired to develop a case-study for OCWI. As my supervisor outlined the project she envisioned, reviewing the grey literature on the program being evaluated and in locating it in its larger context would be paramount.

Again, it struck me that this term  despite being rather foreign to me  was part of the everyday language of Case Study and Program Evaluation research that I was now venturing into.

So, what is "grey literature"? 

The term seemed so foreign to me, and yet seemed to describe something so familiar or commonplace that I found myself overthinking the concept, and worrying that I was out of my depth as an ethnographer doing Case Study and Program Evaluation research.

As it turns out though, as an anthropologist, I have always worked with grey literature  I just didn't have this blanket term in my methodological vocabulary!

Grey literature encompasses all of the documents that are available, but not intended for wide distribution. Think of the term as contrasting with "black and white" literature that is commercially published (see Western University Library's informative, quick video on the topic of grey lit).

According to the International Conference on Grey Literature (GreyNet), grey literature includes the "multiple document types produced on all levels of government, academics, business, and organization in electronic and print formats not controlled by commercial publishing i.e. where publishing is not the primary activity of the producing body." As such, grey literature includes:

  • government documents and websites
  • white papers
  • evaluations and research reports
  • conference proceedings and unpublished/ working-papers
  • data (e.g. census, geospatial, economic)
  • practice guidelines
  • patents
  • theses and dissertations
  • internal communications
  • community-based websites, newsletters, and blogs

For some anthropologists, perhaps "grey literature" is already a prominent part of your methodological vocabulary. However, in speaking with colleagues, I have found that I am not the only anthropologist who was unfamiliar with this term, even though I am deeply aware of the importance of the kinds of documents it references.

So, to come back to the last question I posed at the beginning of this post...

Why does knowing this language matter?

Knowing this language matters because it allows us to speak about and champion our skills as social researchers to a broader public.

As I have discussed in previous posts in this series, anthropological ethnography has a lot of offer in projects centred on case studies or program evaluation (and beyond). However, in order to show that we can 'walk the walk' of these kinds of projects, and with these kinds of audiences, we first need to learn how to 'talk the talk'.

Quick links and further reading:

15 January 2018

Ethnography & Case Study Research: Applying an ethnographic approach

Last week, I shared my recent experiences being hired as a research assistant at Ontario Centre for Workforce Innovation to conduct a case-study on a local example of innovation in workforce development. In this post, I want to continue that discussion, thinking a bit more about: what case study research is, how it intersects with ethnography, and why this matters for anthropologists.

What is Case Study Research? 

As anthropologists, we might be used to referencing case studies in our classroom. Isn't Malinowski's work among the Trobrianders an interesting case study of non-market exchange? How have Evans-Pritchard's study of the Azande or Gmelch's study of baseball magic become textbook cases for discussing systems of supposedly irrational belief? Or we might ask students to focus their examination of course issues on a specific case-study in a final essay...

This way of understanding a case-study echoes how Robert K. Yin understands the term and frames "Case Study Research" as a (qualitative and/or quantitative) methodological approach in his text Case Study Research: Design and Methods (5th edition).

Interestingly, Yin argues that a case-study approach is useful
in situations when (1) the main research questions are 'how' or 'why' questions;  (2) a researcher has little or no control over behavioral events; and (3) the focus of study is a contemporary (as opposed to entirely historical) phenomenon (Yin 2013, 3).
Case studies, writes Yin, have been popularly used by a wide range of disciplines, including anthropology. Whether conducted from the perspective of a psychologist, political scientist, social worker, nurse or community planner, Yin states that "the distinctive need for case study research arises out of the desire to understand complex social phenomena. In brief, a case study allows investigators to focus on a 'case' and retain a holistic and real-world perspective" (Yin 2013, 4).

How does Case Study research reference or intersect with Ethnography? 

So far, this understanding of when and why to apply a case study research approach seems remarkably similar to ethnographic research! And the parallels continue...

Yin frames the six sources of evidence used in case study research as documentation, archival records, interviews, direct observations, participant-observation, and physical artifacts (Yin 2013, 105-118). Hmmm....

In order to be a good case study researcher, Yin argues (2013, 73-79) that one must:

  • Ask good questions - Essentially, Yin asks researchers to recognize the iterative nature of the research process and to constantly ask questions as the study develops during fieldwork, even as "substantive review" comes later. Yin even discusses grounded theory as an apt approach to case study research.
  • Be a good "listener" - Yin advocates that we be active listeners and pay attention to all the social cues given during interactions with study participants. Importantly, Yin is advocating a certain amount of cultural relativism -- without using this term -- and openness to paying attention to what people do as well as what they say.
  • Stay adaptive - "Few case studies will end exactly as planned. Inevitably, you will have to make minor if not major changes, ranging from the need to pursue an unexpected lead (potentially minor) to the need to identify a new 'case' for study (potentially major)" (Yin 2013, 74). If this doesn't sound like ethnographic research, I am not sure what does!
  • Have a firm grasp on the issues being studied - Yin's argument here is to keep one's research question or purpose in mind throughout the research, and attend to how the data collected might shape or demand a shift in the direction of the research.
  • Avoid biases - Yin urges researchers to be open to what the data shows, rather than trying to force the data to support a preconceived hypothesis. Again this echoes how anthropologists often pose exploratory how and why questions in our research.
  • Conduct research ethically - Here, Yin even cites the AAA's Code of Ethics among other disciplinary and processional ethical codes.

In many ways, how anthropologists approach ethnographic research seem to mirror how Yin frames the case study approach. Yet, Yin frames ethnography as a single method that can be incorporated into case-study research.

A good place to start unpacking the placement of ethnography in his toolkit for case study research is to reflect on how ethnography is understood outside of anthropology.

Although more and more people are being hired in alt-ac careers as professional ethnographers, there is still a misunderstanding or a "watering-down" of ethnography in industry and other professional spheres. What anthropologists are apt of see as "ethnography-lite" is a relatively narrow understanding of what ethnography is, and seems to reflect how this research approach has been popularly adopted by non-anthropologists, for instance in education, nursing, and business. As Marc LeFleur (VP Insights and Co-Head Health at Idea Couture) underscores "something has been lost in this widespread adoption of ethnography outside of academia. Ethnography is on its way to now just becoming another method, simply shorthand for hanging out with people for longer than a focus group would take, another hammer in the market researcher’s toolkit."

The Case for Ethnographers...

Recognizing the overlap and shared strengths of ethnography with case study research presents an interesting opportunity for anthropologists, especially those of us looking to engage a non-academic audience. Not knowing the language of Case Study Research means that anthropologists are missing out on important opportunities to bring our holistic, rigorous, and thickly descriptive approach into social research in the public sphere.

Like ethnography, case studies are used to answer how and why type questions. We can see from Yin's text that the sources of data used for case studies are the same as those any ethnographer would consider. Similarly, the skills and approach to research Yin outlines for case study research sound a lot like my second-year lecture on ethnographic methods.

What makes case study research different from ethnography?

Depending on the type of case study undertaken, there really might not be much difference between this form of research and ethnography. In my work with OCWI, I found that my research approach as a trained ethnographer fit rather seamlessly with the goals and structure sought for this report ― with the exception of framing the research question, anchoring my analysis, and writing for a non-specialist audience.

Research Questions

Importantly, case study research questions may be preset by the organization or funder rather than by the researcher themselves.

In my work with OCWI, I was hired to develop a research report on a single, predefined case that would speak to their mandate regarding workforce innovation. This meant that during research, I had to keep the priorities and interests of my employer (and case program stakeholders) in mind. A challenge here was balancing how to stay on task, while attending to interesting threads that were less pertinent but nonetheless important to understanding the case.

For instance, in my research of the Youth Empowering Parents program, I was interested in following up on the connections and outcomes surrounding voluntarism, community belonging, and the role of the YEP program in immigrant integration. However, with OCWI's key interests around employability, it was important to consider these research threads in relation to the task at hand. YEP's interest in framing their program as a youth empowering initiative and voluntarism opportunity for low-income youth encouraged me to trace connections between youth voluntarism, community/ civic engagement, and OCWI's key interests in employability. Questions around identity and community belonging -- although fascinating -- were less important to pursue in the context of this research.

Anchoring the analysis

While my background in social theory certainly helped me to analyse my data, this report was not to make theory explicit. When I began writing my first draft, I felt a little adrift in terms of how to speak to OCWI's understanding of employability and employment outcomes. This industry-specific language was not something which which I was familiar and I felt a bit out of my depth.

By recognizing my OCWI supervisor as a valuable informant in this field, I approached her for guidance. She steered me to key documents and policies from the Ontario Ministry of Advanced Education and Skills Development (which is funding these case studies). These documents helped to give me an anchor point for considering the Program Evaluation dimension of the intended final report. Grounding my report in this literature was important for helping produce a case study that would speak to our intended audience(s).

Writing for a non-specialist audience

One of the reasons that I was hired by OCWI to develop this case study, was because my supervisor recognized the value and richness of ethnography and wanted to bring this dimension into our report. The format of my report was set by the template developed by OCWI, covering context, history, implementation, and “lessons learned.” Being able to frame the lessons learned, or impact, of the YEP program in people's everyday lives was clearly well-suited to an ethnographic approach.

While case studies might be directed at academic readers, they are also likely to be of interest or use to non-specialists like community stakeholders, policy makers, practitioners, and funders connected to the case you are studying (Yin 2013, 180). In my case study for OCWI, it was clear from the outset that the report should be accessible, not overtly theoretical, but nonetheless based on rigorous empirical evidence and analysis. Our anticipated audience would be community organizers across Ontario (and potentially beyond) interested in implementing similar workforce innovations in their own communities.

In my case, I worked in close contact with both my supervising colleague at OCWI and my key contact at YEP throughout the writing phase. In many ways, this experience reflected the professional relationships that academic anthropologists have with colleagues/ supervisors/ reviewers and their research participants, incorporating and addressing constructive criticism and concerns for representation from both parties. Sharing my early drafts with both partners helped to ensure that my report met the needs of both my employer's mandate, and the needs of our community partner who would use this report to help grow their program in different communities and gain the support of additional funders.

The experience of learning about and doing "case study research" has raised some important questions for me about the intersections of this approach with ethnography -- some of which I have tried to answer here. Next Monday, we'll continue this series by thinking about some more of the parallels between this form of research common in alt-ac spheres and ethnography through the concept of "grey literature." Stay tuned!

If you have experience with case study research as an anthropologist, please let us know! You can tweet us @anthrolens or email us anthrolens@gmail.com.

Quick links and further reading:

08 January 2018

Ethnography, Case-Study Research, and Program Evaluation (Oh my!)

What is Case-Study research? What about Program Evaluation? What role can Ethnography play in these other approaches to social research?

As an anthropologist, "Program Evaluation" and "Case-Study Research" were familiar terms to me. But these were not methodological approaches or reporting styles that I had received formal training in. I had a vague idea of what both of these terms meant and how they might apply to my research. In fact, I had never thought much about what these kinds of approaches would look like combined with ethnography, or how ethnography might be used to do this kind of research... until recently.

Between September and December 2017, I had the opportunity to learn more about these research approaches in my role as a research associate with the Ontario Centre for Workforce Innovation (OCWI). OCWI was launching a new series of case-studies on programs and practices in Ontario that demonstrated an innovative approach to workforce development. Based on rigorous research, and in keeping with OCWI's mandate, a case-study should evaluate the program or practice, and based on this evidence, explain how "these practices can be transferred, scaled up, and made relevant for the broader workforce development sector." The research associate supervising these case-studies envisioned using ethnography to help ground the impacts of the case-study project or practice in the everyday lives of community stakeholders.

One of the great strengths of ethnography is the iterative nature of our approach to social research, allowing you to shape your research questions and follow up on leads that you didn't expect. So,  what does it mean to bring an ethnographic research approach and perspective into a project where the framework is very clearly defined through case-study research? How can ethnographic insights provide meaningful metrics in evaluating a program or process or practice?

As we have noted elsewhere on the blog, it is important (and expected) that anthropologists learn to speak the langauge of their research participants. For an anthropologist venturing into the realm of interdisciplinary, alt-ac research, it is also important to learn to speak the langauge of our employers and their stakeholders.

This is where I chose to start this project: by learning to speak the language of case-study research and program evaluation. As a social research professional, I quickly found that these other approaches are really more like dialects than completely different languages.

For a basic introduction to Program Evaluation, check out these Open Learning modules on Interdisciplinary Program Evaluation from Ryerson University:
  • Module 1: The Basics of Program Evaluation
  • Module 2: Logic Models
  • Module 3: Qualitative Tools in Program Evaluation
  • Module 4: Performance Management
After reviewing these different modules (described as covering about "10-15% of a a one-semester undergraduate, graduate or certificate course"), I had a much better idea of how a program evaluation project is framed. Knowing that program evaluation was a key component of this case-study project, these modules helped orient me in many ways to the task of this case-study project. I could see what ethnographic research could contribute in this type of study, better consider what kinds of questions I should pursue as the project developed, and how I would need to frame my results in my report.

Next Monday, check back for more in our "Ethnography & ..." series exploring the connections between Ethnography and Case-Study research through this project.

Quick links:

04 January 2018

Highly Accurate Pictures of Anthropologists

Here's something fun for back to class: the relatively new tumblr Highly Accurate Pictures of Anthropologists!

Alex Golub posted about his new project on anthro{dendum} back in November, but it is totally worth a re-post here. Golub notes the impetus for this new tumblr as "driven by [his] long-term interest in curating open access material. The Internet is awash in high-quality material. But it’s also awash in… well, let’s just say that sometimes the signal to noise ration on the Internet is not where I’d want it to be. Nowhere is this more true than when it comes to pictures of anthropologists."

Not only a fun/ useful collection to review, but the tumblr account also has lots of great links to the online sources for these photos (#FurtherReading). Golub is also taking requests if there's an anthropologist out there whose highly accurate and well-sourced picture you need.

Happy 2018!