Notes from the training
RESEARCH TRAINING – DAY 1
Karl Popper ... “scientific method” of research:
- start with the theory: it doesn't have to be elaborate, but start with the pre-conceptions we have of how the world operates ...
- from the theory ... we develop some predictions or hypotheses ... i.e. what we expect to see; what we anticipate to happen [predictions/hypotheses are developed on the bases of the theory (our preconceptions)]
- hypotheses take the form of a statement of what we expect ...
how we know whether we can trust the information and conclusions we draw ... we test by:
- waiting for events to transpire ..
- running a research project
research results ... confirm vs corroborate ... we reject or fail to reject hypotheses. research is an ongoing process so we cannot ever confirm categorically and/or for all situations ... but we can corroborate (as likely to be valid)
different approaches to and use of theory:
- dogmatic theory ... like marxism and psycho-analytic theory where the theory remains in tact .. and something else explains why the theory is not confirmed; enormous explanatory power;
critical theory/scientific ... when we test and reject ... we go back to the theory and change it;
1.survey – a research method
2.questionnaires – a research instrument ...
3.quantitative research ... smallest number for sample is 30
- the variables you use ... gender, race etc ... must relate back to the theoretical statement one is working from;
- don't take for granted that any of these variables play a role; [e.g. why do we do a race or gender analysis when it may have no bearing on what we are trying to find out? we can test whether these variables play a role though studying previous research and their methodology]
qualitative research: interviews – e.g. indepth interviews, life histories sample surveying: natural limits to doing IDIs that has to do with logistics and time. IDIs are long and time consuming; consider the law of diminishing returns. .. what value add comes with interviewing one more person. consider: do you have enough information – or does the information reflect all the different kinds of perspectives?
rules of thumb:
- dont interview more than 30 persons;
- length: no longer than 90 mins .. but natural limit is 1 hr
snowballing techniques ... designed to bring you to people/groups that you don't necessarily know or are aware of
research planning – also need to be economical (time and efficiency); more interviews make task of extracting information more difficult... i.e. its both a logistics and substance issue
observation – study events/gatherings rather than what people say; can also participate in the group ... not asking questions but looking what is happening. must have an idea of what you're looking for .... focus groups – replicates the process by which people shape their opinion .. and to capture the debate; makes more sense on aspects that shared within the community; not a time saving mechanism but when you feel the debate will add understanding ... ; try to make the groups homegenous .. people are more comfortable and open among people they consider like themselves surveys ... a type of interview .. but done at mass scale; same questions, same language, same order of questions pilot a questionnaire ... to test that you have all the questions, are concise, easily understand by the research subjects, etc ...
Components of research:
what?: you know the question is specific enough if its clear what information you need to answer the question ... if you cant answer the what question you need to break down the question further. why? : the rationale. external rationale: what is the greater social value/goal; internal rationale .. what is in it for you? how? methodology ... who? where? what instruments etc. time line etc ... team vs individual etc.
Resource: “Developing effective research proposals.”
LITERATURE REVIEW Not necessarily theory ... but work that deal with the same problem you are concerned with; bring/develop analysis that lead to your own unique perspective .. so you don't replicate other existing work with the help of theory ... you formulate an hypotheses [but you don't need to have a hypothesis when you dont have an idea of what you might find] only use a hypotheses if you want to reflect back – i.e test - on a theory.
research questions ... guides the research ito themes/topics ...
all this is the conceptial/pre-empirical stage ... and precedes the empirical/design stage...
- specify questions and data needed to answer questions
- identify places where data can be foundations
... including how you are going to analyse he information
FORMULATING the RESEACH QUESTIONS ... what is the correct level of abstraction?
- start with local capacity to determine at what level you can do the research;
- can start with the big questions but consider your capacity and be more realistic
theory .... don't read more than 3 books ... they should cover most of the perspectives; use it to improve on the practical work lit review in the report is more extensive than in the proposal
e.g. “these are main trends ... and this is where i position myself in relation to the existing literature... you offer a critique .. or alternatively extend/apply the ideas to a new area (rural, developing country etc)
GEM research question? Correct formulation: under what circumstances can GEM be useful ... for ict policies, telecentres, etc ... rather than 'is GEM adaptable?'
Recommended reading: “The choreography of qualitative research design” - Valerie J. Janesik
- start with experience and observations rather than with a theory;
- scientific model not suitable for studyign human behaviour; human world different than natural world, more flexible
- analogy: free style dancing ... but within the parameters of the genre (e.g. modern dance) ... i.e. don't just move in any way you want/desire
“validity of research” is replaced by credibility of the research: that includes reflecting on why it makes sense to organise the information in the way you do, the conclusions you draw ... and whether the conclusion are validated by the community you studied .
Other ways of establishing credibility/validity in qualitative research:
- triangulation – assumption of one object
- crystalisation – an object with different dimensions/angles ... all of which are equally valid [reality is not fixed; each of the perspecives are subjective but valid; enter into a dialogue of perspectives – intersubjectivity]
RESOURCES: Quantitative Research Methods - “Statistics for people who (think they) hate statistics” - Neil J. Salkind
“Research design for Social Research” - de Vaus, Sage Publications
RESEARCH TRAINING - DAY 2
1. Kaca’s research proposal on “Sexualities, Gender, and Internet.”
We identified problems of scope – how to select an appropriate set of countries for a multi-country study, reduce the number of issues covered in the research question. -including end users in the study makes it an enormous project – must reduce the scope, maybe focus just on college students using facebook, for example. -having meeting with Ford Foundation at the end of Feb to discuss the proposal, feedback will inform next phase of research design.
-perhaps focus on sexual health workers in a range of countries, and see what the impact of content regulation has been on them. -Kaca: we can’t focus only on government, it is really about capricious ‘self-regulation’ by the private sector, primarily due to consumer pressure to ‘protect the children.’
Karen B – should we focus on how the filtering software works, who puts it together and runs it? – this could be an intervening factor in the selection of countries. Looking at this issue through a gender lens could be really interesting.
The proposal is still very open at this point, Ford Foundation just wanted something on ‘sexualities and internet.’
Ran suggested moving in two related but different directions, each strand could be developed independently or in a common proposal.
• impact of policies and censorship; what are the different perspectives on the notion of ‘harm’ on the internet? What are the various regulatory mechanisms, both technical and legal? What is the impact of these mechanisms in different areas where they are applied? What are the negative and positive aspects of this impact? • notions of sexuality in society at large; sexual practices and identity construction in online spaces. Select groups of people who use the internet, young people perhaps students, in each country, and start with an exploratory survey.
Children’s consumer rights – bombarded with embedded advertising in social networking sites, what are the ethical issues here? Discourses on sexual topics, from pornography to abstinence, all are discussed/ contested on social networking sites.
2. the ONI-Asia Project Activity Proposal:
• the language is far too complicated, and there is no clear research question. • the issue that Kaca raised yesterday – including both censorship and surveillance, which are two different issues. • Karen B – if you live in a culture of surveillance, you self-censor, so that is a relationship between the two. • Intersection with Kaca’s research – online and offline social behaviour. • Ran’s input: the format, there are issues from the get go, even the title covers too many topics / strategies. This proposal illustrates a very common problem of use of overly complicated language, very difficult to understand. Even theoretically sophisticated readers would prefer something simple and accessible, and not to be bombarded with ‘epistemologies’ and ‘conceptualizations.’ It excludes a large percentage of the potential audience, which is self-defeating. You must consider your intended audience, i.e. policymakers, and making your writing accessible to them. • Ran: The way I read the proposal is that it should focus on the fact that the gender dimensions of internet censorship and surveillance are hidden. We want to use this research to highlight them. The research question should focus on the ways in which these hidden dimensions are going to be uncovered. There is a really interesting research question in this proposal, it just needs to be extracted from this sea of jargon. • Karen B: there is another huge chunk of work proposed here, which is the development of a framework for further research, with a set of indicators and benchmarks for evaluating the state of the censorship and surveillance situation in a given country.
3. Discussion of discourse analysis – there is quite a bit of material on the subject in our reader.
There is a general distinction between discourse analysis and analysis of discourse.
• Discourse analysis is the study of language – it is quite technical, a very common mode of analysis in linguistics, literature, concerned with identifying hidden meanings, beyond the superficial. There are a lot of very technical concerns with the choice of words, phrasing, and so forth. Perhaps not very relevant to the policy research field.
• Analysis of discourse – looking at how people discuss a given subject, in order to identify the assumptions that underpin their perspectives and arguments. Relevant to policy analysis – underlying assumptions about the role of government, the meaning of ‘harm,’ ‘terrorist’ and other conceptions. Little to say about this in terms of concepts and methodology – it isn’t really a research methodology, more just analysis of content and the identification of ideologies and worldviews that ground the content. Ex: terrorism is illegitimate, but who has the power to define a terrorist, to lay out a terrorist’s characteristics and pass judgement on their actions.
• For example, there is always a hidden assumption that growth is good, progress is good, but why is it good? Will development and growth make people’s lives better automatically? What is the relationship between growth and happiness/fulfillment? In South Africa, the economy grows every year, but for normal people, most are becoming poorer.
• You can analyse the discourse in the ONI-Asia proposal, for example. You can go through and highlight words / phrases to be examined. Why is the word ‘epistemology’ in this proposal? Why not plain language? Are you trying to prove your academic credentials through the use of sophisticated language? Karen B brought up the example that many APCers use single quotes to underline our questioning of a concept, such as ‘harmful content’ or ‘information society.’
• Ran – within APC, often work is conducted in groups, but without an external perspective. Are we questioning our assumptions? We fall into a pattern of using our own language and shorthand – because our opinions are generally shared by our coworkers, no need to ground them explicitly.
• Karen B – is it useful to footnote these things? Ran – most people would understand that single quotes mean that you are questioning something, but they may not understand exactly what your objections are. You can also just be explicit in the text, for example explain why you question the notion of harmful content. You can use language in such a way as to position yourself politically, which APCers do all the time without maybe being conscious of it.
• Ran – the crucial point is that you are trying to reach a wider audience, so you must eliminate linguistic barriers such as overly academic language, jargon, acronyms in order to do this.
• Natasha – terms such as empowerment, I have stopped using that word, because although most readers have some conception of what it means, it might not necessarily be what you mean. Is it self-empowerment, are you empowering them?
• Ran – in order to conclude: its not that we have to simplify all of our language – there is more specialized terminology in certain fields for a reason. It’s a question of moderation rather than strict rules.
4. To what extent do specific groups of people from different backgrounds research differently? Are there specific ‘African’ or ‘female’ ways of conducting research?
• Beyond the obvious point that there are gender dimensions to everything, beyond the question of content, are there different ways of doing research if you are male or female, black or white, and so forth?
• Natasha – in the early days of feminist research, most of the research was done by women, so there was a conflation around the gendered content of research, and the gender of the researcher, which led to the question: can men do feminist research? But I think we have moved beyond that and accepted that there is a gendered way of researching that all can use.
• There is a related question that is relevant to APC as an international organization – is there a ‘European’ or ‘Latin American’ way to do research? Beyond the technical linguistic requirements, we kind of have to assume that we all do our work in the same way in order to function cohesively. Analia – where the knowledge is being produced is mainly in the North, and then we adopt and incorporate it? Ran – but does that mean that we are missing something if knowledge production is not indigenous?
5. Research briefs – what to look for when hiring researchers.
• Try to be very clear about the research questions that you put forward, what it is you want to find out and what information you will need. Articulate the methodology you wish to use.
• Sometimes it is difficult to be specific, but the more detailed your brief is, the more the product will reflect your needs.
• Maybe better to incorporate the researcher into the formulation of the research question. Specify timeline and specific outputs, without being too prescriptive.
• Karen B – we had problem with a consultant who didn’t read a reference document we had given them. Ran – maybe an interim report that would give you a preview, an opportunity to ask them to change direction. It’s not an unreasonable demand.
• Set up a cross-programme reference group for further discussion. Dip into the reader, use it as a reference. The focus on discourse analysis in the reader is actually not really what we need, because Kaca and others are more interested in the analysis of discourse.
• Karen B – something interesting about ‘harmful content’ – it is often used in the same phrase as ‘illegal content’, so it is really framed as the content that the powers that be would like to make illegal if they can/could.
• Natasha – I’ve distributed a framework for networked research from a document by Terri Willard, which could help us as we collaborate on the development of proposals. For the online space and the wiki, some of the things we might want to include are ‘virtual walk-in clinics,’ online discussions, and resource sharing.
• Can we develop the current online wiki space into this kind of platform? Karel – we can set up a new wiki, using media wiki. Dafne – then we could use this research wiki for other purposes, and involve non-APCers as well. Karel – how open do we want it to be? Staff and members, viewing and editing permissions? It can be set up in the next week by Adolfo and Karel.
• Analia – can we have a resource person to advise us on research design? That can be Ran, and perhaps also Debbie Budlender.