Developing your Research Methodology
Your methodology should contain the following parts or answer the following questions:
- What type of study are you proposing?
- What variables will you be measuring?
- What is your hypothesis?
- What types of data will you be using?
- What are the components of your study (research questions, Units of analysis)
Submit a document which outlines your proposed methodology. You can use the responses from the questions presented above as a template for how your document will be organized.
Here is some background information to help you format your Methodology document:
Research Questions and Research Design
All research starts with one or more research questions. These are the questions that you want to answer in your research study. For example, you might want to find out why some people vote Democrat and others vote Republican. Or you might want to find out why some people don’t vote at all. Another question you might want to try to answer is why some favor same-sex marriage and others oppose it.
There are lots of ways that we might go about trying to answer these questions. Some might rely on what their friends or family tell them. Others might rely on what people in authority like their religious leaders tell them. Still others might use what is often called common sense to answer these questions. But we’re going to use the scientific approach to try to answer these questions. Thomas Sullivan defined science as a “method of obtaining knowledge about the world through systematic observations.” Notice that science is empirical; it’s based on observations. Also, notice that we’re talking about a particular type of observations – systematic observations.
A research design is your plan of action. It lays out how you plan to go about answering your questions. The research design includes how you plan to select the cases for analysis (sampling), how you will measure concepts, how you plan to collect your data, and how you will analyze the data. Exercises two through five focus on the components of a research design and exercises six through thirteen deal with data analysis.
First, we have to learn how to formulate good research questions. Let’s start by looking at some examples of poor questions. Why are these poor questions?
- Women are more likely than men to vote Democrat in presidential elections. This one is easy. It’s not a question.
- Why are women more likely than men to vote Democrat in presidential elections? This one is a little more difficult. We want to start with the more general question such as why some people vote Democrat and others vote Republican? Then we would consider possible answers to this question. One of these answers might be that gender influences voting. Since science is empirical, we would start by looking at data to see if, in fact, gender does influence voting and we would discover that in most recent presidential elections women are more likely to vote Democrat. This would lead us to ask why women are more likely than men to vote Democrat. But we would start our study with the more general question.
- Why do dogs bark? This is certainly a question and perhaps an interesting question. But it’s not a question that social scientists would be interested in. Social scientists focus on questions that involve behavior, attitudes, and opinions
- What are the characteristics of a good research question?
- We start by looking at general questions such as what influences voting or why do some people favor same-sex marriage and others oppose it. As our study progresses, we move to more focused questions such as why women are more likely to vote Democrat than men.
- We focus on questions that ask about behavior, attitudes, and opinions.
- Good questions are clearly stated. Questions such as what about voting aren’t clear and therefore aren’t useful.
- As with everything we write, we want to make sure that we use correct spelling and good grammar. So proofread everything you write including your questions.
Sampling
Populations are the complete set of individuals that we want to study. For example, a population might be all the individuals that live in the United States at a particular point in time. The U.S. does a complete enumeration of all individuals living in the United States every ten years (i.e., each year ending in a zero). We call this a census. Another example of a population is all the students in a particular school or all college students in your state. Populations are often large and it’s too costly and time consuming to carry out a complete enumeration. So what we do is to select a sample from the population where a sample is a subset of the population and then use the sample data to make an inference about the population.
There are many different ways to select samples. Probability samples are samples in which every individual in the population has a known, non-zero, chance of being in the sample (i.e., the probability of selection). This isn’t the case for non-probability samples. An example of a non-probability sample is an instant poll which you hear about on radio and television shows. A show might invite you to go to a website and answer a question such as whether you favor or oppose same-sex marriage. This is a purely volunteer sample and we have no idea of the probability of selection.
There are a number of different ways of selecting a probability sample.
- The most basic type of probability sample is the simple random sample where every individual in the population has the same chance of being in the sample.
- Samples can also be stratified.
- A proportional stratified random sample is one in which the sample is selected such that the sample has the same proportion on key variables as does the population. For example, 51% of the nation is female and 49% is male. The sample could be stratified on sex in such a way that 51% of the sample is female and 49% is male.
- A disproportional stratified random sample is one in which the sample is selected such that we oversample some segments and undersample other segments of the population. For example, we might undersample whites and oversample non-whites so that our sample is 50% whites and 50% non-whites. This would be useful if we wanted to compare whites and non-whites and wanted to have a larger sample of non-whites for comparison purposes.
Notice that simple random samples and stratified random samples assume that we have a list of the population from which to select our sample. But what if we don’t have such a list? For example, how would we get a sample of high school seniors? There is no list available. But there is a list of all high schools in the United States. So we could select a sample of high schools and then within each high school in our sample select a sample of seniors. This is called a cluster sample because high schools are the clusters where you find seniors.
No sample is ever a perfect representation of the population from which the sample is drawn. This is because every sample contains some amount of sampling error. Sampling error in inevitable. There is always some amount of sampling error present in every sample. The question then is how can we reduce sampling error?
- One way is to increase the sample size. The larger the sample size, the less the sampling error. A simple random sample of 400 will have half the sampling error that a simple random sample of 100 has. To reduce the amount of sampling error by half, you have to quadruple the sample size.
- Stratifying a sample is another way that you can reduce sampling error assuming that the variable you use to stratify the sample is related to whatever you are studying. For example, if you are trying to explain why some people favor same-sex marriage and others oppose it, then you could stratify your sample by sex. Assuming that sex is related to how people feel about same-sex marriage (and it is), this will reduce sampling error.
Sampling is an important component of any research design. You need to carefully think about how you plan to select the cases for your research study.
Measurement
Let’s say that we want to explain support or opposition to same-sex marriage and that we think religion might be related to how people feel about same-sex marriage. We can distinguish between two different dimension of religion – religious preference and religiosity. That means that we’re dealing with three different concepts. Our concepts are:
- support or opposition to same-sex marriage,
- religious preference, and
- religiousity
Concepts can be defined as the abstract ideas that we want to use in our study. Another way to think about concepts is to view them as the tools we’re going to use to try to answer our research questions. Imagine that you go to the dentist. The dentist has a lot of tools to take care of your teeth but not all tools are appropriate. A chain saw is a tool but you wouldn’t want to see a chain saw in your dentist’s office.
Concepts have to be defined. There are two different ways to define concepts.
First, there is the theoretical definition. This answers the question – what do we mean by these concepts.
- Religious preference refers to the religion with which a person identifies. For example, some people identify themselves as Roman Catholic, others as Lutheran, others as Jewish, and still others as Muslim.
- Religiosity refers to how religious a person is. Two individuals could identify themselves as Roman Catholic but one might be much stronger in their religion than the other.
- Opposition or support for same-sex marriage is obvious. Do people define themselves as favoring or opposing same-sex marriage?
Second, there is the operational definition. How do we measure these concepts? What are the operations we go through to measure the concepts?
- Religious identification could be measured by asking people a question such as the following: “What is your present religion, if any? Are you Protestant, Roman Catholic, Mormon, Orthodox such as Greek or Russian Orthodox, Jewish, Muslim, Buddhist, Hindu, atheist, agnostic, something else, or nothing in particular? Concepts can be measured in different ways. Religiosity could be measured by asking people how often they attend religious services, how often they pray, and how important their religion is to them. Here are some questions that have been used in different surveys?
- "Aside from weddings and funerals, how often do you attend religious services... more than once a week, once a week, once or twice a month, a few times a year, seldom, or never?”
- “About how often do you pray?” Categories are several times a day, once a day, several times a week, once a week, less than once a week, never.
- “Would you call yourself a strong [insert religious preference] or a not very strong [insert religious preference]?
- Here’s a question from the 2014 Pew Political Polarization Survey that was used to measure how people feel about same-sex marriage. “Do you strongly favor, favor, oppose, or strongly oppose allowing gays and lesbians to marry legally?”
Your research design should identify the concepts that you want to use in your study and both your theoretical and operational definitions of these concepts.
