Step 5: Coding.
Once you have developed a coding framework, you can begin the actual data collection process. By the end of this step you will know more about your variable of interest than you ever wanted to. There are a few supplies you may want to acquire before beginning this process:
-
A set of different colored highlighters.
-
These are helpful for transferring the information from each article into your spreadsheet.
-
The process goes much faster when you assign a specific color to each piece of information you need to extract from the article. Then, as you read the paper, you can mark it with the various colors, and easily find this information again when it comes time to enter the study's data into the coding spreadsheet.
-
-
A stack of post-its.
-
As you will most likely not remember the specifics of every paper you read, it is helpful to stick a post-it on the front of each paper you read with a brief description of the information contained in that article and/or any unique features.
-
-
An unlimited supply of printer paper and ink.
As you begin to code, it is extremely important that you read each article carefully so as to not miss any relevant information. Enter the data from each study into the corresponding column as you code. In addition, take special note of whether data is reported as means +/- SD or SEM so that you can report the variance in the appropriate column. It is also helpful to mark cells without data as either NR (not reported) or NA (not applicable) in order to indicate that you looked for that data and could not find it. Do not be afraid to ask questions throughout this process, and work together at first to ensure the entire research team is coding articles in the same manner.
DIVIDING STUDIES INTO MULTIPLE EFFECTS:
Perhaps the most complicated aspect of the coding process is the concept of dividing one study into multiple effects. "Effect" is a meta-analytic term used to describe a row of data. Each row corresponds to an article; there can be multiple effects in one article, but not multiple articles in one effect.
Several factors influence the number of effects that can be derived from a single article:
1. Whether or not you need a control group for comparison.
2. How much data the article reports.
If your variable of interest requires a control group (see Analysis: Step 1), then the number of possible effects from a study is limited by the number of corresponding controls. For example, If you are interested in studying the effects of exercise on energy and fatigue, each exercise intervention must be compared with a control group. Thus, if you have a study looking at the varying effects of exercise, diet + exercise, and doing nothing, you can only code one relevant effect (not including the control group) because you have no way of dividing out the interacting effects of diet combined with exercise. However, if this same study also included a diet only group, you would then code two relevant effects because you can compare the exercise group to the nothing group, and the diet + exercise group to the diet only group.
Conversely, if you do not need a control group for each effect, the number of effects you can derive from any particular study is only limited by the amount of data they report. For example, if the study tests both men and women but does not report their data regarding your variable of interest separately, you will only have one effect. However, if they do divide the data into two groups, you can code two effects.
REJECTING ARTICLES:
Articles that have made it past the initial round of collection should not be rejected without careful consideration. It is important to read the entire article before making a decision on its inclusion/exclusion status.
At times, you may be tempted to reject an article because they do not report a part of the necessary information. Depending on your exclusion criteria, you may be able to do this. However, if the information is reported in a graph, you can estimate the values using a technique described in more detail HERE. It may also be possible to calculate the necessary data from reported p or t values. In addition, there is always the option of emailing the corresponding author listed on the paper to request the necessary information. If the author never responds, you may exclude the study in good conscience, knowing you have done your part to try and include it. When you do reject articles, be sure to keep a record of which ones were rejected and why. This can be included in the spreadsheet you used to collect articles.