Approaching Graphics, Avoiding Pitfalls
somatic research
By Ravensara S. Travillian
Originally published in Massage & Bodywork magazine, March/April 2009. Copyright 2009. Associated Bodywork and Massage Professionals. All rights reserved.
If a column chart can be thought of as a snapshot, then a line chart can be thought of as a movie--it can show how a trend forms.
In the last article ("Reading Charts and Graphs," Massage & Bodywork, January/February 2009, page 126), we began exploring how text and graphics fit together in describing the results of a study. Here, we will take a deeper look at kinds of visual representation and what they express, in order to see what information needs are best met by which kind of graphic. We will focus on three of the most important and commonly-used charts, using results from a fictional massage clinic.
Column Charts and Bar Charts
Column charts and bar charts are frequently used to compare different categories or individuals at one point--snapshots of how variables compare at particular moments. The height of the columns represents the values of the categories being compared.
Image 1 compares reported pain scores for five individuals in a massage clinic. The five individuals are listed along the x-axis--the horizontal line at the bottom of the graph. The y-axis--the vertical line at the left of the graph--shows the columns' values. To compare Terry's pain scores to Naomi's, you can look at the x-axis and find Terry's name. The red bar for Terry (the before-massage score) is at the height of 6.5 on the y-axis. Similarly, finding Naomi's name on the x-axis provides a red bar with a height of 8 on the y-axis. Naomi reported more pain (8) before the massage than Terry reported (6.5).
The corresponding blue bars (the after-massage scores), by contrast, show that Terry reported a 4, while Naomi reported a 3.5. So while Naomi reported more pain before the massage, she also reports more relief from the massage, in the form of a lower pain score. While these pain scores report each individual's experience, and thus cannot be compared directly to determine objectively who had more pain relief, we can say that, based on their self-reports of pain scores, Naomi reports more benefit from the massage than does Terry.
We can also make other comparisons between individuals from the graph above. For example, Hamid started out with more self-reported pain than anyone except Naomi, yet finished with the least self-reported pain. Hamid also had the most difference in before and after pain (5 units lower), followed by Naomi (4.5 units lower), Terry (2.5 units lower), and Rick and Isadora (1.5 units lower each). Other similar types of comparisons can be made with the data in Image 1. When you need to compare values among different individuals or different categories at particular points in time, a column or bar chart is a useful tool for visualizing that information.
Data in a column chart or bar chart can be represented vertically or horizontally. By switching the x-axis and the y--axis in Image 1, we would get a horizontal representation of what were vertical columns in Image 1, however, all the information would remain the same.
Line Charts
Line charts are a good way to display information about whether something is increasing, decreasing, or staying steady.
Image 2 shows trend lines for after-massage pain scores for a different series of treatments for two of the clients from the massage clinic, Isadora and Rick.
The usual convention for reading graphs is that time is represented as getting later as you read from left to right. So Treatment 1 is the leftmost category on the x-axis, and it proceeds through time to Treatment 6 as you read to the right. Similarly, toward the top of the page is usually "more" on the y-axis, and toward the bottom of the page is usually "less." Reading Isadora's and Rick's lines on the graph from left to right, we see that they both drop as you read from left to right, so both are reporting less pain over time after massage.
In other ways, however, they are rather different. Isadora's reported pain falls steadily after each massage, plateauing (remaining level) between Treatments 3 and 4, then decreasing again. The picture for Rick, however, is more complex--it falls, then rises, then falls again, and at the end of Treatment 6, is showing another rise. Overall, however, it has still fallen in comparison to where he started.
These are the kinds of observations you can make from line graphs: What does the researcher report that the data does over time? What are the trends in the data? In addition to overall trends from start to finish, what smaller trends occur in the middle of the data? Line graphs tell these stories over time.
Pie Charts
While column/bar and line charts compare different things, either at one point or over time, pie charts are a way of looking at one thing in detail, to see its various elements. Image 3 shows a pie chart representing treatment choices for low-back pain among a clinic population. To make the chart function, we're making an assumption that simplifies things--we assume that people use only one treatment and that everyone has a treatment choice. In that way, the percentages add up to 100 percent, and the pie chart is a good visual representation of the data. For this reason, you will often see the results of surveys presented as pie charts.
The relative size of the sections represents the percentage for each treatment in the pie chart--massage (blue) is the largest at 35 percent, and it has 35 percent of the area of the pie. At 27 percent of the population, herbal medicine (green) has 27 percent of the pie chart area, and so forth. Image 4 has exactly the same information as Image 3 has, but in a different style--it is three-dimensional, and the parts of the pie are "exploded" to separate them from each other, which makes them easier to see and adds visual interest.
Avoiding Common Pitfalls
To avoid common pitfalls in reading charts, ask yourself the following questions:
- Does the picture the chart draws match what the text says about the data?
- Does the chart make clear what trends exist in the data and what they mean?
- Does the size of the effect shown in the chart represent the real size of the effect correctly?
Inconsistencies
When large amounts of information are represented graphically side by side with descriptive text, inconsistencies and errors may be introduced. Watch for these sorts of problems by reading the text and comparing the graphic to the text. Often, if there is a discrepancy, you can figure out what is right from the context--for example, a table, text, and graphic all cite the same data, and two out of three agree, meaning the one that does not agree is probably the one that is wrong. When it is impossible to reconcile what the inconsistencies mean, you may contact the article's author (or corresponding author, if listed) with questions.
Statistical Errors
Averages are very useful because they quickly and efficiently sum up information about a group, but there are things to keep in mind when interpreting charts of averages. Image 5 shows the results of a fictional study and reports average pain scores on a scale from 1-8 (on the vertical y-axis) after each of a series of 8 massages (the horizontal x-axis).
On average, the chart shows a trend toward lowering pain scores, although that trend is not perfectly uniform for all members of the group after every massage, as shown by a couple of rises in the line as it moves from left to right. There are two pitfalls to avoid. First of all, the data is descriptive of what did happen, so we cannot extrapolate (infer) from it or project what will happen after the ninth massage and beyond. The second pitfall is assuming that the data can be mapped exactly to individuals in the group, as the rises in the line show, while the scores for the group as a whole tended to decrease over time, there were actually enough individual increases at times to impact the group scores. So while the line describes the group, it does not--and cannot--provide the reader enough information to make specific predictions about what will happen to particular individuals after future massages.
Illusions of Scale
A final pitfall to watch for is when the information is accurate, but the effect of presenting it visually is to make the outcomes or differences look far larger or far smaller than they are in reality. Image 6 illustrates a fictional study in which the researchers studied how use of massage correlates with educational levels. They found that in their sample, 82 percent of people who never finished a college degree used massage, compared to 77 percent of people who finished their undergraduate degree, and 79 percent of people who finished a graduate degree. They concluded that these 5 and 3 percentage point differences were not statistically significant, and that the usage in each group was effectively the same.
However, that is not the message conveyed by Image 6. At first glance, it appears that people with no college degree are more than twice as likely to use massage, based on the relative size of the first column compared to the last two--a very significant visual difference that falsely implies there is a correlation between educational level and massage use.
The reason? The vertical y-axis starts and ends at arbitrary points that do not take into account the large percentage that is the same among all the groups. In other words, the chart is just showing the tip of the iceberg to make it easier to see the differences between the groups. This is a legitimate presentation technique commonly used to emphasize differences in the media as well as in research, so you will have to use your judgment in many cases to fully understand what the chart is telling you. Where the boundary lies between legitimately showing differences and unreasonably exaggerating those differences is not always obvious.
Image 7, by contrast, is an example of what the same data would look like on a chart where the y-axis is adjusted to show the entire range between 0 and 100 percent, giving a more accurate representation of effect size and showing that there really is very little difference overall between the groups.
So when you see a graph that shows a visually large or small effect size, check it with the text, and check the units on both the x-axis and the y-axis to make sure that the visual effect is faithful to the results as they are reported in other places in the article.
Next Steps In Interpretation
These are three basic types of charts that you will find when reading massage research, as well as the types of questions they are intended to answer and the information needs they are intended to address. In the next article, we'll integrate the Results that we've been examining with the Discussion section. We'll see how the Discussion section ties together the just-the-facts emphasis of the Results section with imagination, and interpretation, and how--still tied to the empirical reality--researchers use this integration to derive meaning from those study results and to translate them into recommendations for real-life practice.
Ravensara S. Travillian is a massage practitioner and biomedical informatician in Seattle, Washington. She has practiced massage at the former Refugee Clinic at Harborview Medical Center and in private practice. In addition to teaching research methods in massage since 1996, she is the author of an upcoming book on research literacy in massage. Contact her at researching.massage@gmail.com with questions and comments.
If a column chart can be thought of as a snapshot, then a line chart can be thought of as a movie--it can show how a trend forms.
In the last article ("Reading Charts and Graphs," Massage & Bodywork, January/February 2009, page 126), we began exploring how text and graphics fit together in describing the results of a study. Here, we will take a deeper look at kinds of visual representation and what they express, in order to see what information needs are best met by which kind of graphic. We will focus on three of the most important and commonly-used charts, using results from a fictional massage clinic.
Column Charts and Bar Charts
Column charts and bar charts are frequently used to compare different categories or individuals at one point--snapshots of how variables compare at particular moments. The height of the columns represents the values of the categories being compared.
Image 1 compares reported pain scores for five individuals in a massage clinic. The five individuals are listed along the x-axis--the horizontal line at the bottom of the graph. The y-axis--the vertical line at the left of the graph--shows the columns' values. To compare Terry's pain scores to Naomi's, you can look at the x-axis and find Terry's name. The red bar for Terry (the before-massage score) is at the height of 6.5 on the y-axis. Similarly, finding Naomi's name on the x-axis provides a red bar with a height of 8 on the y-axis. Naomi reported more pain (8) before the massage than Terry reported (6.5).
The corresponding blue bars (the after-massage scores), by contrast, show that Terry reported a 4, while Naomi reported a 3.5. So while Naomi reported more pain before the massage, she also reports more relief from the massage, in the form of a lower pain score. While these pain scores report each individual's experience, and thus cannot be compared directly to determine objectively who had more pain relief, we can say that, based on their self-reports of pain scores, Naomi reports more benefit from the massage than does Terry.
We can also make other comparisons between individuals from the graph above. For example, Hamid started out with more self-reported pain than anyone except Naomi, yet finished with the least self-reported pain. Hamid also had the most difference in before and after pain (5 units lower), followed by Naomi (4.5 units lower), Terry (2.5 units lower), and Rick and Isadora (1.5 units lower each). Other similar types of comparisons can be made with the data in Image 1. When you need to compare values among different individuals or different categories at particular points in time, a column or bar chart is a useful tool for visualizing that information.
Data in a column chart or bar chart can be represented vertically or horizontally. By switching the x-axis and the y--axis in Image 1, we would get a horizontal representation of what were vertical columns in Image 1, however, all the information would remain the same.
Line Charts
Line charts are a good way to display information about whether something is increasing, decreasing, or staying steady.
Image 2 shows trend lines for after-massage pain scores for a different series of treatments for two of the clients from the massage clinic, Isadora and Rick.
The usual convention for reading graphs is that time is represented as getting later as you read from left to right. So Treatment 1 is the leftmost category on the x-axis, and it proceeds through time to Treatment 6 as you read to the right. Similarly, toward the top of the page is usually "more" on the y-axis, and toward the bottom of the page is usually "less." Reading Isadora's and Rick's lines on the graph from left to right, we see that they both drop as you read from left to right, so both are reporting less pain over time after massage.
In other ways, however, they are rather different. Isadora's reported pain falls steadily after each massage, plateauing (remaining level) between Treatments 3 and 4, then decreasing again. The picture for Rick, however, is more complex--it falls, then rises, then falls again, and at the end of Treatment 6, is showing another rise. Overall, however, it has still fallen in comparison to where he started.
These are the kinds of observations you can make from line graphs: What does the researcher report that the data does over time? What are the trends in the data? In addition to overall trends from start to finish, what smaller trends occur in the middle of the data? Line graphs tell these stories over time.
Pie Charts
While column/bar and line charts compare different things, either at one point or over time, pie charts are a way of looking at one thing in detail, to see its various elements. Image 3 shows a pie chart representing treatment choices for low-back pain among a clinic population. To make the chart function, we're making an assumption that simplifies things--we assume that people use only one treatment and that everyone has a treatment choice. In that way, the percentages add up to 100 percent, and the pie chart is a good visual representation of the data. For this reason, you will often see the results of surveys presented as pie charts.
The relative size of the sections represents the percentage for each treatment in the pie chart--massage (blue) is the largest at 35 percent, and it has 35 percent of the area of the pie. At 27 percent of the population, herbal medicine (green) has 27 percent of the pie chart area, and so forth. Image 4 has exactly the same information as Image 3 has, but in a different style--it is three-dimensional, and the parts of the pie are "exploded" to separate them from each other, which makes them easier to see and adds visual interest.
Avoiding Common Pitfalls
To avoid common pitfalls in reading charts, ask yourself the following questions:
- Does the picture the chart draws match what the text says about the data?
- Does the chart make clear what trends exist in the data and what they mean?
- Does the size of the effect shown in the chart represent the real size of the effect correctly?
Inconsistencies
When large amounts of information are represented graphically side by side with descriptive text, inconsistencies and errors may be introduced. Watch for these sorts of problems by reading the text and comparing the graphic to the text. Often, if there is a discrepancy, you can figure out what is right from the context--for example, a table, text, and graphic all cite the same data, and two out of three agree, meaning the one that does not agree is probably the one that is wrong. When it is impossible to reconcile what the inconsistencies mean, you may contact the article's author (or corresponding author, if listed) with questions.
Statistical Errors
Averages are very useful because they quickly and efficiently sum up information about a group, but there are things to keep in mind when interpreting charts of averages. Image 5 shows the results of a fictional study and reports average pain scores on a scale from 1-8 (on the vertical y-axis) after each of a series of 8 massages (the horizontal x-axis).
On average, the chart shows a trend toward lowering pain scores, although that trend is not perfectly uniform for all members of the group after every massage, as shown by a couple of rises in the line as it moves from left to right. There are two pitfalls to avoid. First of all, the data is descriptive of what did happen, so we cannot extrapolate (infer) from it or project what will happen after the ninth massage and beyond. The second pitfall is assuming that the data can be mapped exactly to individuals in the group, as the rises in the line show, while the scores for the group as a whole tended to decrease over time, there were actually enough individual increases at times to impact the group scores. So while the line describes the group, it does not--and cannot--provide the reader enough information to make specific predictions about what will happen to particular individuals after future massages.
Illusions of Scale
A final pitfall to watch for is when the information is accurate, but the effect of presenting it visually is to make the outcomes or differences look far larger or far smaller than they are in reality. Image 6 illustrates a fictional study in which the researchers studied how use of massage correlates with educational levels. They found that in their sample, 82 percent of people who never finished a college degree used massage, compared to 77 percent of people who finished their undergraduate degree, and 79 percent of people who finished a graduate degree. They concluded that these 5 and 3 percentage point differences were not statistically significant, and that the usage in each group was effectively the same.
However, that is not the message conveyed by Image 6. At first glance, it appears that people with no college degree are more than twice as likely to use massage, based on the relative size of the first column compared to the last two--a very significant visual difference that falsely implies there is a correlation between educational level and massage use.
The reason? The vertical y-axis starts and ends at arbitrary points that do not take into account the large percentage that is the same among all the groups. In other words, the chart is just showing the tip of the iceberg to make it easier to see the differences between the groups. This is a legitimate presentation technique commonly used to emphasize differences in the media as well as in research, so you will have to use your judgment in many cases to fully understand what the chart is telling you. Where the boundary lies between legitimately showing differences and unreasonably exaggerating those differences is not always obvious.
Image 7, by contrast, is an example of what the same data would look like on a chart where the y-axis is adjusted to show the entire range between 0 and 100 percent, giving a more accurate representation of effect size and showing that there really is very little difference overall between the groups.
So when you see a graph that shows a visually large or small effect size, check it with the text, and check the units on both the x-axis and the y-axis to make sure that the visual effect is faithful to the results as they are reported in other places in the article.
Next Steps In Interpretation
These are three basic types of charts that you will find when reading massage research, as well as the types of questions they are intended to answer and the information needs they are intended to address. In the next article, we'll integrate the Results that we've been examining with the Discussion section. We'll see how the Discussion section ties together the just-the-facts emphasis of the Results section with imagination, and interpretation, and how--still tied to the empirical reality--researchers use this integration to derive meaning from those study results and to translate them into recommendations for real-life practice.
Ravensara S. Travillian is a massage practitioner and biomedical informatician in Seattle, Washington. She has practiced massage at the former Refugee Clinic at Harborview Medical Center and in private practice. In addition to teaching research methods in massage since 1996, she is the author of an upcoming book on research literacy in massage. Contact her at researching.massage@gmail.com with questions and comments.
