wonder cow
01-07-2007, 01:49 PM
On Sources
One of the things I have noticed in this new age of the interweb, is that people are not as discriminating about sources as they should be.
When making an argument on any particular point, especially points that may be controversial, it is important not just to provide a source but a credible source.
Providing a poor source is the same or worse than providing none.
Although it is debatable about what constitutes a credible source, some sources are clearly not credible.
Here are some examples.
Bloggers
Although not automatically a poor source, anyone can spend 15 minutes and throw up a blog. Be very suspicious of any information you get from these quasi wannabe journalists. Do not be surprised if you use some blog as a source and someone challenges it.
"This morning I ordered a blueberry muffin from the coffee shop, but when I got to the office I realized they had given me a doughnut instead. By the way, 90% of death row inmates once worked for McDonald's."
Wikis
Wiki’s are not always wrong and not always a poor source. But we must exercise caution when citing wkikis. Anyone can register an account and add to or edit a wiki. This means two bad things. Firstly, any controversial issue is a target for manipulation. Secondly, the people providing the information may not really be as well informed on the subject as they think they are.
So, if the wiki article is about the Presidency of Grover Cleveland, it should be fairly safe. If the wiki article is about the Confederate States of America, you may want to proceed with caution, because there are many amongst the neo-confederate movement who are motivated to manipulate the history of that era.
"News" articles
Firstly, news articles may or may not be a legitimate source, depending on the subject. If the subject is some event that took place, a news article from a known newspaper may be sufficient. If the subject is something more technical, then the question of credibility falls on the credibility of the author as an expert in that area and/or on the sources the author himself provides.
If you provide a news article as a source, use a known, non-sensationalistic publication. The National Enquirer is no good. SomeRandomUnknownNewsSite.info is no good either. The New York Times or Washington Post or AP or your local paper are all much better.
Extremely Bias sources
Almost everything is bias. But it is a matter of degrees. The more bias, the less credible. The New York Times is more credible than moveon.org. The Washington Post is more credible than the drudge report. Pundits and political hacks rarely provide information that is not distorted. Anything from Bill O or M. Moore should be investigated further.
Research institutes
Sounds credible. And sometimes they are. But be aware of who funds them and what bias they hold. The American Heritage Foundation, for example, is a conservative group. It does not mean that everything they come out with is illegitimate. But it does mean we need to be careful when we concede or defend a point based on data from such organizations.
Experts and people with credentials
Not all experts are really experts. Not all people with credentials have good or relevant credentials. An expertise in one area does not lend credibility to me in another. Just because Nancy Grace and other cable news programs invite me on and put me forward as an expert does not make me a true expert. Cable news people have to fill up air time 24 hours per day, 7 days per week. We are all likely to be on cable "news" eventually.
If I claim to have a Masters Degree in some field, the first question should be "from where?". The second question should be, is the field relevant to the subject at hand. And note that even if I have a Masters from Harvard in toilet paper design, that does not mean that all my statements about toilet paper design are valid.
Polls
Any poll that starts with "this is not a scientific poll" is worthless. And most polls we see printed in papers and spewed across TV land are such polls and thus just more fodder to help fill up the insatiable mediaverse.
Furthermore, be careful about what the poll is actually revealing. Most of the time it is only revealing the attitudes or opinions of the respondents.
"90% of the people polled say George Bush is a jackass."
1 month later and a few wind changes:
"75% of the people polled say George Bush, although a jackass, is a hell of a nice guy"
I mean, really. Who cares?
"4 out of 5 dentist who have patients that chew gum, say George Bush is a jackass."
Statistics and causal inferences
Everyone loves statistics because they seem so legit. Statistics from sources that are not credible, are themselves not credible. If a source claims their statistics come from the FBI Uniform Crime Report, go to the FBI uniform crime report and verify.
What happens often is that the source in question manipulates the stats, takes them out of context, or flat out lies about them.
Furthermore, often foolish errors are made in the collection of the data and the methodology of the research on which the statistics are based. Even by competent unbiased experts. So, Ann Coulter’s "research" on media bias based on text searches of the lexus-nexus database just doesn't cut it.
Lastly, even if statistics are legitimate, be very careful of the conclusions you draw from these statistics. It is very easy to make assumptions about causation that are just not true and are unsupported by the data. What makes it worse, is that there are many unscrupulous individuals and political operatives and all around nut cases who understand this failing all to well and are more than happy to exploit it.
Example:
"Blacks have a higher percentage of high school drop outs than whites."
source:http://nces.ed.gov/pubs2002/droppub_2001/
Racists, for example, love statements like the above. But when you check this against income, you find another prevailing factor. Blacks have a higher rate of poverty, and so therefore have a higher drop out rate.
One of the things I have noticed in this new age of the interweb, is that people are not as discriminating about sources as they should be.
When making an argument on any particular point, especially points that may be controversial, it is important not just to provide a source but a credible source.
Providing a poor source is the same or worse than providing none.
Although it is debatable about what constitutes a credible source, some sources are clearly not credible.
Here are some examples.
Bloggers
Although not automatically a poor source, anyone can spend 15 minutes and throw up a blog. Be very suspicious of any information you get from these quasi wannabe journalists. Do not be surprised if you use some blog as a source and someone challenges it.
"This morning I ordered a blueberry muffin from the coffee shop, but when I got to the office I realized they had given me a doughnut instead. By the way, 90% of death row inmates once worked for McDonald's."
Wikis
Wiki’s are not always wrong and not always a poor source. But we must exercise caution when citing wkikis. Anyone can register an account and add to or edit a wiki. This means two bad things. Firstly, any controversial issue is a target for manipulation. Secondly, the people providing the information may not really be as well informed on the subject as they think they are.
So, if the wiki article is about the Presidency of Grover Cleveland, it should be fairly safe. If the wiki article is about the Confederate States of America, you may want to proceed with caution, because there are many amongst the neo-confederate movement who are motivated to manipulate the history of that era.
"News" articles
Firstly, news articles may or may not be a legitimate source, depending on the subject. If the subject is some event that took place, a news article from a known newspaper may be sufficient. If the subject is something more technical, then the question of credibility falls on the credibility of the author as an expert in that area and/or on the sources the author himself provides.
If you provide a news article as a source, use a known, non-sensationalistic publication. The National Enquirer is no good. SomeRandomUnknownNewsSite.info is no good either. The New York Times or Washington Post or AP or your local paper are all much better.
Extremely Bias sources
Almost everything is bias. But it is a matter of degrees. The more bias, the less credible. The New York Times is more credible than moveon.org. The Washington Post is more credible than the drudge report. Pundits and political hacks rarely provide information that is not distorted. Anything from Bill O or M. Moore should be investigated further.
Research institutes
Sounds credible. And sometimes they are. But be aware of who funds them and what bias they hold. The American Heritage Foundation, for example, is a conservative group. It does not mean that everything they come out with is illegitimate. But it does mean we need to be careful when we concede or defend a point based on data from such organizations.
Experts and people with credentials
Not all experts are really experts. Not all people with credentials have good or relevant credentials. An expertise in one area does not lend credibility to me in another. Just because Nancy Grace and other cable news programs invite me on and put me forward as an expert does not make me a true expert. Cable news people have to fill up air time 24 hours per day, 7 days per week. We are all likely to be on cable "news" eventually.
If I claim to have a Masters Degree in some field, the first question should be "from where?". The second question should be, is the field relevant to the subject at hand. And note that even if I have a Masters from Harvard in toilet paper design, that does not mean that all my statements about toilet paper design are valid.
Polls
Any poll that starts with "this is not a scientific poll" is worthless. And most polls we see printed in papers and spewed across TV land are such polls and thus just more fodder to help fill up the insatiable mediaverse.
Furthermore, be careful about what the poll is actually revealing. Most of the time it is only revealing the attitudes or opinions of the respondents.
"90% of the people polled say George Bush is a jackass."
1 month later and a few wind changes:
"75% of the people polled say George Bush, although a jackass, is a hell of a nice guy"
I mean, really. Who cares?
"4 out of 5 dentist who have patients that chew gum, say George Bush is a jackass."
Statistics and causal inferences
Everyone loves statistics because they seem so legit. Statistics from sources that are not credible, are themselves not credible. If a source claims their statistics come from the FBI Uniform Crime Report, go to the FBI uniform crime report and verify.
What happens often is that the source in question manipulates the stats, takes them out of context, or flat out lies about them.
Furthermore, often foolish errors are made in the collection of the data and the methodology of the research on which the statistics are based. Even by competent unbiased experts. So, Ann Coulter’s "research" on media bias based on text searches of the lexus-nexus database just doesn't cut it.
Lastly, even if statistics are legitimate, be very careful of the conclusions you draw from these statistics. It is very easy to make assumptions about causation that are just not true and are unsupported by the data. What makes it worse, is that there are many unscrupulous individuals and political operatives and all around nut cases who understand this failing all to well and are more than happy to exploit it.
Example:
"Blacks have a higher percentage of high school drop outs than whites."
source:http://nces.ed.gov/pubs2002/droppub_2001/
Racists, for example, love statements like the above. But when you check this against income, you find another prevailing factor. Blacks have a higher rate of poverty, and so therefore have a higher drop out rate.