The Perils of Arbitrary and False Precision

[selil]

When I am assessing the amount of information that students have ended up with in a class (based on all of the learning methods utilized and all experiences in and out of class) and based on the learning literacy of the assessment I can find nice gaussian bell curves. Some students will be bulls eyes with accuracy and reliability. Some students will be reliably wrong. Some will hit all over the target. If students who are otherwise reliable and accurate get something wrong I look to see if it was graded wrong or I taught it wrong. That is how I use accuracy and reliability.

Unfortunately the bell curve has limited utility in much of my research. I deal with binary data that the outliers are the important element. Averages have little in relationship to the rest of the environment. Myself I don't believe very much in trend analysis or other predictive methods. Only that which can be observed. Sure we all do it and it is fun, but prediction even with high reliability is rarely scientific.

[Ron Humphrey]

When I am assessing the amount of information that students have ended up with in a class (based on all of the learning methods utilized and all experiences in and out of class) and based on the learning literacy of the assessment I can find nice gaussian bell curves. Some students will be bulls eyes with accuracy and reliability. Some students will be reliably wrong. Some will hit all over the target. If students who are otherwise reliable and accurate get something wrong I look to see if it was graded wrong or I taught it wrong. That is how I use accuracy and reliability.

Unfortunately the bell curve has limited utility in much of my research. I deal with binary data that the outliers are the important element. Averages have little in relationship to the rest of the environment. Myself I don't believe very much in trend analysis or other predictive methods. Only that which can be observed. Sure we all do it and it is fun, but prediction even with high reliability is rarely scientific.

In relation to Predictive methods but possibly in a different context than you might expect. At its base would it not be reasonable to suggest that that which has often been considered predictive would actually be more acurately referred to as recognitive. By this I mean it seeks to look for similar characteristics to that which it has seen before and simply infer within acceptable bounds to attempt to approach an end solution through that lense.

I think about computer viruses and how although many may differ there are always similarities which if taken as a whole can eventually help to define the actual virus itself and even possibly from whence it came. Same with DOS attacks although they may come in different forms the ability to recognize and react to them allows for an almost predictive quality to ones preparations for such attacks.

Or how about finance how many types of applications exist which can at least in some format provide "good enough" answers to provide international level entities to make decisions on how to press forward and stay away from given actions.

Long and Short
I'm not quite sure there's really so much wrong with reasonable predictions based on known historical factors, rather that those predictions should never be blindly followed with upfront expectations that you don't know what you don't know until you get there

[selil]

Long and Short
I'm not quite sure there's really so much wrong with reasonable predictions based on known historical factors, rather that those predictions should never be blindly followed with upfront expectations that you don't know what you don't know until you get there

excellent.

[Van]

Seli, could you define 'reliability' in this context, please?

I hope I am misunderstanding your usage, it sounds like a variation on the accuracy/precision thing.

Also, your gaussian curve of answers makes me antsy. Clustering around different possibilities invites more productive discussion.

[selil]

Seli, could you define 'reliability' in this context, please?

I hope I am misunderstanding your usage, it sounds like a variation on the accuracy/precision thing.

Also, your gaussian curve of answers makes me antsy. Clustering around different possibilities invites more productive discussion.

It is a variation on the accuracy/precision element.

Reliability/repeatability does not have to be a bulls eye. It can be the same spot on the target but a clear miss. It has an unfortunate tendency to be related to causation but is more indicative of clustering.

The gaussian curve will occur in a large enough set of classes. I have NEVER had a perfect bell with one course of 20 students. I'm just not that consistent and the students, time of day, and so many other variables just keep it from happening. But, feeding five or six courses with an n+100+ and looking at that the curve looks like Monte Carlo data. I do have a course that has a dumbbell shape to them. F's and A's. That is because it has one project and you either pass or you don't. Some don't.

[120mm]

Seli, could you define 'reliability' in this context, please?

I hope I am misunderstanding your usage, it sounds like a variation on the accuracy/precision thing.

Also, your gaussian curve of answers makes me antsy. Clustering around different possibilities invites more productive discussion.

Van,

I like this website for all my definitional needs:

http://www.socialresearchmethods.net/kb/reliable.php

Drill up or down for a 'splanation of what you need.

[selil]

Van,

I like this website for all my definitional needs:

http://www.socialresearchmethods.net/kb/reliable.php

Drill up or down for a 'splanation of what you need.

And they even have the targets http://www.socialresearchmethods.net/kb/relandval.php

[davidbfpo]

For those who are interested in 'Black Swans' here is French analyst's review:http://www.redanalysis.org/2013/01/2...-of-foresight/

Not that I'm obsessed with Mali; there is a link to a forecast of a coup in Mali in 2012 correctly, after some number crunching:http://dartthrowingchimp.wordpress.c...asts-for-2013/