Obama has made a statement that is rattling the test-makers, but
undoubtedlygettingsighs of relief from teachers, students and parents
who believe that the current testing mania in most schools, is not
helping the learning process. In fact, it may be a hindrance to good
teachers who fail to deliver the scores and students who may be learning
to take tests rather than important concepts.
Having worked extensively with test development including reliability and validity analysis – first for students at San Jose State, then later with employees at an insurance company – I can state with confidence that all these tests for students have not been validated. This means that there are no studies that indicate a relationship between higher test scores and whatever criteria they are meant to predict. Would it be better jobs? More money? Better quality of life? How about college or even high school graduation?
A validity study requires first the definition of criteria. Let’s say we want a test that will predict high school graduation – simple enough. We then set up a hypothesis that says something like: “Students scoring above a certain point on this test will have a significantly higher rate of high school graduation.” Significant means higher probability than random, and how much higher is pre-established.
When we design such a study, we have to include our method of tracking these students for at least 4 or more years and then identifying them by their previous test scores, once we know if they did or did not graduate. We may find a very significant relationship, or we may find little or no correlation, meaning the tests did not work for our purposes. But, this is the ONLY way to validate the tests.
Since students are now taking new tests, there has not been a chance to track them and compare with future data. With this, I feel it is important to consider these tests as data-gathering instruments and definitely not used as the basis for decision-making regarding student, teacher, school or district performance. If researchers can have access to these scores and track the students, we may have good validity studies in a few years – and THEN decide how to use them (or discard them).
In the meantime, I feel testing is critical and useful for measuring learning for each student as long as the results are used for feedback to improve the learning program for that student. We want to see growth and progress. And, we want the opportunity to step in when it does not occur. We may also want to compare aggregate scores between classes, schools, districts or countries to see if we can learn better ways of doing things, but the focus should be on improvements in scores, not whether they meet some external benchmark.
At Pinecone, we rely on results of the weekly tests in our booklets, along with homework and class performance. Does the student need more practice? An explanation of a particular concept? Or has the student mastered this concept and can comfortably move on.
While we do not have rigorous studies (because we do not have access to future grades and scores), we have plenty of anecdotal data that indicate the effectiveness of our programs in terms of several indicators of success: higher grades, honor rolls, graduations, post-grad work, and even great careers. You can check the testimonials on our website.
I would agree with Obama that we need to address the fanatic testing and misapplication of results. Hopefully, the Smarter Balanced Testing will yield good information to help each teacher provide, and each student receive an enhanced education.
Having worked extensively with test development including reliability and validity analysis – first for students at San Jose State, then later with employees at an insurance company – I can state with confidence that all these tests for students have not been validated. This means that there are no studies that indicate a relationship between higher test scores and whatever criteria they are meant to predict. Would it be better jobs? More money? Better quality of life? How about college or even high school graduation?
A validity study requires first the definition of criteria. Let’s say we want a test that will predict high school graduation – simple enough. We then set up a hypothesis that says something like: “Students scoring above a certain point on this test will have a significantly higher rate of high school graduation.” Significant means higher probability than random, and how much higher is pre-established.
When we design such a study, we have to include our method of tracking these students for at least 4 or more years and then identifying them by their previous test scores, once we know if they did or did not graduate. We may find a very significant relationship, or we may find little or no correlation, meaning the tests did not work for our purposes. But, this is the ONLY way to validate the tests.
Since students are now taking new tests, there has not been a chance to track them and compare with future data. With this, I feel it is important to consider these tests as data-gathering instruments and definitely not used as the basis for decision-making regarding student, teacher, school or district performance. If researchers can have access to these scores and track the students, we may have good validity studies in a few years – and THEN decide how to use them (or discard them).
In the meantime, I feel testing is critical and useful for measuring learning for each student as long as the results are used for feedback to improve the learning program for that student. We want to see growth and progress. And, we want the opportunity to step in when it does not occur. We may also want to compare aggregate scores between classes, schools, districts or countries to see if we can learn better ways of doing things, but the focus should be on improvements in scores, not whether they meet some external benchmark.
At Pinecone, we rely on results of the weekly tests in our booklets, along with homework and class performance. Does the student need more practice? An explanation of a particular concept? Or has the student mastered this concept and can comfortably move on.
While we do not have rigorous studies (because we do not have access to future grades and scores), we have plenty of anecdotal data that indicate the effectiveness of our programs in terms of several indicators of success: higher grades, honor rolls, graduations, post-grad work, and even great careers. You can check the testimonials on our website.
I would agree with Obama that we need to address the fanatic testing and misapplication of results. Hopefully, the Smarter Balanced Testing will yield good information to help each teacher provide, and each student receive an enhanced education.
No comments:
Post a Comment