An important aspect of the production of interactive learning material is the evaluation of the material, both formative evaluation to improve the material before a release, and summative evaluation, to compare its learning effectiveness with other compatible material. Unfortunately funds limit what is possible.
With the Scientific Reasoning Series no funds for summative evaluation have been available. We did conduct extensive formative evaluations with students, both in schools, and in public libraries. Public library testing was valuable in revealing motivationally weak sections. We recorded where many people left. These motivationally weak sections were reworked and retested. Through this empirical technique we can develop material that holds student attention. Many sections were improved, using computer stored information. Now the programs deal successfully with a wide range of students.
One important aspect of formative evaluation is improving the response of the program to free-form student replies to questions. We saw responses that could not be analyzed, suggesting new tests that were added to the program.
We would not, however, claim that the materials have attained perfection. There is always room for improvement.