NSBA's Blog summarizes how parents feel about testing

BoardBuzz posting a link to an Associated Press Poll about issues in educational problems and possible solutions. One of the interesting findings is the gap between fairly solid support for tests as instruments of school accountability but a belief that classroom work better represented student learning.

Chris

Collaborative provide V-A Training Support in Ohio

The Ohio Resource Center is an interesting collaborative that combines scholars from the different schools of education in in Ohio and is supported by the legislature and Ohio Board of Regents. The "about" page describes the Center as follows:

ORC's resources are available primarily via the web and are coordinated with other state and regional efforts to improve student achievement and teacher effectiveness in preK–12 mathematics, science, and reading. The website is organized around Ohio's academic content standards.

What I find interesting, as a V-A guy, is that the Center has also developed a three module set of lessons on understanding value-added outcome metrics. It builds up knowledge about assessments and their uses, in general, and then goes on to describe how one should use V-A metrics.

A big thumbs up for the ORC!

Chris

Science Cannot Be Secret

This blog post does a great job laying out some of the core concerns about complexity and proprietary information in accountability systems. Our mantra has been "simple is better, unless it's wrong". In the VA work we do, we might prefer a simpler model that has "strong" assumptions. However, we think it is important to develop model enhancements that deal with cases in which the assumptions cannot be met. We look at the results of both models to see what difference violating the assumption makes. I very much agree with the author of this blog that this is a policy decision and that the educational agency (state, local, etc.) must be engaged with the model design and be able to understand and explain any and all modeling choices.

Chris

Value-Added doesn't get any more real than this

This is the face of classroom value-added on the ground. The Ohio Department of Education and Battelle for Kids (who provides training for the interpretation of VA scores) have used classroom level scores to identify teachers who produce large gains in student test scores.

Chris

VA and mobile students

I've been trying to pull together how the different districts and states using VA measures for accountability are dealing with Mobile kids. The differences in how the issue is discussed in the three states currently using SAS's EVAAS for state-wide accountability. Bill Sanders has addressed the issue very clearly in his writings around the EVAAS model. The model being used by EVAAS can handle missing observations that are generated by mobile students. In all of the jurisdictions for which I can find documentation, there is a rule for who falls into the accountability system and his therefore included in a particular year's model. Students who do not meet the definition are excluded from the growth model.

• Ohio Rules - (page 1 of ODE VA FAQ and FAQ at Battelle for Kids)
  "How does high attrition or mobility affect the value-added measure?
  Schools and districts are accountable for students enrolled at that school for a full academic year. Only students who are continuously enrolled from October count week through March testing will be included in the analysis. The Ohio Department of Education will match students test scores across years and schools using the SSID.
  
  The two FAQs seem to imply that mobile students are included, but only when they meet the above definition. It is possible that some non-zero percent of students fall out of the analysis every year.
  
  
• Pennsylvania Rules - (there are more detail proposed rules here - but they do not differ on this issue)
  
  It seems that in Pennsylvania, the district - not the school - is accountable for the performance of students who do not attend for a "full academic year". As in the case of of Ohio, the EVAAS system explicitly takes into account missing data for individual children.
  
  (page 21 of the Pennsylvania Consolidated State Application Accountability Workbook)
  "Schools, LEAs and educational entities are accountable for mobile students in the same manner as they are for other students. The “full academic year” criteria are applied to all students. In Pennsylvania, it is not uncommon for students to move from one school to another within the same district during an academic year. In these instances, the school in which the student is enrolled at the time of the assessment bears responsibility for test administration; however, the district, rather than the school, will be accountable for the student’s performance."
  
  
• Tennessee rules (several districts (pdf page 2 and other external sites) refer to 150 day enrollment requirement before the test )

My concern is for the implications this has for high mobility districts - particularly the large urban settings with high mobility rates. While I can see how techniques for dealing with missing data can be used to make good classroom and grade estimates, there might be a related incentive to not focus as consistently on the learning needs of mobile children - given the pressing needs of those fully included in the accountability system. A model that includes "dosage" or proportional assignment of student growth is what we should be shooting for. This would be relatively easy in a district in which most of the mobility is school-to-school within the district. However, as state-wide data system improve, it should be relatively easy to track student mobile within a state and get access to their test data and current school. A dose-based model does he math right and provides consistent incentives to school staff.

Chris