Monday, June 30, 2008

VA and mobile students

I've been trying to pull together how the different districts and states using VA measures for accountability are dealing with Mobile kids. The differences in how the issue is discussed in the three states currently using SAS's EVAAS for state-wide accountability. Bill Sanders has addressed the issue very clearly in his writings around the EVAAS model. The model being used by EVAAS can handle missing observations that are generated by mobile students. In all of the jurisdictions for which I can find documentation, there is a rule for who falls into the accountability system and his therefore included in a particular year's model. Students who do not meet the definition are excluded from the growth model.
  • Ohio Rules - (page 1 of ODE VA FAQ and FAQ at Battelle for Kids)
    "How does high attrition or mobility affect the value-added measure?

    Schools and districts are accountable for students enrolled at that school for a full academic year. Only students who are continuously enrolled from October count week through March testing will be included in the analysis. The Ohio Department of Education will match students test scores across years and schools using the SSID.

    The two FAQs seem to imply that mobile students are included, but only when they meet the above definition. It is possible that some non-zero percent of students fall out of the analysis every year.

  • Pennsylvania Rules - (there are more detail proposed rules here - but they do not differ on this issue)

    It seems that in Pennsylvania, the district - not the school - is accountable for the performance of students who do not attend for a "full academic year". As in the case of of Ohio, the EVAAS system explicitly takes into account missing data for individual children.

    (page 21 of the Pennsylvania Consolidated State Application Accountability Workbook)
    "Schools, LEAs and educational entities are accountable for mobile students in the same manner as they are for other students. The “full academic year” criteria are applied to all students. In Pennsylvania, it is not uncommon for students to move from one school to another within the same district during an academic year. In these instances, the school in which the student is enrolled at the time of the assessment bears responsibility for test administration; however, the district, rather than the school, will be accountable for the student’s performance."

  • Tennessee rules (several districts (pdf page 2 and other external sites) refer to 150 day enrollment requirement before the test )
My concern is for the implications this has for high mobility districts - particularly the large urban settings with high mobility rates. While I can see how techniques for dealing with missing data can be used to make good classroom and grade estimates, there might be a related incentive to not focus as consistently on the learning needs of mobile children - given the pressing needs of those fully included in the accountability system. A model that includes "dosage" or proportional assignment of student growth is what we should be shooting for. This would be relatively easy in a district in which most of the mobility is school-to-school within the district. However, as state-wide data system improve, it should be relatively easy to track student mobile within a state and get access to their test data and current school. A dose-based model does he math right and provides consistent incentives to school staff.

Chris

No comments: