Successful implementation of analytics

DMReview's 5 Principles of High-Impact Analytics applied to the development of systems for supporting longitudinal analysis of student data.

1. Recognize the Application Imperative
It is relatively easy to try and attack all aspects of decision support. One of the positive outcomes of NCLB is that many states and districts have used energy generated by the legislation to get people excited about using data to improve student learning. If teaching kids remains the critical focus of the analytics, success is more likely.

2. Democratize Information Assets
Many school districts have widely distributed decision making authority. Even in places where that is not the case, aides, teachers, and building leaders operate as relatively independent agencies. Decision support systems designed to support a few central office folks using annual data are not going to be received well. The lack of timely data is one of the common complaints heard from critics of district information system initiatives. Systems for schools and classrooms must be far more responsive if they are to find broad support.

3. Build Discipline in Decision-Making Processes
What the authors mean with this point is that there must be a link between the analysis provided by the system and well understood practices that affect the related outcomes. It must be clear that the analytics "have a place in the organization". Feedback that is uninterpretabil will be of little value.

4. Recognize New Skills Required for Knowledge Workers
Professional development for users of new decision support resources is vital. It will certainly cost far more than the system developed. This is another great reason to keep the scope of initial development narrow. This will keep training costs from blossoming out of control. New tools that no one has the resources to learn to use will undermine the effort.

5. Deal with Complexity: Closed-Loop, Adaptive Systems
The point of decisions support systems is to support improvement. This should lead to a virtuous circle in which analysis generated by the system is used to increase the quality of the data coming in and the related decisions. The system itself should be exposed to the same scrutiny. Lessons learned in early implementation should be incorporated as the system is extended into new areas.

Chris