How Do We Improve Data Analysis Skill?

It seems coming from education that we should have a rather simple answer to this question (it would likely include some rubric and self-assessment). The reality is that practice is the only sure way to improve analytic skill among administrators and teachers.

In grad school I would work feverishly to collect data, scrub the data, run descriptive and multivariate tests, and generate as many possible scatter plots as my Pentium I computer could handle. I would print my "results" and stare with black pen in hand to mark the compelling patterns. Without fail my advisor would come and join me for these analytic sessions. The story seemed the same every time: (1) he would set aside the statistical analysis suggesting that there was nothing to see there, (2) he would look at the patterns I identified (usually strong positive or negative correlations) and politely point out that there was nothing interesting in the auto correlations I observed, and (3) finally, he would draw odd shaped circles around groups of data on the scatter plots. These circles usually looked like amoebas, but definitely the patterns I had learned about in statistics class. He would then say, "I want to know more about these cases, try running this same relationship, but hold some variable constant." He found a compelling pattern on almost every printout. When I returned for more analysis I almost always found something complex, but compelling in the way he suggested I re-explore.

My advisor (Lewis Binford) never taught me to analyze data. He engage me and encouraged me to be curious. I learned by watching and practicing.

A culture of high-quality data analysis depends on leadership. It depends on high expectations for all members of the culture (teachers and administrators). Administrators must accept the mantle to be data analysis leaders. To be more prepared than their staff. Administrators need not come with the answers, but with the questions. Administrators need to nurture teachers and challenge them to be curious. Administrators should lead individuals to be kick-ass analysts, not simply facilitate the group to agreement on data analysis and cause.