The pitfalls of big data

As you may know, I love a good way of tying sports into examples and in this case, Steve Hirdt examines how sports can be related to big data. Makes for an interesting way to explore the pitfalls of big data and about remembering that all data can really give us is a best guess scenario, we have to use our brains to get all the surrounding facts from there.

When college officials talk about using “Big Data” to improve higher education—the focus of a SUNY conference here this week—they often draw an analogy to Moneyball. The movie recounts how Billy Beane, the Oakland A’s general manager, revived his ailing baseball team by analyzing data in new ways.

So what might sports teach higher education about data mining? In academe the stakes are higher than in baseball, but progress toward making good use of data has been uneven. Nonetheless, colleges are busy mining students’ data trails to build software that does things like suggest what mathematics problems they should work on or even what classes they should take. Continue…