There was a nice blog postby Matt Reed on Inside Higher Ed a few weeks back that highlighted a continuing challenge on many campuses: mistrust of data. After reading the piece, I found myself reflecting on behaviors I have seen over the years that can quickly destroy a data informed culture.
- The surprise witness: Have you ever been in a meeting where someone presented data from an unknown or unfamiliar source? Usually they have extracted the data from a number of different systems and then performed some kind of black magic in Excel or an esoteric statistics program to tie the information together to prove their particular point. This approach will at best cause some people to shut down completely. However, a more likely outcome will be a meeting transformed into an inquisition of the data rather than a discussion about solutions to a known problem.
- The secret society or if you build it, they will come: The genesis of this behavior generally occurs with a data warehouse or BI initiative. The budget is tight and in the interest of time a small subset of individuals come together and decide that they are the data experts for the institution. Most IT consultants love this approach because they get quick answers to their questions and then they can go off and quickly code a solution and generate a lot of cool looking reports and charts. Unfortunately, when other stakeholders are finally included, they immediately spot flaws in the business definitions for their particular department and they distrust the data - and the technology. Many of the data models are flawed and need to be re-worked and there is usually little money left in the budget to resuscitate the project.
- Perfection: Matt touches on this in his post and we have discussed this in previous blogs as well. Data doesn't need to be perfect unless you are talking about engineering projects. Even accountants allow for immateriality, when an amount is so small that it would not mislead an investor. So if your enrollment of 10,000 is off by 10 students (a margin of error of .1%), are you telling methat you can't use that data to help make some directional enrollment management decisions and that it can't be trusted? Right or wrong, hyper-critiquing the accuracy of data is often used as a defensive technique against surprise witness or secret society behaviors.
Ultimately, transparency and an open dialog about institutional data and the business processes that produce it must be a first step in developing a data informed culture. The "Data Day" at the community college in Michigan is a great example of how to build trust in data and a sense of shared governance. Without this shared view, we are like the blind men and the elephant: forever arguing about our narrow view of the world and never solving the larger problems of the institution.
What behaviors have you seen destroy a data informed culture? Better yet, what strategies and techniques have you successfully used to foster a data informed culture? We would be interested in hearing more about what you are doing so post a comment below. Or, if you are interested in learning more about how ASR Analytics approaches BI from a people, process technology perspective, feel free to contact us.