What Makes a Good Measure?
December 18, 2011 | Leave a Comment
I travel a lot on United Airlines and since their merger with Continental, the new CEO, Jeff Smisek proudly states at the opening of the safety video that he and thousands of his colleagues are “creating the world’s leading airline.” Now, more recently, Etihad Airways has been advertising that they are building the worlds leading airline. What? Two leading airlines?? Now we have a fight on our hands!
But what does “leading” really mean? The first time I heard the phrase my immediate reaction was: Huh? That sounds terrible. Are they not going to strive to be the best airline? Aren’t they trying to be #1 like most would assume is the goal of a merger? But by what measure? Size? Revenue? Fleet age? Service and satisfaction? Destinations served? Complaints? Lost bags? Cost management?
For an industry with dozens of closely watched measures of performance, creating a public marketing message to be “leading” is vague and pointless to me. It’s also a bit risky. After all, vague or non-existent goals will always make you successful, but maybe not in the way you intended. It is a safe way to go, though, if you are not sure what you’re doing or how things might go. Maybe the new United will be able to say by the end of next year, “We’re leading with the worst ontime performance of any airline!”. That’s nice.
We’ve all heard the phrase “for good measure.” Hey, throw in some extra salt for good measure! Maybe you do it just in case what you’re cooking tastes terrible. It seems rather arbitrary. Why not taste it first? So, a more thoughtful approach to planning and success may be in order. I have worked with plenty of clients who do not understand how to make a good measure. Their five year plans are a wealth of vague, uncertain and impossible to measure goals, usually in an attempt to placate many differing views.
Think about this as you are setting goals for the new year for yourself or your organization. Are the measures meaningful? Can they really be measured? Is the necessary data collected? How will you know you have achieved the goal? Is it actually a good measure people will recognize as success?
After all, if you are ”leading” the airline industry that still has a terrible overall reputation for service, you haven’t really set a good measure or accomplished much.
Good from Bad
December 9, 2011 | Leave a Comment
It’s become particularly evident to me in all kinds of recent experiences, whether at work or with family, that when something bad or unpleasant happens, there is a good side. Now, you might think my analytical mind has gone completely awry, but let me give a few examples.
- I prepared a project proposal that was not accepted by the prospective client. Unexpectedly, another nearly identical request came along resulting in very little work to respond quickly. The first round was not wasted effort!
- I’ve beenworking with a client to resolve performance issues with their BI platform. The visibility of the project has imposed a lot of pressure on the team to make signficant improvements. Progress has been frustratingly slow. Yet, I am learning an incredible amount about the more detailed workings of this platform and how similar configuration improvements can be made elsewhere to benefit other clients.
- A coworker and I have had tense disagreement about the strategy for a proposed solution. We usually get on quite well. But the frustration, arguments, and counter arguments helped lead to a breakthrough “aha” moment as to how to proceed.
What are some examples you’ve experienced? Whenever something bad happens, look a little deeper… wait a little bit… look at another angle. There will very likely be something good to come of it.
Does placement testing predict college course success?
October 20, 2011 | Leave a Comment
Working primarily in the higher education world, I find this article from Washington Monthly to be quite an eye-opener about the current state of placement testing and developmental education in the community college world. Some interesting statistics are presented, much of which comes from the Community College Research Center. Yet, there isn’t any explanation about how the information was gathered and the methodology used to arrive at some rather interesting and controversial results. I am predisposed to ask detailed questions about “what and how” so I can’t help feeling a little skeptical. I want to investigate further to see for myself what the data might say. It’s a bit like the accomplished musician who can’t stand listening to someone else’s performance because they over analyze it and pick out the faults.
Many of the clients we work with are asking the same kinds of questions proposed in the article. These questions are a logical extension of the research results. They want better insight into student behavior and achievement and what administrators can do to increase student success:
- Are students who place in a developmental Math or English course taking that course? Do they succeed in that course and then succeed in the college level course?
- Are students ignoring the developmental course requirement and do they succeed at any different rate than those who pass placement tests? (The article suggests students are quite good at self-selecting into the college level course and ignoring the test results!)
- What are the characteristics of students who do not succeed at developmental or college level courses?
- Are there specific courses or pathways that are more successful than others based on historical activity?
Of course, all of this really gets to the efficacy of the entire placement and developmental education system. Some states, with the help of research foundations, are attempting to address significant problems in misaligned curriculum between the developmental course and the college level course that should follow. Similarly, they are also applying new research in learning to redesign curriculum for greater real world application and effectiveness. The Developmental Education Initiative is one example.
Our job as BI consultants is to effectively understand the business questions and help institutions with the collection of useful data that can be presented in a way that informs the change process. We have several client engagements currently underway that are doing just that. I’ll be able to see for myself from exposure to specific institutions what the state of developmental education success is for their students.
BI and the NHL Playoffs
May 25, 2011 | Leave a Comment
I find that living in the DC area, one doesn’t find a whole lot of hockey fans in your everyday interactions. Most people are into baseball, football, and even soccer. But hockey – not so much — not nearly as much as in my hometown. Living here for almost 20 years now, I’ve become a Caps fan. This season was promising and the begining of the post-season even more hopeful of a run at the Stanley Cup. We know how that ended, but more on that in a moment.
When a Canadian friend of mine who happens to work for SAP Business Objects, forwarded this link to me that showcases their BI platform, I was ingtrigued. It takes full advantage of their analytics and data exploration technologies using hockey statistics. I mostly deal with higher education related data like student enrollment, retention, financial aid, and human resources. This was different and fun!
I took a look at how Washington stacked up against their second round rival Tampa Bay. Hmm…. Not such a good picture. Tampa Bay had higher average Goals For and lower Goals Against. Their offense and defense looked better by the numbers. I looked at the goalie save percentages. I compared some key individual players from each team. Everyone thought the Caps would keep winning and go to the finals. After exploring some of the data and visualizations, I wasn’t so sure of a spot in the finals. And, in fact, it didn’t happen. Sadly, the numbers seemed to support that outcome. Certainly there is more to hockey than just numbers. Passion for play, pure skill, wanting to win, and luck sometimes create amazing upsets. That’s what happened in last year’s post-season. (And seemingly in every year’s March Madness for all of you college basketball fans!)
Of course statistics don’t always tell the whole story. Lots of other variables can come into play. And often good analysis includes domain knowledge with the human element to enhance any interpretation. But I can’t help thinking that those stats didn’t lie, and the results certainly bear that out. Now with Vancouver in the Cup finals and Tampa Bay winning tonight to force a game 6, It’s time to go back and do a bit more research and exploration!
Take a look at the site. Play around. Even if you don’t know much about hockey, it’s a good way to become familiar with some of the great analysis and visualization tools available in Business Objects. Maybe you can improve your chances of winning the office Stanley Cup pool!
Are You Telling Stories With Your Data?
January 20, 2011 | Leave a Comment
What a great quote from Dr. Brene Brown at the University of Houston. You can see her TED Talk here for more about how she elevates the use of data to make change at her institution.
(Via Brand Autopsy)
And then there was Ownership
December 7, 2010 | Leave a Comment
The change was dramatic and jarring. I couldn’t help but notice it immediately. Up until this point in every project conference call or client visit we would hear comments from one particular key stakeholder like: “the reports YOU built” or “YOUR reporting tool” or “YOUR data”. Each time I would cringe because the only way a BI initiative can be successful is if there is full engagement and ownership by those who will ultimately be the users and maintainers of the system. Occasionally, I would remind the team that it is THEIR system and THEIR data and we are only here to help and facilitate them reaching the long term goals in their BI Strategy.
And then it happened. We all were working together, as client and consultant team, to address the punch-list of items that needed to be fixed before “turning on the switch” and going live to the broader set of business users. I heard it. The language from this person changed. The ownership suddenly appeared. ”I have to research this data anomaly” and “We have to finish the user cheat sheets and handbook” and “I have to test this to be sure the security setup is working correctly.”
To be honest, I am not entirely sure what triggered the change. Maybe it was the deadline of this new reporting environment being put out for all to use and either accept or reject. And, naturally, someone deeply involved in that kind of project and commitment ultimately wants their positive imprint on a successful outcome. I know I sure did. I suppose it is like the bird leaving the nest, or the child going off to school on their own for the first time, or maybe a teenager taking the car out for a drive by themselves after getting their full license. Taking ownership or responsibility is perhaps a natural part of the human process of becoming more independent.
I am gratified to see this change andexperience the benefits of the client ownership in their evolving BI environment. Although the details of this true story may seem to be unique and even readily identified by those involved, this happens in every engagement. It is not unusual, nor a cause for concern–unless, of course, the ownership never materializes .
ASR and Howard Community College Present Keys to a Successful BI Project at SEDUG
November 23, 2010 | Leave a Comment
Last week ASR presented with Howard Community College to an audience of IT professionals and end-user/decision-makers at the Southeast Datatel User Group. We shared best practices and lessons learned for implementing business intelligence using the Business Objects platform and Datatel’s DataOrchestrator ODS. The presentation included a demonstration of several focused areas of analysis, including Course Success, Grade Distribution, and Student Retention. This invited questions from several in the audience who were under the impression that the Business Objects platform was intended solely for operational reporting. This could not be further from the truth. While I will go into more detail about how to make the operational data store from Datatel into a strategic data warehouse in a subsequent post, suffice to say, that it can be done. In fact, it may be necessary to obtain buy-in from end-users and give them a leg up on their reporting and analytic journey. More on this in the future. For now, please feel free to download the presentation and contact us if you have questions.
Keys to a Successful BI Implementation with Howard CC
Ready to Roll
October 16, 2010 | Leave a Comment
In the model train hobby I enjoy during my spare time, this means “take it out of the package, put it on the tracks and it is ready to go!” There’s no assembly, painting, adjusting, setup, muss or fuss. Unfortunately, getting a business intelligence solution ready for production isn’t quite so easy.
Lately, I’ve been working on a solution delivery scheduled for the end of this month. Here are a few key things we’ve been tracking with this client to ensure a smooth and successful rollout. Some may seem obvious, but it is easy to forget little details:
- Get the complete list of users who need access. Along with this list, it is important to have planned their access and security credentials. When considering groups and who should have access to what, make sure to assess both the data they should see, as well as the functionality they should be allowed to use in the BI platform.
- Test using actual user credentials setup as defined in the access plan. Everything can look like it is working great when logged in as the administrator or developer, but as we found out, simple things can be overlooked like the ability to change predefined queries or overwrite a standard report.
- Ensure there is adequate licensing. Understand the type and concurrency of the license scheme you have. Often the first couple of days will see a lot of activity from the curious users. If they have trouble logging in because of limited licensing, you may have a hard time coaxing them back to the BI system again.
- Prepare a quick start or users’s guide. As much as you might think “it’s all intuitive”, for many users new to BI, it’s not so obvious. List explicity the steps they need to take and use screen shots of the actual system with actual data so they know they are doing things correctly. There can be a lot of useful tips and tricks for navigating around reports and analyses they’ll appreciate too. It helps build confidence and buy-in to using the system.
- Validate the reports and analytics one last time. What worked a week ago or a month ago might not necessarily work today. Like the security and access plan, the report design can sometimes be affected by little things like a changover in a month or business cycle. A back end data source may not be updated as it should have been and no alert was in place to raise the flag. New data may not fit the original formatting if boundary values weren’t well understood. There are many possibilities to keep an eye on. While it is unlikley all issues will be caught, one last review is a good idea.
I am looking forward to the end of the month to see how well things go. Things should be ready to roll!
Dashboards in Minutes?
September 23, 2010 | Leave a Comment
Previously, I posted on a couple of higher education vendors touting their “out of the box” dashboards. Here’s another example of messaging that should make every BI project lead or CIO cringe. “Data to Dashboards in Minutes” is the byline for the Pentaho Agile BI Tour that started recently. It sounds appealing: a half day workshop to help you “deliver successful BI solutions to users more quickly and at significantly lower cost.”
But what part of the dashboard deployment process are they talking about? I doubt the half day seminar with hands on demo includes the challenging, up front, and most important part of the process. That is, to answer questions surrounding data governance, ownership, quality and integration. I also suspect it sidesteps the contentious debate that occurs during most dashboard design and deployment efforts on what metrics, goals and visual presentation should be made for the various user personas who will access the them. In my experience, every user wants their own view and metric definitions of what matters for their job responsibilities. Dashboards are notoriously difficult to create with the right level of detail and interactivity, except at the very highest levels of summary. And, those are often the least useful for real decision making.
So, I am curious as to how comprehensive the Agile BI Methodology is with respect to the people and process side of the BI equation. Ignoring this key aspect and touting ”dashboards in minutes” or “out of the box” really does a disservice to everyone involved–the end user, IT, professional services, and BI vendor communities because it sets a false expectation of what is possible. It doesn’t tell the whole story of what resources are required to get the organization where it wants to go. That leaves BI professional service companies like ASR having to do more education on the realities that go beyond just the BI technologies. The limiting factor is often the organization’s own ability to deal with the internal transformations needed to understand and manage with data.
That is the big nut to crack.
Leading for Success with your Business Intelligence Initiative: Part 4
September 3, 2010 | Leave a Comment
In my previous post Leading for Success with your Business Intelligence Initiative: Part 3, I discussed the value of increasing your understanding of the BI ecosystem and how it can enable the use of self-organizing teams. The next key to improving your BI success is maintaining the commitment of key constituents, improving your institutional communication around the BI initiative and setting the stage for your BI development teams’ success.
The C’s in Success
Communication
In my experience, there’s usually a poor communication strategy behind any really good idea that doesn’t quite get off the ground. Too often BI leaders tend to focus too long on the larger business case (ROI, institutional benefits, etc) and fizzle out on building the personal commitment, the “hearts and minds” part. What is required is continuous and focused communication with key constituents before, during and after the launch. In my experience, the success of an institutional wide BI initiative is as dependent on your political success as it is on your technical success.
To keep the institutional BI visibility high and retain the commitment of the key constituents take a cue from the masters: politicians. Political campaign platforms are based on the three C principles: (1) Crisp and Clear; (2) Context Centric; and (3) Consistent and Consistently. Every stump speech, every sound bite, every public conversation and every written message needs to be rigorously “on message,” All the BI sponsors and members of the BI development team need to follow the three C’s principles.
1. Crisp and Clear
How do you describe the BI initiative and what value will the BI initiative have for the person you are talking to? Let’s go back to that tried-and-true technique–the elevator pitch. Can you clearly describe the goal/value of the BI initiative in 30 seconds or less? When you talk to someone about the BI initiative does your description hold that person’s attention? Or do their eyes glaze over or wander across the room?
Being crisp is about informing people about the value, what you plan to accomplish for them, in as few words as possible, and using that same crisp message in written materials.
Take this Test
Find a friend who is the least likely to understand your BI initiative, and test your “message crispness” on them. Tell them what you are doing with BI in two or three sentences. Avoid industry jargon and technical terms that only people in the BI field will understand. Then ask them to repeat what they think you are doing back to you. If they don’t come back with the right answer, the message isn’t crisp and clear.
2. Context Centric for each Stakeholder type
You need to communicate the role they play and the value proposition to each Stakeholder type (executives, BI developers, end-users, etc). You need to inform stakeholders about what you are doing, why you are doing it, their role in the BI initiative and the value it has for them. People need convincing as to why they should spend their time and limited resources with you. Your story should focus on how they benefit from the BI initiative (communicated from a “what’s in it for them?” perspective).
3. Consistent and Consistently
Once you’ve nailed down your crisp message, and you’re telling your story from your stakeholder’s perspective, make sure you tell it consistently in your conversations, e-mails, in print materials, via etc.
Nothing is more disconcerting to stakeholders than hearing one story from one communications channel or individual and then hearing or reading a different version of the story from someone else or someplace else. They don’t know which version to believe. Reestablish who you are and what you are doing with every stakeholder interaction. Reinforce your story as often as possible.
In the next post we’ll explore some additional C’s to you BI development teams’ success.