When does predictive analytics go too far?
March 20, 2013 | Leave a Comment
This recent news item about Target using analytics to target (pun intended) promotions to newly pregnant mothers and the controversy surrounding it illustrates a profound dilemma. How should data be used for predictive purposes? The privacy issues, loss of control on the sharing of personal information, much less the risks of unexpected consequences, raise serious questions for those of us in the industry who develop the models to answer business questions of apparent importance to an organization.
Target’s business question seems innocent enough–determine as quickly as possible those customers who are likely to be pregnant and interested in certain products and promotions to capture their purchase and loyalty before losing it to the competition. But at what cost? In the case of the father who found out his teen daughter might be pregnant because of coupons sent to the home before she shared the information with the family, Target is facing more than just a public relations challenge. A false positive for this family might have created a bit of an awkward firestorm at home. In this case, the correct prediction did more than create a firestorm, it changed their lives and took the choice and control away from their customer. Is that really the desired result?
What should we be asking? What is the appropriate use of source data? What are the possible implications of accurate predictions and false positives? False positives in a predictive model to identify fraudulent tax refunds might only embarrass the taxpayer or delay the processing while scruitinized in a deeper review. There may be no lasting damage other than a frustrated taxpayer. Furthermore, correct predictions may have no negative consequences for the tax agency, but appropriately negative consequences for the perpetrator of refund fraud. Determining which students may be likely to drop out of university, accept an offer of admission to a program, or be delinquent in tuition payments also seem relatively innocuous.
What do we ever really know about what organizations might be doing with information collected about us? Very little. Should this level of use be disclosed and required in privacy notices? Should it depend on the type of use? Recently when my mortgage was sold to another servicer, I received a privacy disclosure that made it very clear that I had no rights or choice on how the bank used my personal and loan data for internal purposes. The notice pointed out this was legal under federal law. I only could indicate my preferences for how data was used with affiliates and how it was shared outside the bank. That still leaves the problem of how they might use personal data internally for their own predictive modeling that I may find inappropriate.
As BI professionals we should consider more than just the technical accuracy of a predictive model and the selected target variable. Is would also seem appropriate to consider privacy, potential consequences, and whether the end customer has a choice in saying how that data is to be used or not for decision making purposes. Perhaps the Golden Rule would be a prudent test.
Moonlight and Other Correlated Factors
November 5, 2012 | Leave a Comment
Today’s Financial Times ”Weekly Review of the Fund Management Industry” has an interesting front page article that really struck me. The article describes how a firm which specializes in longevity research for pension funds recently discovered a spike in death rates when more than half of the moon is visible in the night sky.
My immediate reaction, and one that I think is relevant for any of us in the field of research and predictive modeling, is “Who even thought of data on the moon phases as an input variable to this research??” Some think predictive modeling is an automatic magical black box exercise. But it really is just math and depends on the capacity to throw the net wide, so to speak, across a range of seemingly completely unrelated data to see if patterns emerge.
Now, does knowing that there is a higher death rate at certain points in the lunar cycle help with predicting longevity? The article suggests not, but it does help with predicting payout patterns, which is of concern to pensions as well.
Now, let’s see… what kinds of things might cause students to drop out? Donors to increase giving? Students to default on Financial Aid payments?? Maybe that crazy full moon has something to do with it! And now I have the King Harvest “Dancing in the Moonlight” song stuck in my head…
BI and the NHL Playoffs
May 25, 2011 | Leave a Comment
I find that living in the DC area, one doesn’t find a whole lot of hockey fans in your everyday interactions. Most people are into baseball, football, and even soccer. But hockey – not so much — not nearly as much as in my hometown. Living here for almost 20 years now, I’ve become a Caps fan. This season was promising and the begining of the post-season even more hopeful of a run at the Stanley Cup. We know how that ended, but more on that in a moment.
When a Canadian friend of mine who happens to work for SAP Business Objects, forwarded this link to me that showcases their BI platform, I was ingtrigued. It takes full advantage of their analytics and data exploration technologies using hockey statistics. I mostly deal with higher education related data like student enrollment, retention, financial aid, and human resources. This was different and fun!
I took a look at how Washington stacked up against their second round rival Tampa Bay. Hmm…. Not such a good picture. Tampa Bay had higher average Goals For and lower Goals Against. Their offense and defense looked better by the numbers. I looked at the goalie save percentages. I compared some key individual players from each team. Everyone thought the Caps would keep winning and go to the finals. After exploring some of the data and visualizations, I wasn’t so sure of a spot in the finals. And, in fact, it didn’t happen. Sadly, the numbers seemed to support that outcome. Certainly there is more to hockey than just numbers. Passion for play, pure skill, wanting to win, and luck sometimes create amazing upsets. That’s what happened in last year’s post-season. (And seemingly in every year’s March Madness for all of you college basketball fans!)
Of course statistics don’t always tell the whole story. Lots of other variables can come into play. And often good analysis includes domain knowledge with the human element to enhance any interpretation. But I can’t help thinking that those stats didn’t lie, and the results certainly bear that out. Now with Vancouver in the Cup finals and Tampa Bay winning tonight to force a game 6, It’s time to go back and do a bit more research and exploration!
Take a look at the site. Play around. Even if you don’t know much about hockey, it’s a good way to become familiar with some of the great analysis and visualization tools available in Business Objects. Maybe you can improve your chances of winning the office Stanley Cup pool!
What’s the difference between BI and Analytics?
June 22, 2010 | Leave a Comment
Perhaps there is no other industry with more buzzwords than the Business Intelligence (BI) “industry.” As a result, there are frequently semantic arguments over what is meant by specific terminology employed by the tool vendors, industry analysts, and consulting firms in the business. The most recent semantic battle pits the term or phrase “BI” against the term “Analytics.” With Analytics in our name, I thought it wise to weigh-in on this minor industry dust-up.
Ultimately I agree with Boris Evelson at Forrester, analytics is essentially a subset of business intelligence:
I think the effort of trying to differentiate analytics from BI is a vendor-invented hype, since many BI vendors are running out of ways to differentiate themselves… I also disagree with the “old BI = bad”, “new analytics = good” premise that I see in many analysts’ papers. You and I know that you can’t build analytics (OLAP, advanced analytics, etc.) without basic ETL, DW, MDM, etc. So nothing’s really changed as far as I am concerned: we are still fighting the same battles – silos, data quality, etc.
Fundamentally, BI refers to all methods that use data to help decision-makers and end-users gain a greater insight to their business and make better decisions. Advanced analytics (e.g. OLAP, Predictive Modeling, etc.) play a significant role in this capacity, but are only a part of the overall approach.
User-Friendly Predictive Analytics Recruitment Solution Unveiled at NACAC Conference
September 23, 2009 | Leave a Comment
Baltimore, MD September 23, 2009 — While attending the National Association for College Admission Counselors (NACAC) 65th Annual Conference, ASR unveiled its new Predictive Analytics Solution for Recruitment and Admission. The new solution is designed to put the power of advanced analytics directly into the hands of enrollment management professionals for better evidence-based decision making. ASR’s Recruitment Analytic Models leverage institution specific data to estimate statistically valid forecasts of future enrollment, net tuition revenue, and even retention rates. These models provide enrollment managers with evidence based predictions for shaping the incoming class with the ‘ideal’ students for their institution.
ASR’s solution is different. Admissions professionals will be able to interact with the models to build various enrollment scenarios and change the model’s assumptions. This helps them understand the inevitable trade-offs that can happen when they simulate various policy ‘levers.’ ASR’s solution focuses on making these models accessible to non-technical admissions professionals. Most of the solutions on the market require an IT professional to extract data in a specific file format to provide to a third party that estimates an analytic model. The institution receives a static report to guide planning decisions, but it doesn’t let them simulate a variety of scenarios.
ASR’s new Predictive Analytics Solution for Recruitment and Admission will help the institution develop its recruitment strategy and at the same time enable better day-to-day tactical decision making. The solution will help institutions to:
- Identify causal factors for enrollment
- Analyze a prospect pool for more effective list purchases
- Simulate a multitude of enrollment scenarios
- Forecast enrollment on a daily basis throughout the recruitment lifecycle
- Perform decision impact analysis and assessment
There were three main goals in development of the framework:
- Provide a user-friendly way for busy enrollment management professionals to interact with predictive models to aid in institutional planning.
- Produce a solution that works with existing tools and technology already in use at the institution.
- Eliminate the need for clients to pay new recurring software license fees.
The secret to successfully meeting these goals lies in ASR’s ability to develop analytic solutions that help institutions integrate their people, process, and technology. “We think it’s critically important that advanced analytics are put directly into the hands of those that do the planning and make the decisions” said, Dr. Peter Arena, ASR’s founding principal and chief statistician for higher education. “Using simple, point-and-click interfaces – enrollment professionals can bring data and information to life. The result is a rich user experience that makes it easier to visualize data, simulate decisions before they are made, and ultimately optimize recruiting.”
To get people started with predictive analytics, ASR is offering a special NACAC conference rate for 50% off the company’s Prospect Scoring Report service. Attendees can visit the ASR booth for more details and to see a demonstration of the full solution.
To learn more about ASR’s solutions for higher education visit:
ASR’s Predictive Analytics Solution for Recruitment and Admissions
ASR’s solutions for Higher Education
ASR’s business intelligence blog
About ASR Analytics, LLC
ASR Analytics LLC (ASR) provides high-end business intelligence and analytic consulting services to clients in higher education. ASR aims to provide institutional decision makers with self-service decision support tools to help them be more effective in their recruitment, retention, and accountability initiatives. To learn more about our solutions visit: http://www.asranalytics.com/
Recruiting with Predictive Analytics in Uncertain Times
September 22, 2009 | Leave a Comment
It’s late September and the new academic year has settled upon us. Admission offices across the land are already busy recruiting the next set of classes, while at the same time conducting postmortems on the good and the bad of the most recent campaign. Some institutions have found the 2009 recruitment cycle to be quite unpredictable.
Recently, the Providence Journal ran a story about the difficulty some Rhode Island institutions had enrolling and managing the inflow of new students for the fall semester. This, despite the fact that nationally the number of graduating high school students remains at nearly record highs (although down slightly from the previous 2 or 3 record years). What was driving the instability? The economy, of course.
Private institutions in Rhode Island, according to the article, seem to have been hit the hardest and many were short of their goals. What struck me, however, were the tactics that were used by admission officers to stave off the potential declines. Seemingly, institutions broadly applied a combination of two tactics; either, 1) increase the amount of institutional gift aid to effectively discount the tuition, or, 2) lower admission standards to increase the eligible pool of applicants. Sometimes they did both.
While I may be over simplifying the true remedies attempted by these institutions, I can tell you from experience that the typical responses to shifts in the marketplace for many admission officers tend to be linear and broadly applied:
- When the demographic pool shrinks – buy more search names.
- If times are tough economically – increase the discount.
- When the numbers don’t come through as expected and the cause is not clear – lower admission standards.
The problem with these approaches are that they often conflict with other strategic goals of the institution (e.g. quality, diversity, etc.).
Enter predictive analytics. Although this may seem contradictory to the premise that 2009 was an unpredictable year, statistical modeling techniques are not only warranted in times like these, but even more necessary than during ‘normal’ recruitment cycles. You see, predictive modeling does more than forecast aggregate numbers. The idea is really to understand the key drivers of enrollment using mathematical models. Once you truly understand who and why students enroll at your institution, you can better segment or target the audience and more effectively recruit the students that are more likely to enroll.
As a result, you will be able to address the problems I’ve listed above with more surgical precision. So instead of giving a blanket discount to all accepted students, you will be able to identify the students that will benefit the most from the award through the analysis of a combination of driving factors.
“What-if scenario” decision support applications make it possible to interact with predictive models so that you can “war-game” various scenarios. So, for example, you could make the output of your model more conservative in an uncertain recruitment cycle by applying limits or weighing certain variables (like household income) more heavily in your enrollment propensity models.
The lesson is that there is almost always a way to use data to your advantage, even in turbulent times. Broadly applying tactics across all of the prospective student body should be a last resort.
Press Release: ASR Analytics Puts Predictive Analytics Directly into the Hands of Enrollment Managers
September 29, 2008 | Leave a Comment
Seattle, WA, September 25, 2008: While attending the National Association for College Admission Counselors (NACAC) 64th Annual Conference, ASR unveiled its new Predictive Analytics Solution for Recruitment and Admission. The new solution is designed to put the power of advanced analytics directly into the hands of enrollment management professionals for better evidence-based decision making. ASR’s Recruitment Analytic Models leverage institution specific data to estimate statistically valid forecasts of future enrollment, net tuition revenue, and even retention rates. These models provide enrollment managers with evidence based predictions for shaping the incoming class with the ‘ideal’ students for their institution.
ASR’s solution is different. Admissions professionals will be able to interact with the models to build various enrollment scenarios and change the model’s assumptions. This helps them understand the inevitable trade-offs that can happen when they simulate various policy ‘levers.’ ASR’s solution focuses on making these models accessible to non-technical admissions professionals. Most of the solutions on the market require an IT professional to extract data in a specific file format to provide to a third party that estimates an analytic model. The institution receives a static report to guide planning decisions, but it doesn’t let them simulate a variety of scenarios.
ASR’s new Predictive Analytics Solution for Recruitment and Admission will help the institution develop its recruitment strategy and at the same time enable better day-to-day tactical decision making. The solution will help institutions to:
- Identify causal factors for enrollment
- Analyze a prospect pool for more effective list purchases
- Simulate a multitude of enrollment scenarios
- Forecast enrollment on a daily basis throughout the recruitment lifecycle
- Perform decision impact analysis and assessment
There were three main goals in development of the framework:
- Provide a user-friendly way for busy enrollment management professionals to interact with predictive models to aid in institutional planning.
- Produce a solution that works with existing tools and technology already in use at the institution.
- Eliminate the need for clients to pay new recurring software license fees.
The secret to successfully meeting these goals lies in ASR’s ability to develop analytic solutions that help institutions integrate their people, process, and technology. “We think it’s critically important that advanced analytics are put directly into the hands of those that do the planning and make the decisions” said, Dr. Peter Arena, ASR’s founding principal and chief statistician for higher education. “Using simple, point-and-click interfaces – enrollment professionals can bring data and information to life. The result is a rich user experience that makes it easier to visualize data, simulate decisions before they are made, and ultimately optimize recruiting.”
To learn more about ASR’s solutions for higher education visit:
ASR’s Predictive Analytics Solution for Recruitment and Admissions: http://www.asranalytics.com/solutions/education/recruitment-analytics/
ASR’s solutions for Higher Education: http://www.asranalytics.com/solutions/education/
ASR’s business intelligence blog: http://www.asranalytics.com/category/blog/
About ASR Analytics, LLC
ASR Analytics LLC (ASR) provides high-end business intelligence and analytic consulting services to clients in higher education. ASR aims to provide institutional decision makers with self-service decision support tools to help them be more effective in their recruitment, retention, and accountability initiatives. To learn more about our solutions visit: http://www.asranalytics.com/
The art and science of ‘Elevatoring’
April 16, 2008 | Leave a Comment
Elevators go up and they go down. Not very interesting are they?
Actually, there are some changes afoot in the elevator industry that are quite fascinating such as the proliferation of gigantic skyscrapers being built across the globe. But, aside from this video of a man trapped in an elevator for 41 hours, what caught my eye in this article in The New Yorker was the discussion of analytics being applied by elevator consultants, a practice known as ‘Elevatoring.’
Elevatoring practitioners are employed by architects to determine the correct number, size, speed, and layout of elevators in a new building. Get it wrong and the building is doomed. Elevatorists (is that a word?) must apply predictive analytic techniques to get the design just right. How a building will be used is important, but so is cultural nuance. All kinds of variables must be considered. For example, people get very upset if they have to wait more than 20 seconds for an elevator in an office building, while they will tolerate 30 or 40 seconds in a hotel or apartment building.
Here are a few more interesting facts:
- Probable stop rule of thumb: 10 people in an elevator serving 10 floors will make 6.5 stops. 10 people in an elevator serving 30 floors; 9.5 stops.
- There should be enough elevators operating efficiently enough to move 13% of the occupants of a building within 5 minutes.
- Standard elevator measure is about 2 square feet per person.
- People in Asia will tolerate less personal space than people in the U.S. and willingly cram onto elevators at much greater density rates.
Anyway, if you can find a few minutes to read this quite lengthy article I guarantee you will never look at elevators in the same way again.
(Found via Kottke)