Previous month:
July 2011
Next month:
September 2011

August 2011

Two Analytics Success Stories

Here are two success stories about analytics that initially appeared in the Winter 2011 issue of the MIT Sloan Management Review.

Helping students graduate in the Gwinnett County School District

The first one is related to education. It dates back from 2002, when Gwinnett County in the Greater Atlanta public school system decided to analyze the troves of data it had to improve high school graduation rates. Because data is currently used nationally to rate teachers and decide on tenure decisions, and also because people who understand analytics only superficially can generate analytical rules of dubious quality (see the linear regression with 32 decision variables to rate NY teachers I wrote about here), analytical approaches can have a bad reputation in the educational system.

But properly designed analytical studies can generate valuable insights too. The researchers found out that the single best predictor of high school completion was completion of Algebra I, which is a ninth-grade course. The issue was a lot of high-risk students had struggled in math courses before reaching Algebra I. But when they performed a follow-up analysis, the researchers realized that the single best predictor of Algebra I completion was... completion of creative writing in eighth grade. Creative writing. So the school district focused on helping students succeed at creative writing, and in turns that helped them succeed at algebra and finish high school, for reasons I'm not sure anyone truly understands. But it worked. In 2010 Gwinnett County School District receive the $1million Broad Prize for its work toward closing the achievement gap.

For those of you interested in learning more, this MIT SMR article is currently available for free view on the Forbes website.

Retaining customers at Assurant Solutions

The other analytics success story is about a company that sells credit insurance and debt protection products, and wants to improve its customer retention rate from its original 16% rate, which is said to be in line with the rest of the industry (meaning that when customers see the $10 or more they have to pay each month, 5 out of 6 change their mind and decide they don't need protection anymore).

The analytics team that was brought in did not consider current metrics such as average speed of answering phone calls, and instead focused on the end result: was the customer retained after the call (success) or not (failure)? What they found out is that, in the words of the author, "some customer service reps are extremely successful at dealing with certain types of customers. Matching each specific in-calling customer to a specific CSR made a difference." [I think that in-calling customers have to type in an identifiant or an account code so that the company is able to identify who they are and what premium they are paying right away.]

People don't really understand why some CSRs are more successful with more segments of their customer base than others - in particular customers with low premium vs customers with high premium - but understanding why was not important. In the past the customer center had routed the calls based on "expertise", but expertise, as the vice president of targeted solutions at the company explains, "is a subjective term" and is estimated based on anecdote. What was taken in consideration in the new study was the output: the success rate in convincing callers not to cancel. So the company developed an evidence-driven affinity-based routing that significantly improved its retention rate.

The affinity-based assignment was hampered by the fact that "the best matches are almost always not available... because that CSR was on the phone" which resulted in the creation of another complementary predictive model to estimate time to availability. This was driven by the fact that customers were willing to wait on the line a lot longer than the company had thought, balking at the 60-second mark rather than the 20- or 25-second mark.

The interviewee only makes a good point regarding why more people are not doing this: there is little appetite to adopt a new technology if the old system has been working fine. On the other hand, the program that IBM developed to help Assurant, called Real-Time Analytics Matching Platform, is now available to other companies. This made me happy to think that IBM hires so many of our IE graduates at Lehigh. Hopefully they get to play some role in the development of high-impact analytical solutions that help companies throughout the country in their chosen line of business - not only for the sake of improved customer experience, but also for the sake of the employees who benefit from working at a successful, prosperous company.

David Foster Wallace's Commencement Speech

A few weeks ago I discovered thanks to Elizabeth Collins the 2005 Commencement speech of writer David Foster Wallace at Kenyon College. The speech deserves mention not only because it is very good (more on that later), but also because it is one of the very few public speeches Foster Wallace gave before committing suicide in 2008.

The speech is, broadly speaking, on life, work and adulthood or, in the words of a writer for the New York Times, "the essential lonesomeness of adult life". It starts with a parable about an old fish and two young fish (who are so conditioned to be in water they don't realize what it is, which leads Foster Wallace to a definition of liberal education as learning how to think, i.e., recognizing the water around us).

I also liked his take on the "Capital T-Truth", the description of after-work grocery shopping (!) and his comment that "[t]here happen to be whole large parts of adult American life that nobody talks about in commencement speeches. One such part involves boredom, routine, and petty frustration."

I imagine these words have only gained accuracy in the 6 years since Foster Wallace made his speech, since more and more well-trained and eager college graduates find themselves under-utilized - their earning power possibly diminished for years to come - due to the economic situation. (I have taught many good students over the years, most of whom find jobs without too much difficulty, and even then I always wonder whether the companies that hire them will give them enough of a chance to display the range of their talents.)

Most importantly, the "conscious decision about how to think" provides graduates with some guidelines on how to deal with these feelings. The solution sounds trite, but as Foster Wallace explains, choosing to believe in alternate, kinder scenarios about the people you see at the store - the people who delay you at the store when you're tired and want to get home and eat and sleep before yet another day - is the first step in the never-ending quest to remain grounded and fight disillusionment in spite of the (occasional? repeated?) dreariness of everyday life. And hopefully every day becomes a good one once people have learned that the only thing that matters is the way they react to what happens to them, although Foster Wallace is not quite as positive.

In DFW's words: "The really important kind of freedom involves attention, and awareness, and discipline, and effort, and being able truly to care about other people and to sacrifice for them, over and over, in myriad petty little unsexy ways, every day."